Test Statistics

Total Statistics
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
All Tests
835000:01:13
Statistics by Tag
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
1node
835000:01:13
critical
835000:01:13
netconf
835000:01:13
Statistics by Suite
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
netconf-clustering-scale.txt
835000:01:22

Test Execution Log

00:01:21.705 SUITE netconf-clustering-scale.txt
Full Name: netconf-clustering-scale.txt
Documentation:

netconf-restperfclient update performance test suite (clustered setup).

Copyright (c) 2016 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

Perform given count of update operations on device data mounted onto a netconf connector (using the netconf-testtool-restperfclient tool) and see how much time it took. More exactly, it sends the data to a restconf mountpoint of the netconf connector belonging to the device, which turns out to turn the first request sent to a "create" request and the remaining requests to "update" requests (due to how the testtool device behavior is implemented).

The difference from the "single node" test suite (see ../scale/performance.robot) is that the device is configured and the data on it created using one node in the cluster and the update operations are issued on a different node. This forces the cluster nodes to communicate with each other about the data to be sent to the device.
Source: /w/workspace/netconf-csit-3node-clustering-scale-only-scandium/test/csit/suites/netconf/clusteringscale/performance.robot
Start / End / Elapsed: 20250127 00:21:50.372 / 20250127 00:23:12.077 / 00:01:21.705
Status: 8 tests total, 3 passed, 5 failed, 0 skipped
00:00:07.746 SETUP Setup_Everything  
Documentation:

Setup everything needed for the test cases.
Start / End / Elapsed: 20250127 00:21:50.912 / 20250127 00:21:58.658 / 00:00:07.746
00:00:00.349 TEARDOWN Teardown_Everything  
Documentation:

Teardown the test infrastructure, perform cleanup and release all resources.
Start / End / Elapsed: 20250127 00:23:11.727 / 20250127 00:23:12.076 / 00:00:00.349
00:00:07.065 TEST Start_Testtool
Full Name: netconf-clustering-scale.txt.Start_Testtool
Documentation:

Deploy and start test tool, then wait for all its devices to become online.
Tags: 1node, critical, netconf
Start / End / Elapsed: 20250127 00:21:58.658 / 20250127 00:22:05.723 / 00:00:07.065
Status: PASS
00:00:02.410 TEST Configure_Device_On_Netconf
Full Name: netconf-clustering-scale.txt.Configure_Device_On_Netconf
Documentation:

Configure the testtool device on Netconf connector, using node 1.
Tags: 1node, critical, netconf
Start / End / Elapsed: 20250127 00:22:05.723 / 20250127 00:22:08.133 / 00:00:02.410
Status: PASS
00:00:02.595 TEST Wait_For_Device_To_Become_Connected
Full Name: netconf-clustering-scale.txt.Wait_For_Device_To_Become_Connected
Documentation:

Wait until the device becomes available through Netconf on node 1.
Tags: 1node, critical, netconf
Start / End / Elapsed: 20250127 00:22:08.133 / 20250127 00:22:10.728 / 00:00:02.595
Status: PASS
00:01:00.690 TEST Wait_For_Device_Data_To_Be_Seen
Full Name: netconf-clustering-scale.txt.Wait_For_Device_Data_To_Be_Seen
Documentation:

Wait until the device data show up at node 2.
Tags: 1node, critical, netconf
Start / End / Elapsed: 20250127 00:22:10.729 / 20250127 00:23:11.419 / 00:01:00.690
Status: FAIL
Message: ... click for list of related bugs or create a new one if needed (with the "netconf_clustering_scale_txt_wait_for_device_data_to_be_seen" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=netconf_clustering_scale_txt_wait_for_device_data_to_be_seen&order=bug_status" Keyword 'Check_Data_Present' failed after retrying for 1 minute. The last error was: HTTPError: 503 Server Error: Service Unavailable for url: http://10.30.170.220:8181/rests/data/network-topology:network-topology/topology=topology-netconf/node=17830-sim-device/yang-ext:mount?content=config
00:00:00.276 SETUP SetupUtils . Setup_Test_With_Logging_And_Fast_Failing  
Documentation:

Test case setup which skips on previous failure. If not, logs test case name to Karaf log. Recommended to be used as the default test case setup.
Start / End / Elapsed: 20250127 00:22:10.730 / 20250127 00:22:11.006 / 00:00:00.276
00:01:00.399 KEYWORD BuiltIn . Wait Until Keyword Succeeds   ${DEVICE_DATA_CONNECT_TIMEOUT} 1s Check_Data_Present
Documentation:

Runs the specified keyword and retries if it fails.
Start / End / Elapsed: 20250127 00:22:11.006 / 20250127 00:23:11.405 / 00:01:00.399
Message: 57 failing items removed using the --remove-keywords option.
00:00:00.027 KEYWORD Check_Data_Present  
Start / End / Elapsed: 20250127 00:23:11.377 / 20250127 00:23:11.404 / 00:00:00.027
00:23:11.405 FAIL Keyword 'Check_Data_Present' failed after retrying for 1 minute. The last error was: HTTPError: 503 Server Error: Service Unavailable for url: http://10.30.170.220:8181/rests/data/network-topology:network-topology/topology=topology-netconf/node=17830-sim-device/yang-ext:mount?content=config
00:00:00.013 TEARDOWN SetupUtils . Teardown_Test_Show_Bugs_And_Start_Fast_Failing_If_Test_Failed  
Documentation:

Test case teardown. Show linked bugs and start fast failing in case of failure.
Start / End / Elapsed: 20250127 00:23:11.406 / 20250127 00:23:11.419 / 00:00:00.013
00:00:00.013 TEST Create_Device_Data
Full Name: netconf-clustering-scale.txt.Create_Device_Data
Documentation:

Send some sample test data into the device through node 2 and check that the request went OK.
Tags: 1node, critical, netconf
Start / End / Elapsed: 20250127 00:23:11.419 / 20250127 00:23:11.432 / 00:00:00.013
Status: FAIL
Message: Setup failed: SKIPPED due to a failure in a previous fundamental test case.
00:00:00.014 TEST Run_Restperfclient
Full Name: netconf-clustering-scale.txt.Run_Restperfclient
Documentation:

Deploy and execute restperfclient, asking it to send the specified amount of requests to the netconf connector of the device through node 3. The duration of this test case is the main performance metric.
Tags: 1node, critical, netconf
Start / End / Elapsed: 20250127 00:23:11.432 / 20250127 00:23:11.446 / 00:00:00.014
Status: FAIL
Message: Setup failed: SKIPPED due to a failure in a previous fundamental test case.
00:00:00.014 TEST Check_For_Failed_Requests
Full Name: netconf-clustering-scale.txt.Check_For_Failed_Requests
Documentation:

Make sure there are no failed requests in the restperfclient log. This is a separate test case to distinguish between restperfclient failure and failed requests. Failed requests are rejected because we don't want to test performance of ODL rejecting our requests. If this test case fails, then the duration of Run_Restperfclient cannot be trusted to show the real performance of the cluster.
Tags: 1node, critical, netconf
Start / End / Elapsed: 20250127 00:23:11.447 / 20250127 00:23:11.461 / 00:00:00.014
Status: FAIL
Message: Setup failed: SKIPPED due to a failure in a previous fundamental test case.
00:00:00.266 TEST Cleanup_And_Collect
Full Name: netconf-clustering-scale.txt.Cleanup_And_Collect
Documentation:

Deconfigure the testtool device on Netconf connector using node 1.
Tags: 1node, critical, netconf
Start / End / Elapsed: 20250127 00:23:11.461 / 20250127 00:23:11.727 / 00:00:00.266
Status: FAIL
Message: ... click for list of related bugs or create a new one if needed (with the "netconf_clustering_scale_txt_cleanup_and_collect" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=netconf_clustering_scale_txt_cleanup_and_collect&order=bug_status" '' == ''