Test Statistics

Total Statistics
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
All Tests
413000:13:20
Statistics by Tag
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
critical
413000:13:20
Statistics by Suite
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
bgpcep-bgpclustering-longevity.txt
413000:16:25
bgpcep-bgpclustering-longevity . txt . Restart Odl With Tell Based True
312000:13:25
bgpcep-bgpclustering-longevity . txt . Singlepeer Pc Shm 300Kroutes Longevity
101000:03:00

Test Execution Log

00:16:24.767 SUITE bgpcep-bgpclustering-longevity.txt
Full Name: bgpcep-bgpclustering-longevity.txt
Start / End / Elapsed: 20250322 23:13:56.612 / 20250322 23:30:21.379 / 00:16:24.767
Status: 4 tests total, 1 passed, 3 failed, 0 skipped
00:13:24.739 SUITE Restart Odl With Tell Based True
Full Name: bgpcep-bgpclustering-longevity.txt.Restart Odl With Tell Based True
Documentation:

Set tell-based protocol usage

Copyright (c) 2016 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

Suite stops all odl nodes, un-comment usage of tell-based protocol in config file (means make it true) and starts all nodes again.
Source: /w/workspace/bgpcep-csit-3node-bgpclustering-longevity-only-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
Start / End / Elapsed: 20250322 23:13:56.635 / 20250322 23:27:21.374 / 00:13:24.739
Status: 3 tests total, 1 passed, 2 failed, 0 skipped
00:00:04.666 SETUP SetupUtils . Setup_Utils_For_Setup_And_Teardown   http_timeout=125
Documentation:

Prepare both FailFast and Karaf logging, log that the caller suite has started. This keyword is to be used in suite setup, and without it the other keywords may not work properly.
Start / End / Elapsed: 20250322 23:13:57.182 / 20250322 23:14:01.848 / 00:00:04.666
00:00:00.000 TEARDOWN SSHLibrary . Close All Connections  
Documentation:

Closes all open connections.
Start / End / Elapsed: 20250322 23:27:21.374 / 20250322 23:27:21.374 / 00:00:00.000
00:06:09.838 TEST Stop_All_Members
Full Name: bgpcep-bgpclustering-longevity.txt.Restart Odl With Tell Based True.Stop_All_Members
Documentation:

Stop every odl node.
Tags: critical
Start / End / Elapsed: 20250322 23:14:01.848 / 20250322 23:20:11.686 / 00:06:09.838
Status: FAIL
Message: Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
00:06:09.837 KEYWORD ClusterManagement . Stop_Members_From_List_Or_All  
Documentation:

If the list is empty, stops all ODL instances. Otherwise stop members based on ${stop_index_list} If ${confirm} is True, verify stopped instances are not there anymore. The KW will return a list of available members: ${updated index_list}=${original_index_list}-${member_index_list}
Start / End / Elapsed: 20250322 23:14:01.848 / 20250322 23:20:11.685 / 00:06:09.837
00:00:00.001 KEYWORD ${stop_index_list} = ClusterManagement . List_Indices_Or_All   given_list=${member_index_list}
Documentation:

Utility to allow ${EMPTY} as default argument value, as the internal list is computed at runtime. This keyword always returns a (shallow) copy of given or default list, so operations with the returned list should not affect other lists. Also note that this keyword does not consider empty list to be ${EMPTY}.
Start / End / Elapsed: 20250322 23:14:01.852 / 20250322 23:14:01.853 / 00:00:00.001
00:00:00.001 KEYWORD ${index_list} = ClusterManagement . List_Indices_Or_All   given_list=${original_index_list}
Documentation:

Utility to allow ${EMPTY} as default argument value, as the internal list is computed at runtime. This keyword always returns a (shallow) copy of given or default list, so operations with the returned list should not affect other lists. Also note that this keyword does not consider empty list to be ${EMPTY}.
Start / End / Elapsed: 20250322 23:14:01.854 / 20250322 23:14:01.855 / 00:00:00.001
00:00:02.445 KEYWORD ClusterManagement . Run_Bash_Command_On_List_Or_All   command=${NODE_STOP_COMMAND} member_index_list=${member_index_list}
Documentation:

Cycle through indices (or all), run command on each.
Start / End / Elapsed: 20250322 23:14:01.855 / 20250322 23:14:04.300 / 00:00:02.445
00:00:00.000 KEYWORD ${updated_index_list} = BuiltIn . Create List   @{index_list}
Documentation:

Returns a list containing given items.
Start / End / Elapsed: 20250322 23:14:04.301 / 20250322 23:14:04.301 / 00:00:00.000
00:00:00.000 KEYWORD Collections . Remove Values From List   ${updated_index_list} @{stop_index_list}
Documentation:

Removes all occurrences of given values from list.
Start / End / Elapsed: 20250322 23:14:04.301 / 20250322 23:14:04.301 / 00:00:00.000
00:00:00.000 IF not ${confirm}  
Start / End / Elapsed: 20250322 23:14:04.301 / 20250322 23:14:04.301 / 00:00:00.000
00:06:07.383 FOR ${index} IN @{stop_index_list}  
Start / End / Elapsed: 20250322 23:14:04.302 / 20250322 23:20:11.685 / 00:06:07.383
00:00:02.737 ITERATION ${index} = 1  
Start / End / Elapsed: 20250322 23:14:04.302 / 20250322 23:14:07.039 / 00:00:02.737
00:00:02.776 ITERATION ${index} = 2  
Start / End / Elapsed: 20250322 23:14:07.040 / 20250322 23:14:09.816 / 00:00:02.776
00:06:01.869 ITERATION ${index} = 3  
Start / End / Elapsed: 20250322 23:14:09.816 / 20250322 23:20:11.685 / 00:06:01.869
00:06:01.869 KEYWORD BuiltIn . Wait Until Keyword Succeeds   ${timeout} 2s Verify_Karaf_Is_Not_Running_On_Member member_index=${index}
Documentation:

Runs the specified keyword and retries if it fails.
Start / End / Elapsed: 20250322 23:14:09.816 / 20250322 23:20:11.685 / 00:06:01.869
Message: 150 failing items removed using the --remove-keywords option.
00:00:00.405 KEYWORD ClusterManagement . Verify_Karaf_Is_Not_Running_On_Member   member_index=${index}
Documentation:

Fail if non-zero karaf instances are counted on member of given index.
Start / End / Elapsed: 20250322 23:20:11.279 / 20250322 23:20:11.684 / 00:00:00.405
00:00:00.403 KEYWORD ${count} = ClusterManagement . Count_Running_Karafs_On_Member   member_index=${member_index}
Documentation:

Remotely execute grep for karaf process, return count as string.
Start / End / Elapsed: 20250322 23:20:11.280 / 20250322 23:20:11.683 / 00:00:00.403
00:00:00.001 KEYWORD BuiltIn . Should Be Equal   0 ${count} Found running Karaf count: ${count}
Documentation:

Fails if the given objects are unequal.
Start / End / Elapsed: 20250322 23:20:11.684 / 20250322 23:20:11.685 / 00:00:00.001
23:20:11.684 FAIL Found running Karaf count: 1: 0 != 1
23:20:11.684 FAIL Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
00:00:00.000 KEYWORD ClusterManagement . Run_Bash_Command_On_List_Or_All   command=netstat -pnatu | grep 2550
Documentation:

Cycle through indices (or all), run command on each.
Start / End / Elapsed: 20250322 23:20:11.685 / 20250322 23:20:11.685 / 00:00:00.000
00:00:00.000 RETURN   ${updated_index_list}
Start / End / Elapsed: 20250322 23:20:11.685 / 20250322 23:20:11.685 / 00:00:00.000
00:00:09.659 TEST Set_Tell_Based_Protocol_Usage
Full Name: bgpcep-bgpclustering-longevity.txt.Restart Odl With Tell Based True.Set_Tell_Based_Protocol_Usage
Documentation:

Un-comment the flag usage in config file. Also clean most data except data/log/.
Tags: critical
Start / End / Elapsed: 20250322 23:20:11.686 / 20250322 23:20:21.345 / 00:00:09.659
Status: PASS
00:07:00.027 TEST Start_All_And_Sync
Full Name: bgpcep-bgpclustering-longevity.txt.Restart Odl With Tell Based True.Start_All_And_Sync
Documentation:

Start each member and wait for sync.
Tags: critical
Start / End / Elapsed: 20250322 23:20:21.346 / 20250322 23:27:21.373 / 00:07:00.027
Status: FAIL
Message: Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.170.148', port=8181): Read timed out. (read timeout=125.0)
00:07:00.021 KEYWORD ClusterManagement . Start_Members_From_List_Or_All  
Documentation:

If the list is empty, start all cluster members. Otherwise, start members based on present indices. If ${wait_for_sync}, wait for cluster sync on listed members. Optionally karaf_home can be overriden. Optionally specific JAVA_HOME is used for starting. Garbage collection is unconditionally logged to files. TODO: Make that reasonable conditional?
Start / End / Elapsed: 20250322 23:20:21.352 / 20250322 23:27:21.373 / 00:07:00.021
00:00:00.000 KEYWORD ${base_command} = BuiltIn . Set Variable If   """${karaf_home}""" != "" ${karaf_home}/bin/start ${NODE_START_COMMAND}
Documentation:

Sets variable based on the given condition.
Start / End / Elapsed: 20250322 23:20:21.359 / 20250322 23:20:21.359 / 00:00:00.000
00:00:00.000 KEYWORD ${command} = BuiltIn . Set Variable If   """${export_java_home}""" != "" export JAVA_HOME="${export_java_home}"; ${base_command} ${base_command}
Documentation:

Sets variable based on the given condition.
Start / End / Elapsed: 20250322 23:20:21.360 / 20250322 23:20:21.360 / 00:00:00.000
00:00:00.000 KEYWORD ${epoch} = DateTime . Get Current Date   time_zone=UTC result_format=epoch exclude_millis=False
Documentation:

Returns current local or UTC time with an optional increment.
Start / End / Elapsed: 20250322 23:20:21.360 / 20250322 23:20:21.360 / 00:00:00.000
00:00:00.000 KEYWORD ${gc_filepath} = BuiltIn . Set Variable If   """${karaf_home}""" != "" ${karaf_home}/data/log/gc_${epoch}.log ${GC_LOG_PATH}/gc_${epoch}.log
Documentation:

Sets variable based on the given condition.
Start / End / Elapsed: 20250322 23:20:21.360 / 20250322 23:20:21.360 / 00:00:00.000
00:00:00.000 KEYWORD ${gc_options} = BuiltIn . Set Variable If   "docker" not in """${node_start_command}""" -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:${gc_filepath} ${EMPTY}
Documentation:

Sets variable based on the given condition.
Start / End / Elapsed: 20250322 23:20:21.361 / 20250322 23:20:21.361 / 00:00:00.000
00:00:01.173 KEYWORD ClusterManagement . Run_Bash_Command_On_List_Or_All   command=${command} ${gc_options} member_index_list=${member_index_list}
Documentation:

Cycle through indices (or all), run command on each.
Start / End / Elapsed: 20250322 23:20:21.361 / 20250322 23:20:22.534 / 00:00:01.173
00:06:56.942 KEYWORD BuiltIn . Wait Until Keyword Succeeds   ${timeout} 10s Verify_Members_Are_Ready ${member_index_list} ${wait_for_sync} ${verify_restconf} ${check_system_status} ${service_list}
Documentation:

Runs the specified keyword and retries if it fails.
Start / End / Elapsed: 20250322 23:20:22.534 / 20250322 23:27:19.476 / 00:06:56.942
Message: 4 failing items removed using the --remove-keywords option.
00:02:05.293 KEYWORD ClusterManagement . Verify_Members_Are_Ready   ${member_index_list} ${wait_for_sync} ${verify_restconf} ${check_system_status} ${service_list}
Documentation:

Verifies the specified readiness conditions for the given listed members after startup. If ${verify_cluster_sync}, verifies the datastores have synced with the rest of the cluster. If True, verifies RESTCONF is available. If ${verify_system_status}, verifies the system services are OPERATIONAL.
Start / End / Elapsed: 20250322 23:25:14.182 / 20250322 23:27:19.475 / 00:02:05.293
00:02:05.292 IF ${verify_cluster_sync}  
Start / End / Elapsed: 20250322 23:25:14.183 / 20250322 23:27:19.475 / 00:02:05.292
00:02:05.291 KEYWORD ClusterManagement . Check_Cluster_Is_In_Sync   ${member_index_list}
Documentation:

Fail if no-sync is detected on a member from list (or any).
Start / End / Elapsed: 20250322 23:25:14.183 / 20250322 23:27:19.474 / 00:02:05.291
00:00:00.001 KEYWORD ${index_list} = ClusterManagement . List_Indices_Or_All   given_list=${member_index_list}
Documentation:

Utility to allow ${EMPTY} as default argument value, as the internal list is computed at runtime. This keyword always returns a (shallow) copy of given or default list, so operations with the returned list should not affect other lists. Also note that this keyword does not consider empty list to be ${EMPTY}.
Start / End / Elapsed: 20250322 23:25:14.184 / 20250322 23:25:14.185 / 00:00:00.001
00:02:05.289 FOR ${index} IN @{index_list}  
Start / End / Elapsed: 20250322 23:25:14.185 / 20250322 23:27:19.474 / 00:02:05.289
00:00:00.084 ITERATION ${index} = 1  
Start / End / Elapsed: 20250322 23:25:14.185 / 20250322 23:25:14.269 / 00:00:00.084
00:00:00.082 ITERATION ${index} = 2  
Start / End / Elapsed: 20250322 23:25:14.269 / 20250322 23:25:14.351 / 00:00:00.082
00:02:05.122 ITERATION ${index} = 3  
Start / End / Elapsed: 20250322 23:25:14.352 / 20250322 23:27:19.474 / 00:02:05.122
00:02:05.122 KEYWORD ${status} = ClusterManagement . Get_Sync_Status_Of_Member   member_index=${index}
Documentation:

Obtain IP, two GETs from jolokia URIs, return combined sync status as string.
Start / End / Elapsed: 20250322 23:25:14.352 / 20250322 23:27:19.474 / 00:02:05.122
00:00:00.001 KEYWORD ${session} = ClusterManagement . Resolve_Http_Session_For_Member   member_index=${member_index}
Documentation:

Return RequestsLibrary session alias pointing to node of given index.
Start / End / Elapsed: 20250322 23:25:14.352 / 20250322 23:25:14.353 / 00:00:00.001
00:02:05.118 KEYWORD ${conf_text} = TemplatedRequests . Get_As_Json_From_Uri   uri=${JOLOKIA_CONF_SHARD_MANAGER_URI} session=${session}
Documentation:

Specify JSON headers and return Get_From_Uri normalized response text.
Start / End / Elapsed: 20250322 23:25:14.353 / 20250322 23:27:19.471 / 00:02:05.118
00:02:05.115 KEYWORD ${response_text} = TemplatedRequests . Get_From_Uri   uri=${uri} accept=${ACCEPT_EMPTY} session=${session} normalize_json=True http_timeout=${http_timeout} log_response=${log_response}
Documentation:

GET data from given URI, check status code and return response text. ${accept} is a Python object with headers to use. If ${normalize_json}, normalize as JSON text before returning.
Start / End / Elapsed: 20250322 23:25:14.356 / 20250322 23:27:19.471 / 00:02:05.115
00:00:00.000 KEYWORD BuiltIn . Log   ${uri}
Documentation:

Logs the given message with the given level.
Start / End / Elapsed: 20250322 23:25:14.360 / 20250322 23:25:14.360 / 00:00:00.000
00:00:00.000 KEYWORD BuiltIn . Log   ${accept}
Documentation:

Logs the given message with the given level.
Start / End / Elapsed: 20250322 23:25:14.360 / 20250322 23:25:14.360 / 00:00:00.000
00:02:05.108 IF """${http_timeout}""" == """${EMPTY}"""  
Start / End / Elapsed: 20250322 23:25:14.361 / 20250322 23:27:19.469 / 00:02:05.108
00:02:05.105 KEYWORD ${response} = RequestsLibrary . GET On Session   ${session} url=${uri} headers=${accept}
Documentation:

Sends a GET request on a previously created HTTP Session.
Start / End / Elapsed: 20250322 23:25:14.363 / 20250322 23:27:19.468 / 00:02:05.105
23:27:19.463 FAIL ReadTimeout: HTTPConnectionPool(host='10.30.170.148', port=8181): Read timed out. (read timeout=125.0)
00:00:00.001 ELSE  
Start / End / Elapsed: 20250322 23:27:19.468 / 20250322 23:27:19.469 / 00:00:00.001
00:00:00.000 KEYWORD TemplatedRequests . Check_Status_Code   ${response} log_response=${log_response}
Documentation:

Log response text, check status_code is one of allowed ones. In cases where this keyword is called in a WUKS it could end up logging tons of data and it may be desired to skip the logging by passing log_response=False, but by default it remains True.
Start / End / Elapsed: 20250322 23:27:19.470 / 20250322 23:27:19.470 / 00:00:00.000
00:00:00.000 IF not ${normalize_json}  
Start / End / Elapsed: 20250322 23:27:19.470 / 20250322 23:27:19.470 / 00:00:00.000
00:00:00.000 KEYWORD ${text_normalized} = norm_json . Normalize Json Text   ${response.text} jmes_path=${jmes_path} keys_with_volatiles=${keys_with_volatiles}
Documentation:

Attempt to return sorted indented JSON string.
Start / End / Elapsed: 20250322 23:27:19.471 / 20250322 23:27:19.471 / 00:00:00.000
00:00:00.000 RETURN   ${text_normalized}
Start / End / Elapsed: 20250322 23:27:19.471 / 20250322 23:27:19.471 / 00:00:00.000
00:00:00.000 RETURN   ${response_text}
Start / End / Elapsed: 20250322 23:27:19.471 / 20250322 23:27:19.471 / 00:00:00.000
00:00:00.000 KEYWORD ${conf_status} = ClusterManagement . ClusterManagement__Parse_Sync_Status   shard_manager_text=${conf_text}
Documentation:

Return sync status parsed out of given text. Called twice by Get_Sync_Status_Of_Member.
Start / End / Elapsed: 20250322 23:27:19.472 / 20250322 23:27:19.472 / 00:00:00.000
00:00:00.000 IF 'False' == ${conf_status}  
Start / End / Elapsed: 20250322 23:27:19.472 / 20250322 23:27:19.472 / 00:00:00.000
00:00:00.000 KEYWORD ${oper_text} = TemplatedRequests . Get_As_Json_From_Uri   uri=${JOLOKIA_OPER_SHARD_MANAGER_URI} session=${session}
Documentation:

Specify JSON headers and return Get_From_Uri normalized response text.
Start / End / Elapsed: 20250322 23:27:19.473 / 20250322 23:27:19.473 / 00:00:00.000
00:00:00.000 KEYWORD ${oper_status} = ClusterManagement . ClusterManagement__Parse_Sync_Status   shard_manager_text=${oper_text}
Documentation:

Return sync status parsed out of given text. Called twice by Get_Sync_Status_Of_Member.
Start / End / Elapsed: 20250322 23:27:19.473 / 20250322 23:27:19.473 / 00:00:00.000
00:00:00.000 RETURN   ${oper_status}
Start / End / Elapsed: 20250322 23:27:19.473 / 20250322 23:27:19.473 / 00:00:00.000
00:00:00.000 IF 'True' == '${status}'  
Start / End / Elapsed: 20250322 23:27:19.474 / 20250322 23:27:19.474 / 00:00:00.000
00:00:00.000 KEYWORD BuiltIn . Fail   Index ${index} has incorrect status: ${status}
Documentation:

Fails the test with the given message and optionally alters its tags.
Start / End / Elapsed: 20250322 23:27:19.474 / 20250322 23:27:19.474 / 00:00:00.000
00:00:00.000 IF ${verify_restconf}  
Start / End / Elapsed: 20250322 23:27:19.475 / 20250322 23:27:19.475 / 00:00:00.000
00:00:00.000 IF ${verify_system_status} and ("${service_list}" != "[[]]")  
Start / End / Elapsed: 20250322 23:27:19.475 / 20250322 23:27:19.475 / 00:00:00.000
23:27:19.476 FAIL Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.170.148', port=8181): Read timed out. (read timeout=125.0)
00:00:01.897 TEARDOWN ClusterManagement . Run_Bash_Command_On_List_Or_All   command=netstat -pnatu | grep 2550
Documentation:

Cycle through indices (or all), run command on each.
Start / End / Elapsed: 20250322 23:27:19.476 / 20250322 23:27:21.373 / 00:00:01.897
00:00:00.000 KEYWORD BuiltIn . Wait Until Keyword Succeeds   300s 10s ShardStability.Shards_Stability_Get_Details ${DEFAULT_SHARD_LIST} verify_restconf=True
Documentation:

Runs the specified keyword and retries if it fails.
Start / End / Elapsed: 20250322 23:27:21.373 / 20250322 23:27:21.373 / 00:00:00.000
00:00:00.000 KEYWORD ClusterManagement . Run_Bash_Command_On_List_Or_All   ps -ef | grep java
Documentation:

Cycle through indices (or all), run command on each.
Start / End / Elapsed: 20250322 23:27:21.373 / 20250322 23:27:21.373 / 00:00:00.000
00:03:00.002 SUITE Singlepeer Pc Shm 300Kroutes Longevity
Full Name: bgpcep-bgpclustering-longevity.txt.Singlepeer Pc Shm 300Kroutes Longevity
Documentation:

BGP performance of ingesting from 1 iBGP peer, data change counter is NOT used.

Copyright (c) 2015-2017 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

This suite uses play.py as single iBGP peer which talks to single controller in three node cluster configuration. Test suite checks changes of the the example-ipv4-topology on all nodes. RIB is not examined.

singlepeer_pc_shm_300kroutes_longevity.robot: pc - prefix counting shm - shard monitoring (during the process of prefix advertizing)
Source: /w/workspace/bgpcep-csit-3node-bgpclustering-longevity-only-scandium/test/csit/suites/bgpcep/bgpclustering/singlepeer_pc_shm_300kroutes_longevity.robot
Start / End / Elapsed: 20250322 23:27:21.375 / 20250322 23:30:21.377 / 00:03:00.002
Status: 1 test total, 0 passed, 1 failed, 0 skipped
Message: Suite setup failed: ReadTimeout: HTTPConnectionPool(host='10.30.170.148', port=8181): Read timed out. (read timeout=125.0)