Test Statistics

Total Statistics
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
All Tests
936000:07:02
Statistics by Tag
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
No Tags
Statistics by Suite
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
openflowplugin-clustering-perf-bulkomatic.txt
936000:07:02

Test Execution Log

00:07:02.374 SUITE openflowplugin-clustering-perf-bulkomatic.txt
Full Name: openflowplugin-clustering-perf-bulkomatic.txt
Documentation:

Test suite for 3Node Cluster - 100K flows and 10 DPNs in Cluster Scale Up scenario
Source: /w/workspace/openflowplugin-csit-3node-clustering-perf-bulkomatic-only-titanium/test/csit/suites/openflowplugin/Performance_Bulkomatic/100K_Flows_100DPN_3Node_Cluster_Perf_Test.robot
Start / End / Elapsed: 20241229 02:24:00.858 / 20241229 02:31:03.232 / 00:07:02.374
Status: 9 tests total, 3 passed, 6 failed, 0 skipped
00:00:00.023 SETUP ClusterManagement . ClusterManagement_Setup  
Documentation:

Detect repeated call, or detect number of members and initialize derived suite variables. Http sessions are created with parameters to not waste time when ODL is no accepting connections properly.
Start / End / Elapsed: 20241229 02:24:01.419 / 20241229 02:24:01.442 / 00:00:00.023
00:00:00.000 TEARDOWN RequestsLibrary . Delete All Sessions  
Documentation:

Removes all the session objects
Start / End / Elapsed: 20241229 02:31:03.232 / 20241229 02:31:03.232 / 00:00:00.000
00:00:00.024 TEST Check Shards Status And Initialize Variables
Full Name: openflowplugin-clustering-perf-bulkomatic.txt.Check Shards Status And Initialize Variables
Documentation:

Check Status for all shards in OpenFlow application and set the logs across cluster nodes.
Start / End / Elapsed: 20241229 02:24:01.442 / 20241229 02:24:01.466 / 00:00:00.024
Status: FAIL
Message: Dictionary '${Stream_dict}' has no key 'titanium'.
00:00:00.021 KEYWORD ClusterOpenFlow . Check OpenFlow Shards Status  
Documentation:

Check Status for all shards in OpenFlow application.
Start / End / Elapsed: 20241229 02:24:01.442 / 20241229 02:24:01.463 / 00:00:00.021
00:00:00.009 KEYWORD CompareStream . Run_Keyword_If_At_Least_Phosphorus   Collections.Remove Values From List ${SHARD_OPER_LIST} entity-ownership
Documentation:

Compare phosphorus to titanium and in case titanium is at least phosphorus, run ${kw_name} @{varargs} &{kwargs} and return its value.
Start / End / Elapsed: 20241229 02:24:01.453 / 20241229 02:24:01.462 / 00:00:00.009
00:00:00.009 KEYWORD BuiltIn . Run Keyword And Return   Run_Keyword_If_At_Least phosphorus ${kw_name} @{varargs} &{kwargs}
Documentation:

Runs the specified keyword and returns from the enclosing user keyword.
Start / End / Elapsed: 20241229 02:24:01.453 / 20241229 02:24:01.462 / 00:00:00.009
00:00:00.001 KEYWORD CompareStream . Run_Keyword_If_At_Least   phosphorus ${kw_name} @{varargs} &{kwargs}
Documentation:

Compare ${lower_bound} to titanium and in case titanium is at least ${lower_bound}, run Collections.Remove Values From List [['inventory', 'topology', 'default', 'entity-ownership'], 'entity-ownership'] {} and return its value.
Start / End / Elapsed: 20241229 02:24:01.461 / 20241229 02:24:01.462 / 00:00:00.001
00:00:00.000 KEYWORD BuiltIn . Run Keyword And Return If   ${Stream_dict}[${ODL_STREAM}] >= ${Stream_dict}[${lower_bound}] ${kw_name} @{varargs} &{kwargs}
Documentation:

Runs the specified keyword and returns from the enclosing user keyword.
Start / End / Elapsed: 20241229 02:24:01.462 / 20241229 02:24:01.462 / 00:00:00.000
02:24:01.462 FAIL Dictionary '${Stream_dict}' has no key 'titanium'.
02:24:01.462 INFO Returning from the enclosing user keyword.
00:00:00.000 KEYWORD BuiltIn . Log   ${SHARD_OPER_LIST}
Documentation:

Logs the given message with the given level.
Start / End / Elapsed: 20241229 02:24:01.463 / 20241229 02:24:01.463 / 00:00:00.000
00:00:00.000 KEYWORD ClusterManagement . Verify_Leader_Exists_For_Each_Shard   shard_name_list=${SHARD_OPER_LIST} shard_type=operational member_index_list=${controller_index_list}
Documentation:

For each shard name, call Get_Leader_And_Followers_For_Shard. Not much logic there, but single Keyword is useful when using BuiltIn.Wait_Until_Keyword_Succeeds.
Start / End / Elapsed: 20241229 02:24:01.463 / 20241229 02:24:01.463 / 00:00:00.000
00:00:00.000 KEYWORD ClusterManagement . Verify_Leader_Exists_For_Each_Shard   shard_name_list=${SHARD_CONF_LIST} shard_type=config member_index_list=${controller_index_list}
Documentation:

For each shard name, call Get_Leader_And_Followers_For_Shard. Not much logic there, but single Keyword is useful when using BuiltIn.Wait_Until_Keyword_Succeeds.
Start / End / Elapsed: 20241229 02:24:01.463 / 20241229 02:24:01.463 / 00:00:00.000
00:00:00.000 KEYWORD BuiltIn . Wait Until Keyword Succeeds   3x 3s ClusterManagement.Run_Karaf_Command_On_List_Or_All ${karaf_log_level}
Documentation:

Runs the specified keyword and retries if it fails.
Start / End / Elapsed: 20241229 02:24:01.463 / 20241229 02:24:01.463 / 00:00:00.000
00:00:00.000 KEYWORD ${flow_count_after_add} = BuiltIn . Evaluate   ${flow_count_per_switch} * ${switch_count}
Documentation:

Evaluates the given expression in Python and returns the result.
Start / End / Elapsed: 20241229 02:24:01.464 / 20241229 02:24:01.464 / 00:00:00.000
00:00:00.000 KEYWORD BuiltIn . Set Suite Variable   ${flow_count_after_add}
Documentation:

Makes a variable available everywhere within the scope of the current suite.
Start / End / Elapsed: 20241229 02:24:01.464 / 20241229 02:24:01.464 / 00:00:00.000
00:00:00.000 KEYWORD ${temp_json_config_add} = BulkomaticKeywords . Set DPN And Flow Count In Json Add   ${orig_json_config_add} ${switch_count} ${flow_count_per_switch}
Documentation:

Set new DPN count and flows count per DPN in the Bulkomatic Add json file.
Start / End / Elapsed: 20241229 02:24:01.464 / 20241229 02:24:01.464 / 00:00:00.000
00:00:00.000 KEYWORD ${temp_json_config_get} = BulkomaticKeywords . Set DPN And Flow Count In Json Get   ${orig_json_config_get} ${switch_count} ${flow_count_after_add}
Documentation:

Set new DPN count and flows count per DPN in the Bulkomatic Get json file.
Start / End / Elapsed: 20241229 02:24:01.464 / 20241229 02:24:01.464 / 00:00:00.000
00:00:00.000 KEYWORD ${temp_json_config_del} = BulkomaticKeywords . Set DPN And Flow Count In Json Del   ${orig_json_config_del} ${switch_count} ${flow_count_per_switch}
Documentation:

Set new DPN count and flows count per DPN in the Bulkomatic Del json file.
Start / End / Elapsed: 20241229 02:24:01.465 / 20241229 02:24:01.465 / 00:00:00.000
00:00:00.000 KEYWORD BuiltIn . Set Suite Variable   ${temp_json_config_add}
Documentation:

Makes a variable available everywhere within the scope of the current suite.
Start / End / Elapsed: 20241229 02:24:01.465 / 20241229 02:24:01.465 / 00:00:00.000
00:00:00.000 KEYWORD BuiltIn . Set Suite Variable   ${temp_json_config_get}
Documentation:

Makes a variable available everywhere within the scope of the current suite.
Start / End / Elapsed: 20241229 02:24:01.465 / 20241229 02:24:01.465 / 00:00:00.000
00:00:00.000 KEYWORD BuiltIn . Set Suite Variable   ${temp_json_config_del}
Documentation:

Makes a variable available everywhere within the scope of the current suite.
Start / End / Elapsed: 20241229 02:24:01.465 / 20241229 02:24:01.465 / 00:00:00.000
00:00:06.838 TEST Get Inventory Config Shard Follower And Leader
Full Name: openflowplugin-clustering-perf-bulkomatic.txt.Get Inventory Config Shard Follower And Leader
Documentation:

Find a leader and followers in the inventory config shard
Start / End / Elapsed: 20241229 02:24:01.466 / 20241229 02:24:08.304 / 00:00:06.838
Status: PASS
00:00:09.115 TEST Start Mininet Connect To Follower Node1
Full Name: openflowplugin-clustering-perf-bulkomatic.txt.Start Mininet Connect To Follower Node1
Documentation:

Start mininet with connection to follower node1.
Start / End / Elapsed: 20241229 02:24:08.304 / 20241229 02:24:17.419 / 00:00:09.115
Status: PASS
00:00:00.011 TEST Add Bulk Flow From Follower Node2
Full Name: openflowplugin-clustering-perf-bulkomatic.txt.Add Bulk Flow From Follower Node2
Documentation:

100K Flows (1K Flows per DPN) in 100 DPN added via Follower Node2 and verify it gets applied in all instances.
Start / End / Elapsed: 20241229 02:24:17.419 / 20241229 02:24:17.430 / 00:00:00.011
Status: FAIL
Message: Variable '${temp_json_config_add}' not found.
00:00:00.000 KEYWORD ${config_datastore_write_start_time} = DateTime . Get Current Date   result_format=timestamp
Documentation:

Returns current local or UTC time with an optional increment.
Start / End / Elapsed: 20241229 02:24:17.419 / 20241229 02:24:17.419 / 00:00:00.000
00:00:00.004 KEYWORD BulkomaticKeywords . Add Bulk Flow In Node   ${temp_json_config_add} ${Follower_Node_2} ${operation_timeout}
Documentation:

Add Bulk Flow in member ${controller_index} and wait until operation is completed.
Start / End / Elapsed: 20241229 02:24:17.424 / 20241229 02:24:17.428 / 00:00:00.004
02:24:17.428 FAIL Variable '${temp_json_config_add}' not found.
00:00:00.000 KEYWORD ${config_datastore_write_end_time} = DateTime . Get Current Date   result_format=timestamp
Documentation:

Returns current local or UTC time with an optional increment.
Start / End / Elapsed: 20241229 02:24:17.428 / 20241229 02:24:17.428 / 00:00:00.000
00:00:00.000 KEYWORD ${config_datastore_write_time} = DateTime . Subtract Date From Date   ${config_datastore_write_end_time} ${config_datastore_write_start_time}
Documentation:

Subtracts date from another date and returns time between.
Start / End / Elapsed: 20241229 02:24:17.428 / 20241229 02:24:17.428 / 00:00:00.000
00:00:00.000 KEYWORD ${config_datastore_write_rate} = BuiltIn . Evaluate   ${flow_count_after_add} / ${config_datastore_write_time}
Documentation:

Evaluates the given expression in Python and returns the result.
Start / End / Elapsed: 20241229 02:24:17.428 / 20241229 02:24:17.428 / 00:00:00.000
00:00:00.000 KEYWORD BuiltIn . Set Suite Variable   ${config_datastore_write_start_time}
Documentation:

Makes a variable available everywhere within the scope of the current suite.
Start / End / Elapsed: 20241229 02:24:17.428 / 20241229 02:24:17.428 / 00:00:00.000
00:00:00.000 KEYWORD BuiltIn . Log To Console   ${\n}
Documentation:

Logs the given message to the console.
Start / End / Elapsed: 20241229 02:24:17.429 / 20241229 02:24:17.429 / 00:00:00.000
00:00:00.000 KEYWORD BuiltIn . Log To Console   The config_datastore_write_time is ${config_datastore_write_time}
Documentation:

Logs the given message to the console.
Start / End / Elapsed: 20241229 02:24:17.429 / 20241229 02:24:17.429 / 00:00:00.000
00:00:00.000 KEYWORD BuiltIn . Log To Console   The config_datastore_write_rate is ${config_datastore_write_rate}
Documentation:

Logs the given message to the console.
Start / End / Elapsed: 20241229 02:24:17.429 / 20241229 02:24:17.429 / 00:00:00.000
00:00:00.000 KEYWORD BuiltIn . Set Suite Variable   ${config_datastore_write_time}
Documentation:

Makes a variable available everywhere within the scope of the current suite.
Start / End / Elapsed: 20241229 02:24:17.429 / 20241229 02:24:17.429 / 00:00:00.000
00:00:00.000 KEYWORD BuiltIn . Set Suite Variable   ${config_datastore_write_rate}
Documentation:

Makes a variable available everywhere within the scope of the current suite.
Start / End / Elapsed: 20241229 02:24:17.429 / 20241229 02:24:17.429 / 00:00:00.000
00:00:00.006 TEST Verify Flows In Switch
Full Name: openflowplugin-clustering-perf-bulkomatic.txt.Verify Flows In Switch
Documentation:

Verify 100K flows are installed in 10 switches.
Start / End / Elapsed: 20241229 02:24:17.430 / 20241229 02:24:17.436 / 00:00:00.006
Status: FAIL
Message: Variable '${flow_count_after_add}' not found.
00:00:00.000 KEYWORD ${Mininet_write_start_time} = DateTime . Get Current Date   result_format=timestamp
Documentation:

Returns current local or UTC time with an optional increment.
Start / End / Elapsed: 20241229 02:24:17.430 / 20241229 02:24:17.430 / 00:00:00.000
00:00:00.004 KEYWORD MininetKeywords . Verify Aggregate Flow From Mininet Session   ${mininet_conn_id} ${flow_count_after_add} ${operation_timeout}
Documentation:

Verify flow count per switch
Start / End / Elapsed: 20241229 02:24:17.431 / 20241229 02:24:17.435 / 00:00:00.004
02:24:17.435 FAIL Variable '${flow_count_after_add}' not found.
00:00:00.000 KEYWORD ${Mininet_write_end_time} = DateTime . Get Current Date   result_format=timestamp
Documentation:

Returns current local or UTC time with an optional increment.
Start / End / Elapsed: 20241229 02:24:17.435 / 20241229 02:24:17.435 / 00:00:00.000
00:00:00.000 KEYWORD ${Mininet_write_time} = DateTime . Subtract Date From Date   ${Mininet_write_end_time} ${config_datastore_write_start_time}
Documentation:

Subtracts date from another date and returns time between.
Start / End / Elapsed: 20241229 02:24:17.435 / 20241229 02:24:17.435 / 00:00:00.000
00:00:00.000 KEYWORD ${Mininet_write_rate} = BuiltIn . Evaluate   ${flow_count_after_add} / ${Mininet_write_time}
Documentation:

Evaluates the given expression in Python and returns the result.
Start / End / Elapsed: 20241229 02:24:17.435 / 20241229 02:24:17.435 / 00:00:00.000
00:00:00.000 KEYWORD BuiltIn . Log To Console   ${\n}
Documentation:

Logs the given message to the console.
Start / End / Elapsed: 20241229 02:24:17.435 / 20241229 02:24:17.435 / 00:00:00.000
00:00:00.000 KEYWORD BuiltIn . Log To Console   The Mininet_write_time is ${Mininet_write_time}
Documentation:

Logs the given message to the console.
Start / End / Elapsed: 20241229 02:24:17.435 / 20241229 02:24:17.435 / 00:00:00.000
00:00:00.000 KEYWORD BuiltIn . Log To Console   The Mininet_write_rate is ${Mininet_write_rate}
Documentation:

Logs the given message to the console.
Start / End / Elapsed: 20241229 02:24:17.436 / 20241229 02:24:17.436 / 00:00:00.000
00:00:00.000 KEYWORD BuiltIn . Set Suite Variable   ${Mininet_write_time}
Documentation:

Makes a variable available everywhere within the scope of the current suite.
Start / End / Elapsed: 20241229 02:24:17.436 / 20241229 02:24:17.436 / 00:00:00.000
00:00:00.000 KEYWORD BuiltIn . Set Suite Variable   ${Mininet_write_rate}
Documentation:

Makes a variable available everywhere within the scope of the current suite.
Start / End / Elapsed: 20241229 02:24:17.436 / 20241229 02:24:17.436 / 00:00:00.000
00:06:40.910 TEST Verify Flows In Oper DS
Full Name: openflowplugin-clustering-perf-bulkomatic.txt.Verify Flows In Oper DS
Documentation:

Check Flows in Operational Datastore
Start / End / Elapsed: 20241229 02:24:17.436 / 20241229 02:30:58.346 / 00:06:40.910
Status: FAIL
Message: Keyword 'ClusterOpenFlow.Check_Flows_Operational_Datastore_On_Member' failed after retrying for 6 minutes 40 seconds. The last error was: Variable '${flow_count_after_add}' not found.
00:00:04.864 TEST Stop Mininet Connected To Follower Node1 After Reconcilliation
Full Name: openflowplugin-clustering-perf-bulkomatic.txt.Stop Mininet Connected To Follower Node1 After Reconcilliation
Documentation:

Stop mininet and exit connection.
Start / End / Elapsed: 20241229 02:30:58.347 / 20241229 02:31:03.211 / 00:00:04.864
Status: PASS
00:00:00.014 TEST Delete All Flows From Follower Node2
Full Name: openflowplugin-clustering-perf-bulkomatic.txt.Delete All Flows From Follower Node2
Documentation:

100K Flows deleted via Follower Node2 and verify it gets applied in all instances.
Start / End / Elapsed: 20241229 02:31:03.212 / 20241229 02:31:03.226 / 00:00:00.014
Status: FAIL
Message: Variable '${temp_json_config_del}' not found.
00:00:00.006 TEST Log Results And Determine Status
Full Name: openflowplugin-clustering-perf-bulkomatic.txt.Log Results And Determine Status
Documentation:

Log results for plotting.
Start / End / Elapsed: 20241229 02:31:03.226 / 20241229 02:31:03.232 / 00:00:00.006
Status: FAIL
Message: Variable '${config_datastore_write_rate}' not found.