Test Statistics

Total Statistics
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
All Tests
16463101000:10:36
Statistics by Tag
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
No Tags
Statistics by Suite
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
ovsdb-upstream-clustering.txt
16463101000:10:51
ovsdb-upstream-clustering . txt . Southbound Cluster
823250000:05:32
ovsdb-upstream-clustering . txt . Southbound Cluster . Ovsdb Southbound Cluster
442717000:03:34
ovsdb-upstream-clustering . txt . Southbound Cluster . Southbound Cluster Extension
38533000:01:58
ovsdb-upstream-clustering . txt . Southbound Cluster
823151000:05:19
ovsdb-upstream-clustering . txt . Southbound Cluster . Ovsdb Southbound Cluster
44836000:02:16
ovsdb-upstream-clustering . txt . Southbound Cluster . Southbound Cluster Extension
382315000:03:03

Test Execution Log

00:10:51.345 SUITE ovsdb-upstream-clustering.txt
Full Name: ovsdb-upstream-clustering.txt
Start / End / Elapsed: 20250226 02:28:48.506 / 20250226 02:39:39.851 / 00:10:51.345
Status: 164 tests total, 63 passed, 101 failed, 0 skipped
00:05:32.011 SUITE Southbound Cluster
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster
Source: /w/workspace/ovsdb-csit-3node-upstream-clustering-only-titanium/test/csit/suites/ovsdb/Southbound_Cluster
Start / End / Elapsed: 20250226 02:28:48.529 / 20250226 02:34:20.540 / 00:05:32.011
Status: 82 tests total, 32 passed, 50 failed, 0 skipped
00:03:33.780 SUITE Ovsdb Southbound Cluster
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster
Documentation:

Test suite for Ovsdb Southbound Cluster - Owner failover and recover
Source: /w/workspace/ovsdb-csit-3node-upstream-clustering-only-titanium/test/csit/suites/ovsdb/Southbound_Cluster/010__Ovsdb_Southbound_Cluster.robot
Start / End / Elapsed: 20250226 02:28:48.530 / 20250226 02:32:22.310 / 00:03:33.780
Status: 44 tests total, 27 passed, 17 failed, 0 skipped
00:00:04.979 SETUP Suite Setup  
Start / End / Elapsed: 20250226 02:28:49.031 / 20250226 02:28:54.010 / 00:00:04.979
00:00:00.000 TEARDOWN RequestsLibrary . Delete All Sessions  
Documentation:

Removes all the session objects
Start / End / Elapsed: 20250226 02:32:22.310 / 20250226 02:32:22.310 / 00:00:00.000
00:00:08.362 TEST Check Shards Status Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Check Shards Status Before Fail
Documentation:

Check Status for all shards in Ovsdb application.
Start / End / Elapsed: 20250226 02:28:54.010 / 20250226 02:29:02.372 / 00:00:08.362
Status: PASS
00:00:03.965 TEST Start OVS Multiple Connections
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Start OVS Multiple Connections
Documentation:

Connect OVS to all cluster instances.
Start / End / Elapsed: 20250226 02:29:02.373 / 20250226 02:29:06.338 / 00:00:03.965
Status: PASS
00:00:00.752 TEST Check Entity Owner Status And Find Owner and Candidate Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Check Entity Owner Status And Find Owner and Candidate Before Fail
Documentation:

Check Entity Owner Status and identify owner and candidate.
Start / End / Elapsed: 20250226 02:29:06.337 / 20250226 02:29:07.089 / 00:00:00.752
Status: PASS
00:00:01.191 TEST Create Bridge Manually and Verify Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Bridge Manually and Verify Before Fail
Documentation:

Create bridge with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250226 02:29:07.089 / 20250226 02:29:08.280 / 00:00:01.191
Status: PASS
00:00:01.126 TEST Add Port Manually and Verify Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Add Port Manually and Verify Before Fail
Documentation:

Add port with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250226 02:29:08.280 / 20250226 02:29:09.406 / 00:00:01.126
Status: PASS
00:00:02.088 TEST Create Tap Device Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Tap Device Before Fail
Documentation:

Create tap devices to add to the bridge in ovs
Start / End / Elapsed: 20250226 02:29:09.406 / 20250226 02:29:11.494 / 00:00:02.088
Status: PASS
00:00:01.128 TEST Add Tap Device Manually and Verify Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Add Tap Device Manually and Verify Before Fail
Documentation:

Add tap devices to the bridge with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250226 02:29:11.494 / 20250226 02:29:12.622 / 00:00:01.128
Status: PASS
00:00:01.105 TEST Delete the Bridge Manually and Verify Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete the Bridge Manually and Verify Before Fail
Documentation:

Delete bridge with OVS command and verify it gets deleted from all instances.
Start / End / Elapsed: 20250226 02:29:12.622 / 20250226 02:29:13.727 / 00:00:01.105
Status: PASS
00:00:10.953 TEST Create Bridge In Owner and Verify Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Bridge In Owner and Verify Before Fail
Documentation:

Create Bridge in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250226 02:29:13.727 / 20250226 02:29:24.680 / 00:00:10.953
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 10 seconds. The last error was: '{"network-topology:node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":56158,"local-ip":"10.30.170.253","remote-ip":"10.30.170.51"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.170.253:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.177:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.224:6640","number_of_connections":1,"connected":true}]}]}' contains 'br01' 0 times, not 1 time.
00:00:00.378 SETUP SetupUtils . Setup_Test_With_Logging_And_Without_Fast_Failing  
Documentation:

Test case setup which explicitly ignores previous failure and logs test case name to Karaf log. Needed if the recommended default is to be overriden.
Start / End / Elapsed: 20250226 02:29:13.728 / 20250226 02:29:14.106 / 00:00:00.378
00:00:10.566 KEYWORD ClusterOvsdb . Create Sample Bridge And Verify   ${original_owner}
Documentation:

Create bridge br01 in controller ${controller_index} and verify it gets created in all instances in ${controller_index_list}.
Start / End / Elapsed: 20250226 02:29:14.113 / 20250226 02:29:24.679 / 00:00:10.566
00:00:00.000 KEYWORD ${body} = OperatingSystem . Get File   /w/workspace/ovsdb-csit-3node-upstream-clustering-only-titanium/test/csit/libraries/../variables/ovsdb/create_bridge_3node.json
Documentation:

Returns the contents of a specified file.
Start / End / Elapsed: 20250226 02:29:14.115 / 20250226 02:29:14.115 / 00:00:00.000
00:00:00.000 KEYWORD ${body} = String . Replace String   ${body} ovsdb://127.0.0.1:61644 ovsdb://uuid/${ovsdb_uuid}
Documentation:

Replaces search_for in the given string with replace_with.
Start / End / Elapsed: 20250226 02:29:14.116 / 20250226 02:29:14.116 / 00:00:00.000
00:00:00.000 KEYWORD ${body} = String . Replace String   ${body} tcp:controller1:6633 tcp:${ODL_SYSTEM_1_IP}:6633
Documentation:

Replaces search_for in the given string with replace_with.
Start / End / Elapsed: 20250226 02:29:14.116 / 20250226 02:29:14.116 / 00:00:00.000
00:00:00.000 KEYWORD ${body} = String . Replace String   ${body} tcp:controller2:6633 tcp:${ODL_SYSTEM_2_IP}:6633
Documentation:

Replaces search_for in the given string with replace_with.
Start / End / Elapsed: 20250226 02:29:14.117 / 20250226 02:29:14.117 / 00:00:00.000
00:00:00.000 KEYWORD ${body} = String . Replace String   ${body} tcp:controller3:6633 tcp:${ODL_SYSTEM_3_IP}:6633
Documentation:

Replaces search_for in the given string with replace_with.
Start / End / Elapsed: 20250226 02:29:14.117 / 20250226 02:29:14.117 / 00:00:00.000
00:00:00.000 KEYWORD ${body} = String . Replace String   ${body} 127.0.0.1 ${TOOLS_SYSTEM_IP}
Documentation:

Replaces search_for in the given string with replace_with.
Start / End / Elapsed: 20250226 02:29:14.118 / 20250226 02:29:14.118 / 00:00:00.000
00:00:00.000 KEYWORD ${body} = String . Replace String   ${body} br01 ${BRIDGE}
Documentation:

Replaces search_for in the given string with replace_with.
Start / End / Elapsed: 20250226 02:29:14.118 / 20250226 02:29:14.118 / 00:00:00.000
00:00:00.000 KEYWORD ${body} = String . Replace String   ${body} 61644 ${OVSDB_PORT}
Documentation:

Replaces search_for in the given string with replace_with.
Start / End / Elapsed: 20250226 02:29:14.118 / 20250226 02:29:14.118 / 00:00:00.000
00:00:00.000 KEYWORD BuiltIn . Log   ${body}
Documentation:

Logs the given message with the given level.
Start / End / Elapsed: 20250226 02:29:14.119 / 20250226 02:29:14.119 / 00:00:00.000
00:00:00.000 KEYWORD ${TOOLS_SYSTEM_IP1} = String . Replace String   ${TOOLS_SYSTEM_IP} ${TOOLS_SYSTEM_IP} "${TOOLS_SYSTEM_IP}"
Documentation:

Replaces search_for in the given string with replace_with.
Start / End / Elapsed: 20250226 02:29:14.119 / 20250226 02:29:14.119 / 00:00:00.000
00:00:00.000 KEYWORD ${dictionary} = BuiltIn . Create Dictionary   ${TOOLS_SYSTEM_IP1}=1 ${OVSDBPORT}=4 ${BRIDGE}=1
Documentation:

Creates and returns a dictionary based on the given items.
Start / End / Elapsed: 20250226 02:29:14.119 / 20250226 02:29:14.119 / 00:00:00.000
00:00:00.218 KEYWORD BuiltIn . Wait Until Keyword Succeeds   5s 1s ClusterManagement.Put_As_Json_And_Check_Member_List_Or_All ${RFC8040_SOUTHBOUND_NODE_API}uuid%2F${ovsdb_uuid}%2Fbridge%2F${BRIDGE} ${body} ${controller_index} ${controller_index_list}
Documentation:

Runs the specified keyword and retries if it fails.
Start / End / Elapsed: 20250226 02:29:14.120 / 20250226 02:29:14.338 / 00:00:00.218
00:00:10.341 KEYWORD BuiltIn . Wait Until Keyword Succeeds   10s 2s ClusterManagement.Check_Item_Occurrence_Member_List_Or_All uri=${RFC8040_SOUTHBOUND_NODE_API}uuid%2F${ovsdb_uuid}?${RFC8040_OPERATIONAL_CONTENT} dictionary=${dictionary} member_index_list=${controller_index_list}
Documentation:

Runs the specified keyword and retries if it fails.
Start / End / Elapsed: 20250226 02:29:14.338 / 20250226 02:29:24.679 / 00:00:10.341
Message: 5 failing items removed using the --remove-keywords option.
00:00:00.050 KEYWORD ClusterManagement . Check_Item_Occurrence_Member_List_Or_All   uri=${RFC8040_SOUTHBOUND_NODE_API}uuid%2F${ovsdb_uuid}?${RFC8040_OPERATIONAL_CONTENT} dictionary=${dictionary} member_index_list=${controller_index_list}
Documentation:

Send a GET with the supplied uri to all or some members defined in ${member_index_list}. Then check received for occurrences of items expressed in a dictionary {'"10.30.170.51"': '1', '6640': '4', 'br01': '1'}.
Start / End / Elapsed: 20250226 02:29:24.630 / 20250226 02:29:24.680 / 00:00:00.050
00:00:00.002 KEYWORD ${index_list} = ClusterManagement . List_Indices_Or_All   given_list=${member_index_list}
Documentation:

Utility to allow ${EMPTY} as default argument value, as the internal list is computed at runtime. This keyword always returns a (shallow) copy of given or default list, so operations with the returned list should not affect other lists. Also note that this keyword does not consider empty list to be ${EMPTY}.
Start / End / Elapsed: 20250226 02:29:24.631 / 20250226 02:29:24.633 / 00:00:00.002
00:00:00.046 FOR ${index} IN @{index_list}  
Start / End / Elapsed: 20250226 02:29:24.633 / 20250226 02:29:24.679 / 00:00:00.046
00:00:00.046 ITERATION ${index} = 1  
Start / End / Elapsed: 20250226 02:29:24.633 / 20250226 02:29:24.679 / 00:00:00.046
00:00:00.038 KEYWORD ${data} = ClusterManagement . Get_From_Member   uri=${uri} member_index=${index}
Documentation:

Send a GET with the supplied uri to member ${member_index}.
Start / End / Elapsed: 20250226 02:29:24.639 / 20250226 02:29:24.677 / 00:00:00.038
00:00:00.002 KEYWORD Utils . Check Item Occurrence   ${data} ${dictionary}
Documentation:

Check string for occurrences of items expressed in a list of dictionaries {item=occurrences}. 0 occurences means item is not present.
Start / End / Elapsed: 20250226 02:29:24.677 / 20250226 02:29:24.679 / 00:00:00.002
00:00:00.002 FOR ${item} IN @{dictionary_item_occurrence}  
Start / End / Elapsed: 20250226 02:29:24.677 / 20250226 02:29:24.679 / 00:00:00.002
00:00:00.001 ITERATION ${item} = "10.30.170.51"  
Start / End / Elapsed: 20250226 02:29:24.677 / 20250226 02:29:24.678 / 00:00:00.001
00:00:00.000 ITERATION ${item} = 6640  
Start / End / Elapsed: 20250226 02:29:24.678 / 20250226 02:29:24.678 / 00:00:00.000
00:00:00.001 ITERATION ${item} = br01  
Start / End / Elapsed: 20250226 02:29:24.678 / 20250226 02:29:24.679 / 00:00:00.001
00:00:00.000 KEYWORD BuiltIn . Should Contain X Times   ${string} ${item} ${dictionary_item_occurrence}[${item}]
Documentation:

Fails if container does not contain item count times.
Start / End / Elapsed: 20250226 02:29:24.678 / 20250226 02:29:24.678 / 00:00:00.000
02:29:24.678 INFO Item found from container 0 times.
02:29:24.679 FAIL '{"network-topology:node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":56158,"local-ip":"10.30.170.253","remote-ip":"10.30.170.51"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.170.253:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.177:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.224:6640","number_of_connections":1,"connected":true}]}]}' contains 'br01' 0 times, not 1 time.
02:29:24.679 FAIL Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 10 seconds. The last error was: '{"network-topology:node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":56158,"local-ip":"10.30.170.253","remote-ip":"10.30.170.51"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.170.253:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.177:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.224:6640","number_of_connections":1,"connected":true}]}]}' contains 'br01' 0 times, not 1 time.
00:00:10.929 TEST Create Port In Owner and Verify Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Port In Owner and Verify Before Fail
Documentation:

Create Port in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250226 02:29:24.680 / 20250226 02:29:35.609 / 00:00:10.929
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 10 seconds. The last error was: HTTPError: 409 Client Error: Conflict for url: http://10.30.170.253:8181/rests/data/network-topology:network-topology/topology=ovsdb%3A1/node=ovsdb%3A%2F%2Fuuid%2F548ac92b-69d7-4d67-8018-8c0c339a56f8%2Fbridge%2Fbr01/termination-point=vx2?content=nonconfig
00:00:00.414 SETUP SetupUtils . Setup_Test_With_Logging_And_Without_Fast_Failing  
Documentation:

Test case setup which explicitly ignores previous failure and logs test case name to Karaf log. Needed if the recommended default is to be overriden.
Start / End / Elapsed: 20250226 02:29:24.681 / 20250226 02:29:25.095 / 00:00:00.414
00:00:10.506 KEYWORD ClusterOvsdb . Create Sample Port And Verify   ${original_owner}
Documentation:

Add Port vx2 to bridge br01 in controller ${controller_index} and verify it gets added in all instances in ${controller_index_list}.
Start / End / Elapsed: 20250226 02:29:25.102 / 20250226 02:29:35.608 / 00:00:10.506
00:00:00.000 KEYWORD ${sample} = OperatingSystem . Get File   ${OVSDB_CONFIG_DIR}/create_port_3node.json
Documentation:

Returns the contents of a specified file.
Start / End / Elapsed: 20250226 02:29:25.104 / 20250226 02:29:25.104 / 00:00:00.000
00:00:00.000 KEYWORD ${body} = String . Replace String   ${sample} 192.168.1.10 ${TOOLS_SYSTEM_IP}
Documentation:

Replaces search_for in the given string with replace_with.
Start / End / Elapsed: 20250226 02:29:25.105 / 20250226 02:29:25.105 / 00:00:00.000
00:00:00.000 KEYWORD BuiltIn . Log   ${body}
Documentation:

Logs the given message with the given level.
Start / End / Elapsed: 20250226 02:29:25.105 / 20250226 02:29:25.105 / 00:00:00.000
00:00:00.000 KEYWORD BuiltIn . Log   URL is ${RFC8040_SOUTHBOUND_NODE_API}uuid%2F${ovsdb_uuid}%2Fbridge%2F${BRIDGE}/termination-point=vx2
Documentation:

Logs the given message with the given level.
Start / End / Elapsed: 20250226 02:29:25.105 / 20250226 02:29:25.105 / 00:00:00.000
00:00:00.000 KEYWORD ${port_dictionary} = BuiltIn . Create Dictionary   ${BRIDGE}=1 vx2=3
Documentation:

Creates and returns a dictionary based on the given items.
Start / End / Elapsed: 20250226 02:29:25.106 / 20250226 02:29:25.106 / 00:00:00.000
00:00:00.227 KEYWORD ClusterManagement . Put_As_Json_And_Check_Member_List_Or_All   ${RFC8040_SOUTHBOUND_NODE_API}uuid%2F${ovsdb_uuid}%2Fbridge%2F${BRIDGE}/termination-point=vx2 ${body} ${controller_index} ${controller_index_list}
Documentation:

Send a PUT with the supplied uri ${uri} and body ${data} to member ${member_index}. Then check data is replicated in all or some members defined in ${member_index_list}.
Start / End / Elapsed: 20250226 02:29:25.119 / 20250226 02:29:25.346 / 00:00:00.227
00:00:10.262 KEYWORD BuiltIn . Wait Until Keyword Succeeds   10s 2s ClusterManagement.Check_Item_Occurrence_Member_List_Or_All uri=${RFC8040_SOUTHBOUND_NODE_API}uuid%2F${ovsdb_uuid}%2Fbridge%2F${BRIDGE}/termination-point=vx2?${RFC8040_OPERATIONAL_CONTENT} dictionary=${port_dictionary} member_index_list=${controller_index_list}
Documentation:

Runs the specified keyword and retries if it fails.
Start / End / Elapsed: 20250226 02:29:25.346 / 20250226 02:29:35.608 / 00:00:10.262
Message: 5 failing items removed using the --remove-keywords option.
00:00:00.028 KEYWORD ClusterManagement . Check_Item_Occurrence_Member_List_Or_All   uri=${RFC8040_SOUTHBOUND_NODE_API}uuid%2F${ovsdb_uuid}%2Fbridge%2F${BRIDGE}/termination-point=vx2?${RFC8040_OPERATIONAL_CONTENT} dictionary=${port_dictionary} member_index_list=${controller_index_list}
Documentation:

Send a GET with the supplied uri to all or some members defined in ${member_index_list}. Then check received for occurrences of items expressed in a dictionary ${dictionary}.
Start / End / Elapsed: 20250226 02:29:35.579 / 20250226 02:29:35.607 / 00:00:00.028
00:00:00.001 KEYWORD ${index_list} = ClusterManagement . List_Indices_Or_All   given_list=${member_index_list}
Documentation:

Utility to allow ${EMPTY} as default argument value, as the internal list is computed at runtime. This keyword always returns a (shallow) copy of given or default list, so operations with the returned list should not affect other lists. Also note that this keyword does not consider empty list to be ${EMPTY}.
Start / End / Elapsed: 20250226 02:29:35.580 / 20250226 02:29:35.581 / 00:00:00.001
00:00:00.026 FOR ${index} IN @{index_list}  
Start / End / Elapsed: 20250226 02:29:35.582 / 20250226 02:29:35.608 / 00:00:00.026
00:00:00.025 ITERATION ${index} = 1  
Start / End / Elapsed: 20250226 02:29:35.582 / 20250226 02:29:35.607 / 00:00:00.025
00:00:00.021 KEYWORD ${data} = ClusterManagement . Get_From_Member   uri=${uri} member_index=${index}
Documentation:

Send a GET with the supplied uri to member ${member_index}.
Start / End / Elapsed: 20250226 02:29:35.586 / 20250226 02:29:35.607 / 00:00:00.021
00:00:00.001 KEYWORD ${session} = ClusterManagement . Resolve_Http_Session_For_Member   member_index=${member_index}
Documentation:

Return RequestsLibrary session alias pointing to node of given index.
Start / End / Elapsed: 20250226 02:29:35.586 / 20250226 02:29:35.587 / 00:00:00.001
00:00:00.019 KEYWORD ${response_text} = TemplatedRequests . Get_From_Uri   uri=${uri} accept=${access} session=${session}
Documentation:

GET data from given URI, check status code and return response text. ${accept} is a Python object with headers to use. If ${normalize_json}, normalize as JSON text before returning.
Start / End / Elapsed: 20250226 02:29:35.587 / 20250226 02:29:35.606 / 00:00:00.019
00:00:00.000 KEYWORD BuiltIn . Log   ${uri}
Documentation:

Logs the given message with the given level.
Start / End / Elapsed: 20250226 02:29:35.593 / 20250226 02:29:35.593 / 00:00:00.000
00:00:00.000 KEYWORD BuiltIn . Log   ${accept}
Documentation:

Logs the given message with the given level.
Start / End / Elapsed: 20250226 02:29:35.593 / 20250226 02:29:35.593 / 00:00:00.000
00:00:00.011 IF """${http_timeout}""" == """${EMPTY}"""  
Start / End / Elapsed: 20250226 02:29:35.594 / 20250226 02:29:35.605 / 00:00:00.011
00:00:00.009 KEYWORD ${response} = RequestsLibrary . GET On Session   ${session} url=${uri} headers=${accept}
Documentation:

Sends a GET request on a previously created HTTP Session.
Start / End / Elapsed: 20250226 02:29:35.596 / 20250226 02:29:35.605 / 00:00:00.009
00:00:00.000 ELSE  
Start / End / Elapsed: 20250226 02:29:35.605 / 20250226 02:29:35.605 / 00:00:00.000
00:00:00.000 KEYWORD TemplatedRequests . Check_Status_Code   ${response} log_response=${log_response}
Documentation:

Log response text, check status_code is one of allowed ones. In cases where this keyword is called in a WUKS it could end up logging tons of data and it may be desired to skip the logging by passing log_response=False, but by default it remains True.
Start / End / Elapsed: 20250226 02:29:35.606 / 20250226 02:29:35.606 / 00:00:00.000
00:00:00.000 IF not ${normalize_json}  
Start / End / Elapsed: 20250226 02:29:35.606 / 20250226 02:29:35.606 / 00:00:00.000
00:00:00.000 KEYWORD ${text_normalized} = norm_json . Normalize Json Text   ${response.text} jmes_path=${jmes_path} keys_with_volatiles=${keys_with_volatiles}
Documentation:

Attempt to return sorted indented JSON string.
Start / End / Elapsed: 20250226 02:29:35.606 / 20250226 02:29:35.606 / 00:00:00.000
00:00:00.000 RETURN   ${text_normalized}
Start / End / Elapsed: 20250226 02:29:35.606 / 20250226 02:29:35.606 / 00:00:00.000
00:00:00.000 RETURN   ${response_text}
Start / End / Elapsed: 20250226 02:29:35.606 / 20250226 02:29:35.606 / 00:00:00.000
00:00:00.000 KEYWORD Utils . Check Item Occurrence   ${data} ${dictionary}
Documentation:

Check string for occurrences of items expressed in a list of dictionaries {item=occurrences}. 0 occurences means item is not present.
Start / End / Elapsed: 20250226 02:29:35.607 / 20250226 02:29:35.607 / 00:00:00.000
02:29:35.608 FAIL Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 10 seconds. The last error was: HTTPError: 409 Client Error: Conflict for url: http://10.30.170.253:8181/rests/data/network-topology:network-topology/topology=ovsdb%3A1/node=ovsdb%3A%2F%2Fuuid%2F548ac92b-69d7-4d67-8018-8c0c339a56f8%2Fbridge%2Fbr01/termination-point=vx2?content=nonconfig
00:00:00.662 TEST Modify the destination IP of Port In Owner Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Modify the destination IP of Port In Owner Before Fail
Documentation:

Modify the dst ip of existing port in Owner.
Start / End / Elapsed: 20250226 02:29:35.609 / 20250226 02:29:36.271 / 00:00:00.662
Status: PASS
00:00:05.651 TEST Verify Port Is Modified Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Verify Port Is Modified Before Fail
Documentation:

Verify port is modified in all instances.
Start / End / Elapsed: 20250226 02:29:36.272 / 20250226 02:29:41.923 / 00:00:05.651
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: HTTPError: 409 Client Error: Conflict for url: http://10.30.170.253:8181/rests/data/network-topology:network-topology/topology=ovsdb%3A1/node=ovsdb%3A%2F%2Fuuid%2F548ac92b-69d7-4d67-8018-8c0c339a56f8%2Fbridge%2Fbr01?content=nonconfig
00:00:00.724 TEST Delete Port In Owner Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete Port In Owner Before Fail
Documentation:

Delete port in Owner and verify it gets deleted from all instances.
Start / End / Elapsed: 20250226 02:29:41.923 / 20250226 02:29:42.647 / 00:00:00.724
Status: PASS
00:00:00.652 TEST Delete Bridge In Owner And Verify Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete Bridge In Owner And Verify Before Fail
Documentation:

Delete bridge in Owner and verify it gets deleted from all instances.
Start / End / Elapsed: 20250226 02:29:42.647 / 20250226 02:29:43.299 / 00:00:00.652
Status: PASS
00:00:04.230 TEST Kill Owner Instance
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Kill Owner Instance
Documentation:

Kill Owner Instance and verify it is dead
Start / End / Elapsed: 20250226 02:29:43.299 / 20250226 02:29:47.529 / 00:00:04.230
Status: PASS
00:00:11.857 TEST Check Shards Status After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Check Shards Status After Fail
Documentation:

Create original cluster list and check Status for all shards in Ovsdb application.
Start / End / Elapsed: 20250226 02:29:47.529 / 20250226 02:29:59.386 / 00:00:11.857
Status: PASS
00:00:04.349 TEST Check Entity Owner Status And Find Owner and Candidate After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Check Entity Owner Status And Find Owner and Candidate After Fail
Documentation:

Check Entity Owner Status and identify owner and candidate.
Start / End / Elapsed: 20250226 02:29:59.387 / 20250226 02:30:03.736 / 00:00:04.349
Status: PASS
00:00:06.596 TEST Create Bridge Manually and Verify After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Bridge Manually and Verify After Fail
Documentation:

Create bridge with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250226 02:30:03.737 / 20250226 02:30:10.333 / 00:00:06.596
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"ovsdb:1","node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":56158,"local-ip":"10.30.170.253","remote-ip":"10.30.170.51"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.170.253:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.177:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.224:6640","number_of_connections":1,"connected":true}]}]}]}}' contains 'br-s1' 0 times, not 5 times.
00:00:06.163 TEST Add Port Manually and Verify After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Add Port Manually and Verify After Fail
Documentation:

Add port with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250226 02:30:10.333 / 20250226 02:30:16.496 / 00:00:06.163
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"ovsdb:1","node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":56158,"local-ip":"10.30.170.253","remote-ip":"10.30.170.51"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.170.253:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.177:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.224:6640","number_of_connections":1,"connected":true}]}]}]}}' contains 'vx1' 0 times, not 2 times.
00:00:01.756 TEST Create Tap Device After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Tap Device After Fail
Documentation:

Create tap devices to add to the bridge in ovs
Start / End / Elapsed: 20250226 02:30:16.496 / 20250226 02:30:18.252 / 00:00:01.756
Status: PASS
00:00:06.201 TEST Add Tap Device Manually and Verify After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Add Tap Device Manually and Verify After Fail
Documentation:

Add tap devices to the bridge with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250226 02:30:18.252 / 20250226 02:30:24.453 / 00:00:06.201
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"ovsdb:1","node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":56158,"local-ip":"10.30.170.253","remote-ip":"10.30.170.51"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.170.253:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.177:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.224:6640","number_of_connections":1,"connected":true}]}]}]}}' contains 'vport1' 0 times, not 2 times.
00:00:00.821 TEST Delete the Bridge Manually and Verify After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete the Bridge Manually and Verify After Fail
Documentation:

Delete bridge with OVS command and verify it gets deleted from all instances.
Start / End / Elapsed: 20250226 02:30:24.454 / 20250226 02:30:25.275 / 00:00:00.821
Status: PASS
00:00:10.881 TEST Create Bridge In Owner and Verify After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Bridge In Owner and Verify After Fail
Documentation:

Create Bridge in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250226 02:30:25.275 / 20250226 02:30:36.156 / 00:00:10.881
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 10 seconds. The last error was: '{"network-topology:node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":56158,"local-ip":"10.30.170.253","remote-ip":"10.30.170.51"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.170.253:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.177:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.224:6640","number_of_connections":1,"connected":true}]}]}' contains 'br01' 0 times, not 1 time.
00:00:10.716 TEST Create Port In Owner and Verify After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Port In Owner and Verify After Fail
Documentation:

Create Port in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250226 02:30:36.157 / 20250226 02:30:46.873 / 00:00:10.716
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 10 seconds. The last error was: HTTPError: 409 Client Error: Conflict for url: http://10.30.171.177:8181/rests/data/network-topology:network-topology/topology=ovsdb%3A1/node=ovsdb%3A%2F%2Fuuid%2F548ac92b-69d7-4d67-8018-8c0c339a56f8%2Fbridge%2Fbr01/termination-point=vx2?content=nonconfig
00:00:00.494 TEST Modify the destination IP of Port In Owner After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Modify the destination IP of Port In Owner After Fail
Documentation:

Modify the dst ip of existing port in Owner.
Start / End / Elapsed: 20250226 02:30:46.874 / 20250226 02:30:47.368 / 00:00:00.494
Status: PASS
00:00:05.532 TEST Verify Port Is Modified After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Verify Port Is Modified After Fail
Documentation:

Verify port is modified in all instances.
Start / End / Elapsed: 20250226 02:30:47.368 / 20250226 02:30:52.900 / 00:00:05.532
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: HTTPError: 409 Client Error: Conflict for url: http://10.30.171.177:8181/rests/data/network-topology:network-topology/topology=ovsdb%3A1/node=ovsdb%3A%2F%2Fuuid%2F548ac92b-69d7-4d67-8018-8c0c339a56f8%2Fbridge%2Fbr01?content=nonconfig
00:00:15.051 TEST Start Old Owner Instance
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Start Old Owner Instance
Documentation:

Start Owner Instance and verify it is active
Start / End / Elapsed: 20250226 02:30:52.900 / 20250226 02:31:07.951 / 00:00:15.051
Status: PASS
00:00:04.154 TEST Check Shards Status After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Check Shards Status After Recover
Documentation:

Create original cluster list and check Status for all shards in Ovsdb application.
Start / End / Elapsed: 20250226 02:31:07.951 / 20250226 02:31:12.105 / 00:00:04.154
Status: PASS
00:00:00.698 TEST Check Entity Owner Status After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Check Entity Owner Status After Recover
Documentation:

Check Entity Owner Status and identify owner and candidate.
Start / End / Elapsed: 20250226 02:31:12.106 / 20250226 02:31:12.804 / 00:00:00.698
Status: PASS
00:00:06.821 TEST Create Bridge Manually and Verify After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Bridge Manually and Verify After Recover
Documentation:

Create bridge with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250226 02:31:12.804 / 20250226 02:31:19.625 / 00:00:06.821
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"ovsdb:1","node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":56158,"local-ip":"10.30.170.253","remote-ip":"10.30.170.51"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.170.253:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.177:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.224:6640","number_of_connections":1,"connected":true}]}]}]}}' contains 'br-s1' 0 times, not 5 times.
00:00:06.329 TEST Add Port Manually and Verify After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Add Port Manually and Verify After Recover
Documentation:

Add port with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250226 02:31:19.625 / 20250226 02:31:25.954 / 00:00:06.329
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"ovsdb:1","node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":56158,"local-ip":"10.30.170.253","remote-ip":"10.30.170.51"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.170.253:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.177:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.224:6640","number_of_connections":1,"connected":true}]}]}]}}' contains 'vx1' 0 times, not 2 times.
00:00:01.775 TEST Create Tap Device After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Tap Device After Recover
Documentation:

Create tap devices to add to the bridge in ovs
Start / End / Elapsed: 20250226 02:31:25.954 / 20250226 02:31:27.729 / 00:00:01.775
Status: PASS
00:00:06.391 TEST Add Tap Device Manually and Verify After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Add Tap Device Manually and Verify After Recover
Documentation:

Add tap devices to the bridge with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250226 02:31:27.729 / 20250226 02:31:34.120 / 00:00:06.391
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"ovsdb:1","node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":56158,"local-ip":"10.30.170.253","remote-ip":"10.30.170.51"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.170.253:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.177:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.224:6640","number_of_connections":1,"connected":true}]}]}]}}' contains 'vport1' 0 times, not 2 times.
00:00:00.965 TEST Delete the Bridge Manually and Verify After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete the Bridge Manually and Verify After Recover
Documentation:

Delete bridge with OVS command and verify it gets deleted from all instances.
Start / End / Elapsed: 20250226 02:31:34.121 / 20250226 02:31:35.086 / 00:00:00.965
Status: PASS
00:00:05.776 TEST Verify Modified Port After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Verify Modified Port After Recover
Documentation:

Verify modified port exists in all instances.
Start / End / Elapsed: 20250226 02:31:35.087 / 20250226 02:31:40.863 / 00:00:05.776
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: HTTPError: 409 Client Error: Conflict for url: http://10.30.170.253:8181/rests/data/network-topology:network-topology/topology=ovsdb%3A1/node=ovsdb%3A%2F%2Fuuid%2F548ac92b-69d7-4d67-8018-8c0c339a56f8%2Fbridge%2Fbr01?content=nonconfig
00:00:00.607 TEST Delete Port In New Owner After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete Port In New Owner After Recover
Documentation:

Delete port in Owner and verify it gets deleted from all instances.
Start / End / Elapsed: 20250226 02:31:40.863 / 20250226 02:31:41.470 / 00:00:00.607
Status: PASS
00:00:00.677 TEST Delete Bridge In New Owner And Verify After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete Bridge In New Owner And Verify After Recover
Documentation:

Delete bridge in Owner and verify it gets deleted from all instances.
Start / End / Elapsed: 20250226 02:31:41.471 / 20250226 02:31:42.148 / 00:00:00.677
Status: PASS
00:00:11.416 TEST Create Bridge In Old Owner and Verify After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Bridge In Old Owner and Verify After Recover
Documentation:

Create Bridge in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250226 02:31:42.148 / 20250226 02:31:53.564 / 00:00:11.416
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 10 seconds. The last error was: '{"network-topology:node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":56158,"local-ip":"10.30.170.253","remote-ip":"10.30.170.51"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.170.253:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.177:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.224:6640","number_of_connections":1,"connected":true}]}]}' contains 'br01' 0 times, not 1 time.
00:00:11.173 TEST Create Port In Old Owner and Verify After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Port In Old Owner and Verify After Recover
Documentation:

Create Port in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250226 02:31:53.564 / 20250226 02:32:04.737 / 00:00:11.173
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 10 seconds. The last error was: HTTPError: 409 Client Error: Conflict for url: http://10.30.170.253:8181/rests/data/network-topology:network-topology/topology=ovsdb%3A1/node=ovsdb%3A%2F%2Fuuid%2F548ac92b-69d7-4d67-8018-8c0c339a56f8%2Fbridge%2Fbr01/termination-point=vx2?content=nonconfig
00:00:00.664 TEST Modify the destination IP of Port In Old Owner After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Modify the destination IP of Port In Old Owner After Recover
Documentation:

Modify the dst ip of existing port in Owner.
Start / End / Elapsed: 20250226 02:32:04.739 / 20250226 02:32:05.403 / 00:00:00.664
Status: PASS
00:00:05.628 TEST Verify Port Is Modified After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Verify Port Is Modified After Recover
Documentation:

Verify port is modified in all instances.
Start / End / Elapsed: 20250226 02:32:05.403 / 20250226 02:32:11.031 / 00:00:05.628
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: HTTPError: 409 Client Error: Conflict for url: http://10.30.170.253:8181/rests/data/network-topology:network-topology/topology=ovsdb%3A1/node=ovsdb%3A%2F%2Fuuid%2F548ac92b-69d7-4d67-8018-8c0c339a56f8%2Fbridge%2Fbr01?content=nonconfig
00:00:00.662 TEST Delete Port In Old Owner After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete Port In Old Owner After Recover
Documentation:

Delete port in Owner and verify it gets deleted from all instances.
Start / End / Elapsed: 20250226 02:32:11.032 / 20250226 02:32:11.694 / 00:00:00.662
Status: PASS
00:00:00.645 TEST Delete Bridge In Old Owner And Verify After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete Bridge In Old Owner And Verify After Recover
Documentation:

Delete bridge in Owner and verify it gets deleted from all instances.
Start / End / Elapsed: 20250226 02:32:11.694 / 20250226 02:32:12.339 / 00:00:00.645
Status: PASS
00:00:09.970 TEST Cleans Up Test Environment For Next Suite
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Cleans Up Test Environment For Next Suite
Documentation:

Cleans up test environment, close existing sessions in teardown.
Start / End / Elapsed: 20250226 02:32:12.339 / 20250226 02:32:22.309 / 00:00:09.970
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"ovsdb:1","node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":56158,"local-ip":"10.30.170.253","remote-ip":"10.30.170.51"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.170.253:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.177:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.224:6640","number_of_connections":1,"connected":true}]}]}]}}' contains 'ovsdb://uuid' 1 time, not 0 times.
00:01:58.228 SUITE Southbound Cluster Extension
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Southbound Cluster Extension
Documentation:

Test suite for Ovsdb Southbound Cluster - Candidate failover and recover
Source: /w/workspace/ovsdb-csit-3node-upstream-clustering-only-titanium/test/csit/suites/ovsdb/Southbound_Cluster/020__Southbound_Cluster_Extension.robot
Start / End / Elapsed: 20250226 02:32:22.311 / 20250226 02:34:20.539 / 00:01:58.228
Status: 38 tests total, 5 passed, 33 failed, 0 skipped
00:05:19.308 SUITE Southbound Cluster
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster
Source: /w/workspace/ovsdb-csit-3node-upstream-clustering-only-titanium/test/csit/suites/ovsdb/Southbound_Cluster
Start / End / Elapsed: 20250226 02:34:20.541 / 20250226 02:39:39.849 / 00:05:19.308
Status: 82 tests total, 31 passed, 51 failed, 0 skipped