Test Statistics

Total Statistics
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
All Tests
16455109000:08:34
Statistics by Tag
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
No Tags
Statistics by Suite
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
ovsdb-upstream-clustering.txt
16455109000:08:48
ovsdb-upstream-clustering . txt . Southbound Cluster
823646000:05:09
ovsdb-upstream-clustering . txt . Southbound Cluster . Ovsdb Southbound Cluster
442717000:03:32
ovsdb-upstream-clustering . txt . Southbound Cluster . Southbound Cluster Extension
38929000:01:37
ovsdb-upstream-clustering . txt . Southbound Cluster
821963000:03:40
ovsdb-upstream-clustering . txt . Southbound Cluster . Ovsdb Southbound Cluster
441430000:01:44
ovsdb-upstream-clustering . txt . Southbound Cluster . Southbound Cluster Extension
38533000:01:56

Test Execution Log

00:08:48.216 SUITE ovsdb-upstream-clustering.txt
Full Name: ovsdb-upstream-clustering.txt
Start / End / Elapsed: 20250306 01:18:49.101 / 20250306 01:27:37.317 / 00:08:48.216
Status: 164 tests total, 55 passed, 109 failed, 0 skipped
00:05:08.501 SUITE Southbound Cluster
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster
Source: /w/workspace/ovsdb-csit-3node-upstream-clustering-only-scandium/test/csit/suites/ovsdb/Southbound_Cluster
Start / End / Elapsed: 20250306 01:18:49.125 / 20250306 01:23:57.626 / 00:05:08.501
Status: 82 tests total, 36 passed, 46 failed, 0 skipped
00:03:31.827 SUITE Ovsdb Southbound Cluster
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster
Documentation:

Test suite for Ovsdb Southbound Cluster - Owner failover and recover
Source: /w/workspace/ovsdb-csit-3node-upstream-clustering-only-scandium/test/csit/suites/ovsdb/Southbound_Cluster/010__Ovsdb_Southbound_Cluster.robot
Start / End / Elapsed: 20250306 01:18:49.126 / 20250306 01:22:20.953 / 00:03:31.827
Status: 44 tests total, 27 passed, 17 failed, 0 skipped
00:00:04.896 SETUP Suite Setup  
Start / End / Elapsed: 20250306 01:18:49.625 / 20250306 01:18:54.521 / 00:00:04.896
00:00:00.000 TEARDOWN RequestsLibrary . Delete All Sessions  
Documentation:

Removes all the session objects
Start / End / Elapsed: 20250306 01:22:20.953 / 20250306 01:22:20.953 / 00:00:00.000
00:00:06.483 TEST Check Shards Status Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Check Shards Status Before Fail
Documentation:

Check Status for all shards in Ovsdb application.
Start / End / Elapsed: 20250306 01:18:54.521 / 20250306 01:19:01.004 / 00:00:06.483
Status: PASS
00:00:04.239 TEST Start OVS Multiple Connections
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Start OVS Multiple Connections
Documentation:

Connect OVS to all cluster instances.
Start / End / Elapsed: 20250306 01:19:01.004 / 20250306 01:19:05.243 / 00:00:04.239
Status: PASS
00:00:00.748 TEST Check Entity Owner Status And Find Owner and Candidate Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Check Entity Owner Status And Find Owner and Candidate Before Fail
Documentation:

Check Entity Owner Status and identify owner and candidate.
Start / End / Elapsed: 20250306 01:19:05.244 / 20250306 01:19:05.992 / 00:00:00.748
Status: PASS
00:00:01.231 TEST Create Bridge Manually and Verify Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Bridge Manually and Verify Before Fail
Documentation:

Create bridge with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250306 01:19:05.991 / 20250306 01:19:07.222 / 00:00:01.231
Status: PASS
00:00:01.266 TEST Add Port Manually and Verify Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Add Port Manually and Verify Before Fail
Documentation:

Add port with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250306 01:19:07.223 / 20250306 01:19:08.489 / 00:00:01.266
Status: PASS
00:00:01.929 TEST Create Tap Device Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Tap Device Before Fail
Documentation:

Create tap devices to add to the bridge in ovs
Start / End / Elapsed: 20250306 01:19:08.489 / 20250306 01:19:10.418 / 00:00:01.929
Status: PASS
00:00:01.177 TEST Add Tap Device Manually and Verify Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Add Tap Device Manually and Verify Before Fail
Documentation:

Add tap devices to the bridge with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250306 01:19:10.418 / 20250306 01:19:11.595 / 00:00:01.177
Status: PASS
00:00:01.129 TEST Delete the Bridge Manually and Verify Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete the Bridge Manually and Verify Before Fail
Documentation:

Delete bridge with OVS command and verify it gets deleted from all instances.
Start / End / Elapsed: 20250306 01:19:11.596 / 20250306 01:19:12.725 / 00:00:01.129
Status: PASS
00:00:11.001 TEST Create Bridge In Owner and Verify Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Bridge In Owner and Verify Before Fail
Documentation:

Create Bridge in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250306 01:19:12.725 / 20250306 01:19:23.726 / 00:00:11.001
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 10 seconds. The last error was: '{"network-topology:node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":52540,"local-ip":"10.30.171.24","remote-ip":"10.30.170.219"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.171.24:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.222:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.71:6640","number_of_connections":1,"connected":true}]}]}' contains 'br01' 0 times, not 1 time.
00:00:00.389 SETUP SetupUtils . Setup_Test_With_Logging_And_Without_Fast_Failing  
Documentation:

Test case setup which explicitly ignores previous failure and logs test case name to Karaf log. Needed if the recommended default is to be overriden.
Start / End / Elapsed: 20250306 01:19:12.726 / 20250306 01:19:13.115 / 00:00:00.389
00:00:10.603 KEYWORD ClusterOvsdb . Create Sample Bridge And Verify   ${original_owner}
Documentation:

Create bridge br01 in controller ${controller_index} and verify it gets created in all instances in ${controller_index_list}.
Start / End / Elapsed: 20250306 01:19:13.122 / 20250306 01:19:23.725 / 00:00:10.603
00:00:10.911 TEST Create Port In Owner and Verify Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Port In Owner and Verify Before Fail
Documentation:

Create Port in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250306 01:19:23.726 / 20250306 01:19:34.637 / 00:00:10.911
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 10 seconds. The last error was: HTTPError: 409 Client Error: Conflict for url: http://10.30.171.24:8181/rests/data/network-topology:network-topology/topology=ovsdb%3A1/node=ovsdb%3A%2F%2Fuuid%2F548ac92b-69d7-4d67-8018-8c0c339a56f8%2Fbridge%2Fbr01/termination-point=vx2?content=nonconfig
00:00:00.696 TEST Modify the destination IP of Port In Owner Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Modify the destination IP of Port In Owner Before Fail
Documentation:

Modify the dst ip of existing port in Owner.
Start / End / Elapsed: 20250306 01:19:34.638 / 20250306 01:19:35.334 / 00:00:00.696
Status: PASS
00:00:05.652 TEST Verify Port Is Modified Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Verify Port Is Modified Before Fail
Documentation:

Verify port is modified in all instances.
Start / End / Elapsed: 20250306 01:19:35.333 / 20250306 01:19:40.985 / 00:00:05.652
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: HTTPError: 409 Client Error: Conflict for url: http://10.30.171.24:8181/rests/data/network-topology:network-topology/topology=ovsdb%3A1/node=ovsdb%3A%2F%2Fuuid%2F548ac92b-69d7-4d67-8018-8c0c339a56f8%2Fbridge%2Fbr01?content=nonconfig
00:00:00.689 TEST Delete Port In Owner Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete Port In Owner Before Fail
Documentation:

Delete port in Owner and verify it gets deleted from all instances.
Start / End / Elapsed: 20250306 01:19:40.985 / 20250306 01:19:41.674 / 00:00:00.689
Status: PASS
00:00:00.666 TEST Delete Bridge In Owner And Verify Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete Bridge In Owner And Verify Before Fail
Documentation:

Delete bridge in Owner and verify it gets deleted from all instances.
Start / End / Elapsed: 20250306 01:19:41.674 / 20250306 01:19:42.340 / 00:00:00.666
Status: PASS
00:00:03.805 TEST Kill Owner Instance
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Kill Owner Instance
Documentation:

Kill Owner Instance and verify it is dead
Start / End / Elapsed: 20250306 01:19:42.341 / 20250306 01:19:46.146 / 00:00:03.805
Status: PASS
00:00:11.773 TEST Check Shards Status After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Check Shards Status After Fail
Documentation:

Create original cluster list and check Status for all shards in Ovsdb application.
Start / End / Elapsed: 20250306 01:19:46.146 / 20250306 01:19:57.919 / 00:00:11.773
Status: PASS
00:00:06.631 TEST Check Entity Owner Status And Find Owner and Candidate After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Check Entity Owner Status And Find Owner and Candidate After Fail
Documentation:

Check Entity Owner Status and identify owner and candidate.
Start / End / Elapsed: 20250306 01:19:57.919 / 20250306 01:20:04.550 / 00:00:06.631
Status: PASS
00:00:06.579 TEST Create Bridge Manually and Verify After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Bridge Manually and Verify After Fail
Documentation:

Create bridge with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250306 01:20:04.550 / 20250306 01:20:11.129 / 00:00:06.579
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"ovsdb:1","node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":52540,"local-ip":"10.30.171.24","remote-ip":"10.30.170.219"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.171.24:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.222:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.71:6640","number_of_connections":1,"connected":true}]}]}]}}' contains 'br-s1' 0 times, not 5 times.
00:00:06.194 TEST Add Port Manually and Verify After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Add Port Manually and Verify After Fail
Documentation:

Add port with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250306 01:20:11.129 / 20250306 01:20:17.323 / 00:00:06.194
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"ovsdb:1","node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":52540,"local-ip":"10.30.171.24","remote-ip":"10.30.170.219"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.171.24:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.222:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.71:6640","number_of_connections":1,"connected":true}]}]}]}}' contains 'vx1' 0 times, not 2 times.
00:00:01.806 TEST Create Tap Device After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Tap Device After Fail
Documentation:

Create tap devices to add to the bridge in ovs
Start / End / Elapsed: 20250306 01:20:17.324 / 20250306 01:20:19.130 / 00:00:01.806
Status: PASS
00:00:06.212 TEST Add Tap Device Manually and Verify After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Add Tap Device Manually and Verify After Fail
Documentation:

Add tap devices to the bridge with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250306 01:20:19.130 / 20250306 01:20:25.342 / 00:00:06.212
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"ovsdb:1","node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":52540,"local-ip":"10.30.171.24","remote-ip":"10.30.170.219"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.171.24:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.222:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.71:6640","number_of_connections":1,"connected":true}]}]}]}}' contains 'vport1' 0 times, not 2 times.
00:00:00.789 TEST Delete the Bridge Manually and Verify After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete the Bridge Manually and Verify After Fail
Documentation:

Delete bridge with OVS command and verify it gets deleted from all instances.
Start / End / Elapsed: 20250306 01:20:25.342 / 20250306 01:20:26.131 / 00:00:00.789
Status: PASS
00:00:10.855 TEST Create Bridge In Owner and Verify After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Bridge In Owner and Verify After Fail
Documentation:

Create Bridge in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250306 01:20:26.131 / 20250306 01:20:36.986 / 00:00:10.855
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 10 seconds. The last error was: '{"network-topology:node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":52540,"local-ip":"10.30.171.24","remote-ip":"10.30.170.219"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.171.24:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.222:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.71:6640","number_of_connections":1,"connected":true}]}]}' contains 'br01' 0 times, not 1 time.
00:00:10.742 TEST Create Port In Owner and Verify After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Port In Owner and Verify After Fail
Documentation:

Create Port in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250306 01:20:36.987 / 20250306 01:20:47.729 / 00:00:10.742
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 10 seconds. The last error was: HTTPError: 409 Client Error: Conflict for url: http://10.30.171.71:8181/rests/data/network-topology:network-topology/topology=ovsdb%3A1/node=ovsdb%3A%2F%2Fuuid%2F548ac92b-69d7-4d67-8018-8c0c339a56f8%2Fbridge%2Fbr01/termination-point=vx2?content=nonconfig
00:00:00.475 TEST Modify the destination IP of Port In Owner After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Modify the destination IP of Port In Owner After Fail
Documentation:

Modify the dst ip of existing port in Owner.
Start / End / Elapsed: 20250306 01:20:47.730 / 20250306 01:20:48.205 / 00:00:00.475
Status: PASS
00:00:05.558 TEST Verify Port Is Modified After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Verify Port Is Modified After Fail
Documentation:

Verify port is modified in all instances.
Start / End / Elapsed: 20250306 01:20:48.205 / 20250306 01:20:53.763 / 00:00:05.558
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: HTTPError: 409 Client Error: Conflict for url: http://10.30.171.71:8181/rests/data/network-topology:network-topology/topology=ovsdb%3A1/node=ovsdb%3A%2F%2Fuuid%2F548ac92b-69d7-4d67-8018-8c0c339a56f8%2Fbridge%2Fbr01?content=nonconfig
00:00:14.724 TEST Start Old Owner Instance
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Start Old Owner Instance
Documentation:

Start Owner Instance and verify it is active
Start / End / Elapsed: 20250306 01:20:53.764 / 20250306 01:21:08.488 / 00:00:14.724
Status: PASS
00:00:03.376 TEST Check Shards Status After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Check Shards Status After Recover
Documentation:

Create original cluster list and check Status for all shards in Ovsdb application.
Start / End / Elapsed: 20250306 01:21:08.488 / 20250306 01:21:11.864 / 00:00:03.376
Status: PASS
00:00:00.563 TEST Check Entity Owner Status After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Check Entity Owner Status After Recover
Documentation:

Check Entity Owner Status and identify owner and candidate.
Start / End / Elapsed: 20250306 01:21:11.864 / 20250306 01:21:12.427 / 00:00:00.563
Status: PASS
00:00:06.699 TEST Create Bridge Manually and Verify After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Bridge Manually and Verify After Recover
Documentation:

Create bridge with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250306 01:21:12.427 / 20250306 01:21:19.126 / 00:00:06.699
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"ovsdb:1","node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":52540,"local-ip":"10.30.171.24","remote-ip":"10.30.170.219"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.171.24:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.222:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.71:6640","number_of_connections":1,"connected":true}]}]}]}}' contains 'br-s1' 0 times, not 5 times.
00:00:06.363 TEST Add Port Manually and Verify After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Add Port Manually and Verify After Recover
Documentation:

Add port with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250306 01:21:19.126 / 20250306 01:21:25.489 / 00:00:06.363
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"ovsdb:1","node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":52540,"local-ip":"10.30.171.24","remote-ip":"10.30.170.219"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.171.24:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.222:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.71:6640","number_of_connections":1,"connected":true}]}]}]}}' contains 'vx1' 0 times, not 2 times.
00:00:01.753 TEST Create Tap Device After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Tap Device After Recover
Documentation:

Create tap devices to add to the bridge in ovs
Start / End / Elapsed: 20250306 01:21:25.489 / 20250306 01:21:27.242 / 00:00:01.753
Status: PASS
00:00:06.326 TEST Add Tap Device Manually and Verify After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Add Tap Device Manually and Verify After Recover
Documentation:

Add tap devices to the bridge with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250306 01:21:27.242 / 20250306 01:21:33.568 / 00:00:06.326
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"ovsdb:1","node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":52540,"local-ip":"10.30.171.24","remote-ip":"10.30.170.219"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.171.24:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.222:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.71:6640","number_of_connections":1,"connected":true}]}]}]}}' contains 'vport1' 0 times, not 2 times.
00:00:00.925 TEST Delete the Bridge Manually and Verify After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete the Bridge Manually and Verify After Recover
Documentation:

Delete bridge with OVS command and verify it gets deleted from all instances.
Start / End / Elapsed: 20250306 01:21:33.569 / 20250306 01:21:34.494 / 00:00:00.925
Status: PASS
00:00:05.696 TEST Verify Modified Port After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Verify Modified Port After Recover
Documentation:

Verify modified port exists in all instances.
Start / End / Elapsed: 20250306 01:21:34.494 / 20250306 01:21:40.190 / 00:00:05.696
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: HTTPError: 409 Client Error: Conflict for url: http://10.30.171.24:8181/rests/data/network-topology:network-topology/topology=ovsdb%3A1/node=ovsdb%3A%2F%2Fuuid%2F548ac92b-69d7-4d67-8018-8c0c339a56f8%2Fbridge%2Fbr01?content=nonconfig
00:00:00.687 TEST Delete Port In New Owner After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete Port In New Owner After Recover
Documentation:

Delete port in Owner and verify it gets deleted from all instances.
Start / End / Elapsed: 20250306 01:21:40.190 / 20250306 01:21:40.877 / 00:00:00.687
Status: PASS
00:00:00.646 TEST Delete Bridge In New Owner And Verify After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete Bridge In New Owner And Verify After Recover
Documentation:

Delete bridge in Owner and verify it gets deleted from all instances.
Start / End / Elapsed: 20250306 01:21:40.877 / 20250306 01:21:41.523 / 00:00:00.646
Status: PASS
00:00:10.986 TEST Create Bridge In Old Owner and Verify After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Bridge In Old Owner and Verify After Recover
Documentation:

Create Bridge in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250306 01:21:41.523 / 20250306 01:21:52.509 / 00:00:10.986
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 10 seconds. The last error was: '{"network-topology:node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":52540,"local-ip":"10.30.171.24","remote-ip":"10.30.170.219"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.171.24:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.222:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.71:6640","number_of_connections":1,"connected":true}]}]}' contains 'br01' 0 times, not 1 time.
00:00:10.876 TEST Create Port In Old Owner and Verify After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Port In Old Owner and Verify After Recover
Documentation:

Create Port in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250306 01:21:52.510 / 20250306 01:22:03.386 / 00:00:10.876
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 10 seconds. The last error was: HTTPError: 409 Client Error: Conflict for url: http://10.30.171.24:8181/rests/data/network-topology:network-topology/topology=ovsdb%3A1/node=ovsdb%3A%2F%2Fuuid%2F548ac92b-69d7-4d67-8018-8c0c339a56f8%2Fbridge%2Fbr01/termination-point=vx2?content=nonconfig
00:00:00.695 TEST Modify the destination IP of Port In Old Owner After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Modify the destination IP of Port In Old Owner After Recover
Documentation:

Modify the dst ip of existing port in Owner.
Start / End / Elapsed: 20250306 01:22:03.387 / 20250306 01:22:04.082 / 00:00:00.695
Status: PASS
00:00:05.608 TEST Verify Port Is Modified After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Verify Port Is Modified After Recover
Documentation:

Verify port is modified in all instances.
Start / End / Elapsed: 20250306 01:22:04.082 / 20250306 01:22:09.690 / 00:00:05.608
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: HTTPError: 409 Client Error: Conflict for url: http://10.30.171.24:8181/rests/data/network-topology:network-topology/topology=ovsdb%3A1/node=ovsdb%3A%2F%2Fuuid%2F548ac92b-69d7-4d67-8018-8c0c339a56f8%2Fbridge%2Fbr01?content=nonconfig
00:00:00.671 TEST Delete Port In Old Owner After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete Port In Old Owner After Recover
Documentation:

Delete port in Owner and verify it gets deleted from all instances.
Start / End / Elapsed: 20250306 01:22:09.690 / 20250306 01:22:10.361 / 00:00:00.671
Status: PASS
00:00:00.661 TEST Delete Bridge In Old Owner And Verify After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete Bridge In Old Owner And Verify After Recover
Documentation:

Delete bridge in Owner and verify it gets deleted from all instances.
Start / End / Elapsed: 20250306 01:22:10.361 / 20250306 01:22:11.022 / 00:00:00.661
Status: PASS
00:00:09.930 TEST Cleans Up Test Environment For Next Suite
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Cleans Up Test Environment For Next Suite
Documentation:

Cleans up test environment, close existing sessions in teardown.
Start / End / Elapsed: 20250306 01:22:11.022 / 20250306 01:22:20.952 / 00:00:09.930
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"ovsdb:1","node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":52540,"local-ip":"10.30.171.24","remote-ip":"10.30.170.219"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.171.24:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.222:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.71:6640","number_of_connections":1,"connected":true}]}]}]}}' contains 'ovsdb://uuid' 1 time, not 0 times.
00:01:36.671 SUITE Southbound Cluster Extension
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Southbound Cluster Extension
Documentation:

Test suite for Ovsdb Southbound Cluster - Candidate failover and recover
Source: /w/workspace/ovsdb-csit-3node-upstream-clustering-only-scandium/test/csit/suites/ovsdb/Southbound_Cluster/020__Southbound_Cluster_Extension.robot
Start / End / Elapsed: 20250306 01:22:20.954 / 20250306 01:23:57.625 / 00:01:36.671
Status: 38 tests total, 9 passed, 29 failed, 0 skipped
00:03:39.689 SUITE Southbound Cluster
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster
Source: /w/workspace/ovsdb-csit-3node-upstream-clustering-only-scandium/test/csit/suites/ovsdb/Southbound_Cluster
Start / End / Elapsed: 20250306 01:23:57.627 / 20250306 01:27:37.316 / 00:03:39.689
Status: 82 tests total, 19 passed, 63 failed, 0 skipped