Test Statistics

Total Statistics
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
All Tests
16445119000:09:30
Statistics by Tag
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
No Tags
Statistics by Suite
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
ovsdb-upstream-clustering.txt
16445119000:09:44
ovsdb-upstream-clustering . txt . Southbound Cluster
823250000:05:28
ovsdb-upstream-clustering . txt . Southbound Cluster . Ovsdb Southbound Cluster
442717000:03:31
ovsdb-upstream-clustering . txt . Southbound Cluster . Southbound Cluster Extension
38533000:01:58
ovsdb-upstream-clustering . txt . Southbound Cluster
821369000:04:16
ovsdb-upstream-clustering . txt . Southbound Cluster . Ovsdb Southbound Cluster
44836000:02:15
ovsdb-upstream-clustering . txt . Southbound Cluster . Southbound Cluster Extension
38533000:02:01

Test Execution Log

00:09:44.280 SUITE ovsdb-upstream-clustering.txt
Full Name: ovsdb-upstream-clustering.txt
Start / End / Elapsed: 20250327 01:06:27.194 / 20250327 01:16:11.474 / 00:09:44.280
Status: 164 tests total, 45 passed, 119 failed, 0 skipped
00:05:28.374 SUITE Southbound Cluster
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster
Source: /w/workspace/ovsdb-csit-3node-upstream-clustering-only-scandium/test/csit/suites/ovsdb/Southbound_Cluster
Start / End / Elapsed: 20250327 01:06:27.218 / 20250327 01:11:55.592 / 00:05:28.374
Status: 82 tests total, 32 passed, 50 failed, 0 skipped
00:03:30.786 SUITE Ovsdb Southbound Cluster
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster
Documentation:

Test suite for Ovsdb Southbound Cluster - Owner failover and recover
Source: /w/workspace/ovsdb-csit-3node-upstream-clustering-only-scandium/test/csit/suites/ovsdb/Southbound_Cluster/010__Ovsdb_Southbound_Cluster.robot
Start / End / Elapsed: 20250327 01:06:27.219 / 20250327 01:09:58.005 / 00:03:30.786
Status: 44 tests total, 27 passed, 17 failed, 0 skipped
00:00:04.499 SETUP Suite Setup  
Start / End / Elapsed: 20250327 01:06:27.720 / 20250327 01:06:32.219 / 00:00:04.499
00:00:00.000 TEARDOWN RequestsLibrary . Delete All Sessions  
Documentation:

Removes all the session objects
Start / End / Elapsed: 20250327 01:09:58.005 / 20250327 01:09:58.005 / 00:00:00.000
00:00:06.053 TEST Check Shards Status Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Check Shards Status Before Fail
Documentation:

Check Status for all shards in Ovsdb application.
Start / End / Elapsed: 20250327 01:06:32.219 / 20250327 01:06:38.272 / 00:00:06.053
Status: PASS
00:00:05.181 TEST Start OVS Multiple Connections
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Start OVS Multiple Connections
Documentation:

Connect OVS to all cluster instances.
Start / End / Elapsed: 20250327 01:06:38.273 / 20250327 01:06:43.454 / 00:00:05.181
Status: PASS
00:00:00.653 TEST Check Entity Owner Status And Find Owner and Candidate Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Check Entity Owner Status And Find Owner and Candidate Before Fail
Documentation:

Check Entity Owner Status and identify owner and candidate.
Start / End / Elapsed: 20250327 01:06:43.454 / 20250327 01:06:44.107 / 00:00:00.653
Status: PASS
00:00:01.233 TEST Create Bridge Manually and Verify Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Bridge Manually and Verify Before Fail
Documentation:

Create bridge with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250327 01:06:44.107 / 20250327 01:06:45.340 / 00:00:01.233
Status: PASS
00:00:01.211 TEST Add Port Manually and Verify Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Add Port Manually and Verify Before Fail
Documentation:

Add port with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250327 01:06:45.341 / 20250327 01:06:46.552 / 00:00:01.211
Status: PASS
00:00:01.994 TEST Create Tap Device Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Tap Device Before Fail
Documentation:

Create tap devices to add to the bridge in ovs
Start / End / Elapsed: 20250327 01:06:46.552 / 20250327 01:06:48.546 / 00:00:01.994
Status: PASS
00:00:01.160 TEST Add Tap Device Manually and Verify Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Add Tap Device Manually and Verify Before Fail
Documentation:

Add tap devices to the bridge with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250327 01:06:48.546 / 20250327 01:06:49.706 / 00:00:01.160
Status: PASS
00:00:01.032 TEST Delete the Bridge Manually and Verify Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete the Bridge Manually and Verify Before Fail
Documentation:

Delete bridge with OVS command and verify it gets deleted from all instances.
Start / End / Elapsed: 20250327 01:06:49.706 / 20250327 01:06:50.738 / 00:00:01.032
Status: PASS
00:00:10.934 TEST Create Bridge In Owner and Verify Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Bridge In Owner and Verify Before Fail
Documentation:

Create Bridge in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250327 01:06:50.738 / 20250327 01:07:01.672 / 00:00:10.934
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 10 seconds. The last error was: '{"network-topology:node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":51690,"local-ip":"10.30.170.168","remote-ip":"10.30.170.191"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.170.168:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.170.61:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.94:6640","number_of_connections":1,"connected":true}]}]}' contains 'br01' 0 times, not 1 time.
00:00:00.378 SETUP SetupUtils . Setup_Test_With_Logging_And_Without_Fast_Failing  
Documentation:

Test case setup which explicitly ignores previous failure and logs test case name to Karaf log. Needed if the recommended default is to be overriden.
Start / End / Elapsed: 20250327 01:06:50.739 / 20250327 01:06:51.117 / 00:00:00.378
00:00:10.547 KEYWORD ClusterOvsdb . Create Sample Bridge And Verify   ${original_owner}
Documentation:

Create bridge br01 in controller ${controller_index} and verify it gets created in all instances in ${controller_index_list}.
Start / End / Elapsed: 20250327 01:06:51.124 / 20250327 01:07:01.671 / 00:00:10.547
00:00:10.907 TEST Create Port In Owner and Verify Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Port In Owner and Verify Before Fail
Documentation:

Create Port in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250327 01:07:01.672 / 20250327 01:07:12.579 / 00:00:10.907
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 10 seconds. The last error was: HTTPError: 409 Client Error: Conflict for url: http://10.30.170.168:8181/rests/data/network-topology:network-topology/topology=ovsdb%3A1/node=ovsdb%3A%2F%2Fuuid%2F548ac92b-69d7-4d67-8018-8c0c339a56f8%2Fbridge%2Fbr01/termination-point=vx2?content=nonconfig
00:00:00.664 TEST Modify the destination IP of Port In Owner Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Modify the destination IP of Port In Owner Before Fail
Documentation:

Modify the dst ip of existing port in Owner.
Start / End / Elapsed: 20250327 01:07:12.580 / 20250327 01:07:13.244 / 00:00:00.664
Status: PASS
00:00:05.648 TEST Verify Port Is Modified Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Verify Port Is Modified Before Fail
Documentation:

Verify port is modified in all instances.
Start / End / Elapsed: 20250327 01:07:13.243 / 20250327 01:07:18.891 / 00:00:05.648
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: HTTPError: 409 Client Error: Conflict for url: http://10.30.170.168:8181/rests/data/network-topology:network-topology/topology=ovsdb%3A1/node=ovsdb%3A%2F%2Fuuid%2F548ac92b-69d7-4d67-8018-8c0c339a56f8%2Fbridge%2Fbr01?content=nonconfig
00:00:00.722 TEST Delete Port In Owner Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete Port In Owner Before Fail
Documentation:

Delete port in Owner and verify it gets deleted from all instances.
Start / End / Elapsed: 20250327 01:07:18.892 / 20250327 01:07:19.614 / 00:00:00.722
Status: PASS
00:00:00.674 TEST Delete Bridge In Owner And Verify Before Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete Bridge In Owner And Verify Before Fail
Documentation:

Delete bridge in Owner and verify it gets deleted from all instances.
Start / End / Elapsed: 20250327 01:07:19.615 / 20250327 01:07:20.289 / 00:00:00.674
Status: PASS
00:00:03.291 TEST Kill Owner Instance
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Kill Owner Instance
Documentation:

Kill Owner Instance and verify it is dead
Start / End / Elapsed: 20250327 01:07:20.289 / 20250327 01:07:23.580 / 00:00:03.291
Status: PASS
00:00:11.822 TEST Check Shards Status After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Check Shards Status After Fail
Documentation:

Create original cluster list and check Status for all shards in Ovsdb application.
Start / End / Elapsed: 20250327 01:07:23.580 / 20250327 01:07:35.402 / 00:00:11.822
Status: PASS
00:00:06.639 TEST Check Entity Owner Status And Find Owner and Candidate After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Check Entity Owner Status And Find Owner and Candidate After Fail
Documentation:

Check Entity Owner Status and identify owner and candidate.
Start / End / Elapsed: 20250327 01:07:35.403 / 20250327 01:07:42.042 / 00:00:06.639
Status: PASS
00:00:06.635 TEST Create Bridge Manually and Verify After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Bridge Manually and Verify After Fail
Documentation:

Create bridge with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250327 01:07:42.043 / 20250327 01:07:48.678 / 00:00:06.635
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"ovsdb:1","node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":51690,"local-ip":"10.30.170.168","remote-ip":"10.30.170.191"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.170.168:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.170.61:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.94:6640","number_of_connections":1,"connected":true}]}]}]}}' contains 'br-s1' 0 times, not 5 times.
00:00:06.218 TEST Add Port Manually and Verify After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Add Port Manually and Verify After Fail
Documentation:

Add port with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250327 01:07:48.678 / 20250327 01:07:54.896 / 00:00:06.218
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"ovsdb:1","node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":51690,"local-ip":"10.30.170.168","remote-ip":"10.30.170.191"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.170.168:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.170.61:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.94:6640","number_of_connections":1,"connected":true}]}]}]}}' contains 'vx1' 0 times, not 2 times.
00:00:01.802 TEST Create Tap Device After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Tap Device After Fail
Documentation:

Create tap devices to add to the bridge in ovs
Start / End / Elapsed: 20250327 01:07:54.896 / 20250327 01:07:56.698 / 00:00:01.802
Status: PASS
00:00:06.167 TEST Add Tap Device Manually and Verify After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Add Tap Device Manually and Verify After Fail
Documentation:

Add tap devices to the bridge with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250327 01:07:56.698 / 20250327 01:08:02.865 / 00:00:06.167
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"ovsdb:1","node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":51690,"local-ip":"10.30.170.168","remote-ip":"10.30.170.191"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.170.168:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.170.61:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.94:6640","number_of_connections":1,"connected":true}]}]}]}}' contains 'vport1' 0 times, not 2 times.
00:00:00.853 TEST Delete the Bridge Manually and Verify After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete the Bridge Manually and Verify After Fail
Documentation:

Delete bridge with OVS command and verify it gets deleted from all instances.
Start / End / Elapsed: 20250327 01:08:02.866 / 20250327 01:08:03.719 / 00:00:00.853
Status: PASS
00:00:10.842 TEST Create Bridge In Owner and Verify After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Bridge In Owner and Verify After Fail
Documentation:

Create Bridge in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250327 01:08:03.719 / 20250327 01:08:14.561 / 00:00:10.842
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 10 seconds. The last error was: '{"network-topology:node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":51690,"local-ip":"10.30.170.168","remote-ip":"10.30.170.191"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.170.168:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.170.61:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.94:6640","number_of_connections":1,"connected":true}]}]}' contains 'br01' 0 times, not 1 time.
00:00:10.745 TEST Create Port In Owner and Verify After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Port In Owner and Verify After Fail
Documentation:

Create Port in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250327 01:08:14.561 / 20250327 01:08:25.306 / 00:00:10.745
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 10 seconds. The last error was: HTTPError: 409 Client Error: Conflict for url: http://10.30.170.61:8181/rests/data/network-topology:network-topology/topology=ovsdb%3A1/node=ovsdb%3A%2F%2Fuuid%2F548ac92b-69d7-4d67-8018-8c0c339a56f8%2Fbridge%2Fbr01/termination-point=vx2?content=nonconfig
00:00:00.493 TEST Modify the destination IP of Port In Owner After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Modify the destination IP of Port In Owner After Fail
Documentation:

Modify the dst ip of existing port in Owner.
Start / End / Elapsed: 20250327 01:08:25.306 / 20250327 01:08:25.799 / 00:00:00.493
Status: PASS
00:00:05.511 TEST Verify Port Is Modified After Fail
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Verify Port Is Modified After Fail
Documentation:

Verify port is modified in all instances.
Start / End / Elapsed: 20250327 01:08:25.799 / 20250327 01:08:31.310 / 00:00:05.511
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: HTTPError: 409 Client Error: Conflict for url: http://10.30.170.61:8181/rests/data/network-topology:network-topology/topology=ovsdb%3A1/node=ovsdb%3A%2F%2Fuuid%2F548ac92b-69d7-4d67-8018-8c0c339a56f8%2Fbridge%2Fbr01?content=nonconfig
00:00:14.030 TEST Start Old Owner Instance
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Start Old Owner Instance
Documentation:

Start Owner Instance and verify it is active
Start / End / Elapsed: 20250327 01:08:31.310 / 20250327 01:08:45.340 / 00:00:14.030
Status: PASS
00:00:03.446 TEST Check Shards Status After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Check Shards Status After Recover
Documentation:

Create original cluster list and check Status for all shards in Ovsdb application.
Start / End / Elapsed: 20250327 01:08:45.341 / 20250327 01:08:48.787 / 00:00:03.446
Status: PASS
00:00:00.704 TEST Check Entity Owner Status After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Check Entity Owner Status After Recover
Documentation:

Check Entity Owner Status and identify owner and candidate.
Start / End / Elapsed: 20250327 01:08:48.787 / 20250327 01:08:49.491 / 00:00:00.704
Status: PASS
00:00:06.765 TEST Create Bridge Manually and Verify After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Bridge Manually and Verify After Recover
Documentation:

Create bridge with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250327 01:08:49.491 / 20250327 01:08:56.256 / 00:00:06.765
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"ovsdb:1","node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":51690,"local-ip":"10.30.170.168","remote-ip":"10.30.170.191"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.170.168:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.170.61:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.94:6640","number_of_connections":1,"connected":true}]}]}]}}' contains 'br-s1' 0 times, not 5 times.
00:00:06.314 TEST Add Port Manually and Verify After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Add Port Manually and Verify After Recover
Documentation:

Add port with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250327 01:08:56.256 / 20250327 01:09:02.570 / 00:00:06.314
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"ovsdb:1","node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":51690,"local-ip":"10.30.170.168","remote-ip":"10.30.170.191"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.170.168:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.170.61:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.94:6640","number_of_connections":1,"connected":true}]}]}]}}' contains 'vx1' 0 times, not 2 times.
00:00:01.867 TEST Create Tap Device After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Tap Device After Recover
Documentation:

Create tap devices to add to the bridge in ovs
Start / End / Elapsed: 20250327 01:09:02.570 / 20250327 01:09:04.437 / 00:00:01.867
Status: PASS
00:00:06.337 TEST Add Tap Device Manually and Verify After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Add Tap Device Manually and Verify After Recover
Documentation:

Add tap devices to the bridge with OVS command and verify it gets applied from all instances.
Start / End / Elapsed: 20250327 01:09:04.437 / 20250327 01:09:10.774 / 00:00:06.337
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"ovsdb:1","node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":51690,"local-ip":"10.30.170.168","remote-ip":"10.30.170.191"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.170.168:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.170.61:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.94:6640","number_of_connections":1,"connected":true}]}]}]}}' contains 'vport1' 0 times, not 2 times.
00:00:00.926 TEST Delete the Bridge Manually and Verify After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete the Bridge Manually and Verify After Recover
Documentation:

Delete bridge with OVS command and verify it gets deleted from all instances.
Start / End / Elapsed: 20250327 01:09:10.775 / 20250327 01:09:11.701 / 00:00:00.926
Status: PASS
00:00:05.642 TEST Verify Modified Port After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Verify Modified Port After Recover
Documentation:

Verify modified port exists in all instances.
Start / End / Elapsed: 20250327 01:09:11.701 / 20250327 01:09:17.343 / 00:00:05.642
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: HTTPError: 409 Client Error: Conflict for url: http://10.30.170.168:8181/rests/data/network-topology:network-topology/topology=ovsdb%3A1/node=ovsdb%3A%2F%2Fuuid%2F548ac92b-69d7-4d67-8018-8c0c339a56f8%2Fbridge%2Fbr01?content=nonconfig
00:00:00.605 TEST Delete Port In New Owner After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete Port In New Owner After Recover
Documentation:

Delete port in Owner and verify it gets deleted from all instances.
Start / End / Elapsed: 20250327 01:09:17.344 / 20250327 01:09:17.949 / 00:00:00.605
Status: PASS
00:00:00.654 TEST Delete Bridge In New Owner And Verify After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete Bridge In New Owner And Verify After Recover
Documentation:

Delete bridge in Owner and verify it gets deleted from all instances.
Start / End / Elapsed: 20250327 01:09:17.950 / 20250327 01:09:18.604 / 00:00:00.654
Status: PASS
00:00:10.948 TEST Create Bridge In Old Owner and Verify After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Bridge In Old Owner and Verify After Recover
Documentation:

Create Bridge in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250327 01:09:18.604 / 20250327 01:09:29.552 / 00:00:10.948
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 10 seconds. The last error was: '{"network-topology:node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":51690,"local-ip":"10.30.170.168","remote-ip":"10.30.170.191"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.170.168:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.170.61:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.94:6640","number_of_connections":1,"connected":true}]}]}' contains 'br01' 0 times, not 1 time.
00:00:10.913 TEST Create Port In Old Owner and Verify After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Create Port In Old Owner and Verify After Recover
Documentation:

Create Port in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250327 01:09:29.552 / 20250327 01:09:40.465 / 00:00:10.913
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 10 seconds. The last error was: HTTPError: 409 Client Error: Conflict for url: http://10.30.170.168:8181/rests/data/network-topology:network-topology/topology=ovsdb%3A1/node=ovsdb%3A%2F%2Fuuid%2F548ac92b-69d7-4d67-8018-8c0c339a56f8%2Fbridge%2Fbr01/termination-point=vx2?content=nonconfig
00:00:00.699 TEST Modify the destination IP of Port In Old Owner After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Modify the destination IP of Port In Old Owner After Recover
Documentation:

Modify the dst ip of existing port in Owner.
Start / End / Elapsed: 20250327 01:09:40.466 / 20250327 01:09:41.165 / 00:00:00.699
Status: PASS
00:00:05.638 TEST Verify Port Is Modified After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Verify Port Is Modified After Recover
Documentation:

Verify port is modified in all instances.
Start / End / Elapsed: 20250327 01:09:41.166 / 20250327 01:09:46.804 / 00:00:05.638
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: HTTPError: 409 Client Error: Conflict for url: http://10.30.170.168:8181/rests/data/network-topology:network-topology/topology=ovsdb%3A1/node=ovsdb%3A%2F%2Fuuid%2F548ac92b-69d7-4d67-8018-8c0c339a56f8%2Fbridge%2Fbr01?content=nonconfig
00:00:00.603 TEST Delete Port In Old Owner After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete Port In Old Owner After Recover
Documentation:

Delete port in Owner and verify it gets deleted from all instances.
Start / End / Elapsed: 20250327 01:09:46.804 / 20250327 01:09:47.407 / 00:00:00.603
Status: PASS
00:00:00.638 TEST Delete Bridge In Old Owner And Verify After Recover
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Delete Bridge In Old Owner And Verify After Recover
Documentation:

Delete bridge in Owner and verify it gets deleted from all instances.
Start / End / Elapsed: 20250327 01:09:47.408 / 20250327 01:09:48.046 / 00:00:00.638
Status: PASS
00:00:09.957 TEST Cleans Up Test Environment For Next Suite
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Ovsdb Southbound Cluster.Cleans Up Test Environment For Next Suite
Documentation:

Cleans up test environment, close existing sessions in teardown.
Start / End / Elapsed: 20250327 01:09:48.047 / 20250327 01:09:58.004 / 00:00:09.957
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 5 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"ovsdb:1","node":[{"node-id":"ovsdb://uuid/548ac92b-69d7-4d67-8018-8c0c339a56f8","ovsdb:db-version":"8.3.0","ovsdb:interface-type-entry":[{"interface-type":"ovsdb:interface-type-vxlan"},{"interface-type":"ovsdb:interface-type-tap"},{"interface-type":"ovsdb:interface-type-stt"},{"interface-type":"ovsdb:interface-type-internal"},{"interface-type":"ovsdb:interface-type-patch"},{"interface-type":"ovsdb:interface-type-lisp"},{"interface-type":"ovsdb:interface-type-geneve"},{"interface-type":"ovsdb:interface-type-gre"},{"interface-type":"ovsdb:interface-type-system"}],"ovsdb:ovs-version":"2.17.11","ovsdb:connection-info":{"local-port":6640,"remote-port":51690,"local-ip":"10.30.170.168","remote-ip":"10.30.170.191"},"ovsdb:openvswitch-external-ids":[{"external-id-key":"system-id","external-id-value":"4cb0bfbc-0bc4-4591-8cc9-bc6146d75221"},{"external-id-key":"hostname","external-id-value":"ubuntu-22-04-mininet-ovs-217-5ebb25bd-57cf-4a0a-aab5-227d5bff4b"},{"external-id-key":"rundir","external-id-value":"/var/run/openvswitch"}],"ovsdb:datapath-type-entry":[{"datapath-type":"ovsdb:datapath-type-netdev"},{"datapath-type":"ovsdb:datapath-type-system"}],"ovsdb:manager-entry":[{"target":"tcp:10.30.170.168:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.170.61:6640","number_of_connections":1,"connected":true},{"target":"tcp:10.30.171.94:6640","number_of_connections":1,"connected":true}]}]}]}}' contains 'ovsdb://uuid' 1 time, not 0 times.
00:01:57.585 SUITE Southbound Cluster Extension
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster.Southbound Cluster Extension
Documentation:

Test suite for Ovsdb Southbound Cluster - Candidate failover and recover
Source: /w/workspace/ovsdb-csit-3node-upstream-clustering-only-scandium/test/csit/suites/ovsdb/Southbound_Cluster/020__Southbound_Cluster_Extension.robot
Start / End / Elapsed: 20250327 01:09:58.006 / 20250327 01:11:55.591 / 00:01:57.585
Status: 38 tests total, 5 passed, 33 failed, 0 skipped
00:04:15.880 SUITE Southbound Cluster
Full Name: ovsdb-upstream-clustering.txt.Southbound Cluster
Source: /w/workspace/ovsdb-csit-3node-upstream-clustering-only-scandium/test/csit/suites/ovsdb/Southbound_Cluster
Start / End / Elapsed: 20250327 01:11:55.593 / 20250327 01:16:11.473 / 00:04:15.880
Status: 82 tests total, 13 passed, 69 failed, 0 skipped