Test Statistics

Total Statistics
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
All Tests
20918425000:24:32
Statistics by Tag
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
https://bugs.opendaylight.org/show_bug.cgi?id=6177
303000:01:01
Statistics by Suite
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
openflowplugin-clustering.txt
20918425000:25:05
openflowplugin-clustering . txt . Cluster HA Owner Failover
51474000:04:31
openflowplugin-clustering . txt . Cluster HA Owner Restart
51483000:04:00
openflowplugin-clustering . txt . Cluster HA Data Recovery Leader Follower Failover
21210000:05:30
openflowplugin-clustering . txt . 010 Group Flows
72657000:09:23
openflowplugin-clustering . txt . 010 Switch Disconnect
303000:00:05
openflowplugin-clustering . txt . 020 Cluster Node Failure
303000:00:04
openflowplugin-clustering . txt . 030 Cluster Sync Problems
505000:01:24
openflowplugin-clustering . txt . 9145
330000:00:09

Test Execution Log

00:25:05.267 SUITE openflowplugin-clustering.txt
Full Name: openflowplugin-clustering.txt
Start / End / Elapsed: 20250210 02:33:05.680 / 20250210 02:58:10.947 / 00:25:05.267
Status: 209 tests total, 184 passed, 25 failed, 0 skipped
00:04:31.006 SUITE Cluster HA Owner Failover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover
Documentation:

Test suite for Cluster HA - Device Owner failover
Source: /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/010__Cluster_HA_Owner_Failover.robot
Start / End / Elapsed: 20250210 02:33:05.706 / 20250210 02:37:36.712 / 00:04:31.006
Status: 51 tests total, 47 passed, 4 failed, 0 skipped
00:00:05.387 SETUP SetupUtils . Setup_Utils_For_Setup_And_Teardown  
Documentation:

Prepare both FailFast and Karaf logging, log that the caller suite has started. This keyword is to be used in suite setup, and without it the other keywords may not work properly.
Start / End / Elapsed: 20250210 02:33:06.240 / 20250210 02:33:11.627 / 00:00:05.387
00:00:00.000 TEARDOWN RequestsLibrary . Delete All Sessions  
Documentation:

Removes all the session objects
Start / End / Elapsed: 20250210 02:37:36.711 / 20250210 02:37:36.711 / 00:00:00.000
00:00:11.658 TEST Check Shards Status Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Check Shards Status Before Fail
Documentation:

Check Status for all shards in OpenFlow application.
Start / End / Elapsed: 20250210 02:33:11.627 / 20250210 02:33:23.285 / 00:00:11.658
Status: PASS
00:00:09.868 TEST Start Mininet Multiple Connections
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Start Mininet Multiple Connections
Documentation:

Start mininet tree,2 with connection to all cluster instances.
Start / End / Elapsed: 20250210 02:33:23.285 / 20250210 02:33:33.153 / 00:00:09.868
Status: PASS
00:00:00.207 TEST Check Entity Owner Status And Find Owner and Successor Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Check Entity Owner Status And Find Owner and Successor Before Fail
Documentation:

Check Entity Owner Status and identify owner and successor for first switch s1.
Start / End / Elapsed: 20250210 02:33:33.153 / 20250210 02:33:33.360 / 00:00:00.207
Status: PASS
00:00:03.921 TEST Reconnect Extra Switches To Successors And Check OVS Connections
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Reconnect Extra Switches To Successors And Check OVS Connections
Documentation:

Connect switches s2 and s3 to successor instances.
Start / End / Elapsed: 20250210 02:33:33.361 / 20250210 02:33:37.282 / 00:00:03.921
Status: PASS
00:00:00.465 TEST Check Network Operational Information Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Check Network Operational Information Before Fail
Documentation:

Check devices in operational inventory and topology in all cluster instances.
Start / End / Elapsed: 20250210 02:33:37.283 / 20250210 02:33:37.748 / 00:00:00.465
Status: PASS
00:00:03.678 TEST Add Configuration In Owner and Verify Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Add Configuration In Owner and Verify Before Fail
Documentation:

Add Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250210 02:33:37.749 / 20250210 02:33:41.427 / 00:00:03.678
Status: PASS
00:00:02.573 TEST Modify Configuration In Owner and Verify Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Modify Configuration In Owner and Verify Before Fail
Documentation:

Modify Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250210 02:33:41.427 / 20250210 02:33:44.000 / 00:00:02.573
Status: PASS
00:00:03.483 TEST Delete Configuration In Owner and Verify Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Delete Configuration In Owner and Verify Before Fail
Documentation:

Delete Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250210 02:33:44.001 / 20250210 02:33:47.484 / 00:00:03.483
Status: PASS
00:00:03.637 TEST Add Configuration In Successor and Verify Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Add Configuration In Successor and Verify Before Fail
Documentation:

Add Flow in Successor and verify it gets applied from all instances.
Start / End / Elapsed: 20250210 02:33:47.484 / 20250210 02:33:51.121 / 00:00:03.637
Status: PASS
00:00:02.521 TEST Modify Configuration In Successor and Verify Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Modify Configuration In Successor and Verify Before Fail
Documentation:

Modify Flow in successor and verify it gets applied from all instances.
Start / End / Elapsed: 20250210 02:33:51.121 / 20250210 02:33:53.642 / 00:00:02.521
Status: PASS
00:00:03.397 TEST Delete Configuration In Successor and Verify Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Delete Configuration In Successor and Verify Before Fail
Documentation:

Delete Flow in Successor and verify it gets removed from all instances.
Start / End / Elapsed: 20250210 02:33:53.643 / 20250210 02:33:57.040 / 00:00:03.397
Status: PASS
00:00:02.747 TEST Send RPC Add to Owner and Verify Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Send RPC Add to Owner and Verify Before Fail
Documentation:

Add Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250210 02:33:57.040 / 20250210 02:33:59.787 / 00:00:02.747
Status: PASS
00:00:02.518 TEST Send RPC Delete to Owner and Verify Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Send RPC Delete to Owner and Verify Before Fail
Documentation:

Delete Flow in Owner and verify it gets removed from all instances.
Start / End / Elapsed: 20250210 02:33:59.787 / 20250210 02:34:02.305 / 00:00:02.518
Status: PASS
00:00:03.566 TEST Send RPC Add to Successor and Verify Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Send RPC Add to Successor and Verify Before Fail
Documentation:

Add Flow in Successor and verify it gets applied from all instances.
Start / End / Elapsed: 20250210 02:34:02.304 / 20250210 02:34:05.870 / 00:00:03.566
Status: PASS
00:00:02.567 TEST Send RPC Delete to Successor and Verify Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Send RPC Delete to Successor and Verify Before Fail
Documentation:

Delete Flow in Successor and verify it gets removed from all instances.
Start / End / Elapsed: 20250210 02:34:05.871 / 20250210 02:34:08.438 / 00:00:02.567
Status: PASS
00:00:20.866 TEST Modify Network And Verify Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Modify Network And Verify Before Fail
Documentation:

Take a link down and verify port status in all instances.
Start / End / Elapsed: 20250210 02:34:08.438 / 20250210 02:34:29.304 / 00:00:20.866
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 20 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"flow:1","node":[{"node-id":"openflow:2","opendaylight-topology-inventory:inventory-node-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']","termination-point":[{"tp-id":"openflow:2:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:LOCAL\']"},{"tp-id":"openflow:2:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:1\']"},{"tp-id":"openflow:2:2","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:2\']"},{"tp-id":"openflow:2:3","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:3\']"}]},{"node-id":"openflow:3","opendaylight-topology-inventory:inventory-node-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']","termination-point":[{"tp-id":"openflow:3:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:LOCAL\']"},{"tp-id":"openflow:3:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/nod... [ Message content over the limit has been removed. ] ...3","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:3\']"}]},{"node-id":"openflow:1","opendaylight-topology-inventory:inventory-node-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']","termination-point":[{"tp-id":"openflow:1:2","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:2\']"},{"tp-id":"openflow:1:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:LOCAL\']"},{"tp-id":"openflow:1:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:1\']"}]}],"link":[{"link-id":"openflow:2:3","source":{"source-node":"openflow:2","source-tp":"openflow:2:3"},"destination":{"dest-tp":"openflow:1:1","dest-node":"openflow:1"}},{"link-id":"openflow:3:3","source":{"source-node":"openflow:3","source-tp":"openflow:3:3"},"destination":{"dest-tp":"openflow:1:2","dest-node":"openflow:1"}},{"link-id":"openflow:1:1","source":{"source-node":"openflow:1","source-tp":"openflow:1:1"},"destination":{"dest-tp":"openflow:2:3","dest-node":"openflow:2"}},{"link-id":"openflow:1:2","source":{"source-node":"openflow:1","source-tp":"openflow:1:2"},"destination":{"dest-tp":"openflow:3:3","dest-node":"openflow:3"}}]}]}}' contains 'openflow:1' 21 times, not 16 times.
00:00:20.862 KEYWORD ClusterOpenFlow . Take OpenFlow Device Link Down and Verify  
Documentation:

Take a link down and verify port status in all instances in ${controller_index_list}.
Start / End / Elapsed: 20250210 02:34:08.442 / 20250210 02:34:29.304 / 00:00:20.862
00:00:00.000 KEYWORD ${dictionary} = BuiltIn . Create Dictionary   "link-down":true=1
Documentation:

Creates and returns a dictionary based on the given items.
Start / End / Elapsed: 20250210 02:34:08.445 / 20250210 02:34:08.445 / 00:00:00.000
00:00:00.013 KEYWORD ${ouput} = MininetKeywords . Send Mininet Command   ${mininet_conn_id} link s1 s2 down
Documentation:

Sends Command ${cmd} to Mininet session ${mininet_conn} and returns read buffer response.
Start / End / Elapsed: 20250210 02:34:08.451 / 20250210 02:34:08.464 / 00:00:00.013
00:00:00.155 KEYWORD BuiltIn . Wait Until Keyword Succeeds   5s 1s ClusterManagement.Check_Item_Occurrence_Member_List_Or_All uri=${operational_port_1} dictionary=${dictionary} member_index_list=${controller_index_list}
Documentation:

Runs the specified keyword and retries if it fails.
Start / End / Elapsed: 20250210 02:34:08.465 / 20250210 02:34:08.620 / 00:00:00.155
00:00:00.000 KEYWORD ${dictionary} = BuiltIn . Create Dictionary   openflow:1=16 openflow:2=14 openflow:3=19
Documentation:

Creates and returns a dictionary based on the given items.
Start / End / Elapsed: 20250210 02:34:08.620 / 20250210 02:34:08.620 / 00:00:00.000
00:00:20.684 KEYWORD BuiltIn . Wait Until Keyword Succeeds   20s 2s ClusterManagement.Check_Item_Occurrence_Member_List_Or_All uri=${RFC8040_OPERATIONAL_TOPO_API} dictionary=${dictionary} member_index_list=${controller_index_list}
Documentation:

Runs the specified keyword and retries if it fails.
Start / End / Elapsed: 20250210 02:34:08.621 / 20250210 02:34:29.305 / 00:00:20.684
Message: 10 failing items removed using the --remove-keywords option.
00:00:00.331 TEST Restore Network And Verify Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Restore Network And Verify Before Fail
Documentation:

Take the link up and verify port status in all instances.
Start / End / Elapsed: 20250210 02:34:29.305 / 20250210 02:34:29.636 / 00:00:00.331
Status: PASS
00:00:04.361 TEST Kill Owner Instance
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Kill Owner Instance
Documentation:

Kill Owner Instance and verify it is dead
Start / End / Elapsed: 20250210 02:34:29.637 / 20250210 02:34:33.998 / 00:00:04.361
Status: PASS
00:00:02.462 TEST Check Shards Status After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Check Shards Status After Fail
Documentation:

Create original cluster list and check Status for all shards in OpenFlow application.
Start / End / Elapsed: 20250210 02:34:33.998 / 20250210 02:34:36.460 / 00:00:02.462
Status: PASS
00:00:13.839 TEST Check Entity Owner Status And Find Owner and Successor After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Check Entity Owner Status And Find Owner and Successor After Fail
Documentation:

Check Entity Owner Status and identify owner and successor.
Start / End / Elapsed: 20250210 02:34:36.460 / 20250210 02:34:50.299 / 00:00:13.839
Status: PASS
00:00:00.263 TEST Check Network Operational Information After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Check Network Operational Information After Fail
Documentation:

Check devices in operational inventory and topology in all cluster instances.
Start / End / Elapsed: 20250210 02:34:50.299 / 20250210 02:34:50.562 / 00:00:00.263
Status: PASS
00:00:03.473 TEST Add Configuration In Owner and Verify After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Add Configuration In Owner and Verify After Fail
Documentation:

Add Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250210 02:34:50.562 / 20250210 02:34:54.035 / 00:00:03.473
Status: PASS
00:00:02.398 TEST Modify Configuration In Owner and Verify After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Modify Configuration In Owner and Verify After Fail
Documentation:

Modify Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250210 02:34:54.035 / 20250210 02:34:56.433 / 00:00:02.398
Status: PASS
00:00:03.307 TEST Delete Configuration In Owner and Verify After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Delete Configuration In Owner and Verify After Fail
Documentation:

Delete Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250210 02:34:56.433 / 20250210 02:34:59.740 / 00:00:03.307
Status: PASS
00:00:03.492 TEST Add Configuration In Successor and Verify After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Add Configuration In Successor and Verify After Fail
Documentation:

Add Flow in Successor and verify it gets applied from all instances.
Start / End / Elapsed: 20250210 02:34:59.740 / 20250210 02:35:03.232 / 00:00:03.492
Status: PASS
00:00:02.412 TEST Modify Configuration In Successor and Verify After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Modify Configuration In Successor and Verify After Fail
Documentation:

Modify Flow in Successor and verify it gets applied from all instances.
Start / End / Elapsed: 20250210 02:35:03.232 / 20250210 02:35:05.644 / 00:00:02.412
Status: PASS
00:00:03.348 TEST Delete Configuration In Successor and Verify After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Delete Configuration In Successor and Verify After Fail
Documentation:

Delete Flow in Successor and verify it gets applied from all instances.
Start / End / Elapsed: 20250210 02:35:05.644 / 20250210 02:35:08.992 / 00:00:03.348
Status: PASS
00:00:02.718 TEST Send RPC Add to Owner and Verify After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Send RPC Add to Owner and Verify After Fail
Documentation:

Add Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250210 02:35:08.993 / 20250210 02:35:11.711 / 00:00:02.718
Status: PASS
00:00:03.710 TEST Send RPC Delete to Owner and Verify After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Send RPC Delete to Owner and Verify After Fail
Documentation:

Delete Flow in Owner and verify it gets removed from all instances.
Start / End / Elapsed: 20250210 02:35:11.711 / 20250210 02:35:15.421 / 00:00:03.710
Status: PASS
00:00:02.855 TEST Send RPC Add to Successor and Verify After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Send RPC Add to Successor and Verify After Fail
Documentation:

Add Flow in Successor and verify it gets applied from all instances.
Start / End / Elapsed: 20250210 02:35:15.422 / 20250210 02:35:18.277 / 00:00:02.855
Status: PASS
00:00:02.523 TEST Send RPC Delete to Successor and Verify After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Send RPC Delete to Successor and Verify After Fail
Documentation:

Delete Flow in Successor and verify it gets removed from all instances.
Start / End / Elapsed: 20250210 02:35:18.277 / 20250210 02:35:20.800 / 00:00:02.523
Status: PASS
00:00:20.806 TEST Modify Network and Verify After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Modify Network and Verify After Fail
Documentation:

Take a link down and verify port status in all instances.
Start / End / Elapsed: 20250210 02:35:20.800 / 20250210 02:35:41.606 / 00:00:20.806
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 20 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"flow:1","node":[{"node-id":"openflow:2","opendaylight-topology-inventory:inventory-node-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']","termination-point":[{"tp-id":"openflow:2:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:LOCAL\']"},{"tp-id":"openflow:2:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:1\']"},{"tp-id":"openflow:2:2","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:2\']"},{"tp-id":"openflow:2:3","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:3\']"}]},{"node-id":"openflow:3","opendaylight-topology-inventory:inventory-node-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']","termination-point":[{"tp-id":"openflow:3:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:LOCAL\']"},{"tp-id":"openflow:3:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/nod... [ Message content over the limit has been removed. ] ...3","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:3\']"}]},{"node-id":"openflow:1","opendaylight-topology-inventory:inventory-node-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']","termination-point":[{"tp-id":"openflow:1:2","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:2\']"},{"tp-id":"openflow:1:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:LOCAL\']"},{"tp-id":"openflow:1:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:1\']"}]}],"link":[{"link-id":"openflow:2:3","source":{"source-node":"openflow:2","source-tp":"openflow:2:3"},"destination":{"dest-tp":"openflow:1:1","dest-node":"openflow:1"}},{"link-id":"openflow:3:3","source":{"source-node":"openflow:3","source-tp":"openflow:3:3"},"destination":{"dest-tp":"openflow:1:2","dest-node":"openflow:1"}},{"link-id":"openflow:1:1","source":{"source-node":"openflow:1","source-tp":"openflow:1:1"},"destination":{"dest-tp":"openflow:2:3","dest-node":"openflow:2"}},{"link-id":"openflow:1:2","source":{"source-node":"openflow:1","source-tp":"openflow:1:2"},"destination":{"dest-tp":"openflow:3:3","dest-node":"openflow:3"}}]}]}}' contains 'openflow:1' 21 times, not 16 times.
00:00:00.216 TEST Restore Network and Verify After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Restore Network and Verify After Fail
Documentation:

Take the link up and verify port status in all instances.
Start / End / Elapsed: 20250210 02:35:41.607 / 20250210 02:35:41.823 / 00:00:00.216
Status: PASS
00:00:35.444 TEST Start Old Owner Instance
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Start Old Owner Instance
Documentation:

Start old Owner Instance and verify it is up
Start / End / Elapsed: 20250210 02:35:41.824 / 20250210 02:36:17.268 / 00:00:35.444
Status: PASS
00:00:05.520 TEST Check Shards Status After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Check Shards Status After Recover
Documentation:

Create original cluster list and check Status for all shards in OpenFlow application.
Start / End / Elapsed: 20250210 02:36:17.268 / 20250210 02:36:22.788 / 00:00:05.520
Status: PASS
00:00:00.210 TEST Check Entity Owner Status After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Check Entity Owner Status After Recover
Documentation:

Check Entity Owner Status and identify owner and successor.
Start / End / Elapsed: 20250210 02:36:22.788 / 20250210 02:36:22.998 / 00:00:00.210
Status: PASS
00:00:00.450 TEST Check Network Operational Information After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Check Network Operational Information After Recover
Documentation:

Check devices in operational inventory and topology in all cluster instances.
Start / End / Elapsed: 20250210 02:36:22.998 / 20250210 02:36:23.448 / 00:00:00.450
Status: PASS
00:00:01.491 TEST Add Configuration In Owner and Verify After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Add Configuration In Owner and Verify After Recover
Documentation:

Add Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250210 02:36:23.448 / 20250210 02:36:24.939 / 00:00:01.491
Status: PASS
00:00:02.504 TEST Modify Configuration In Owner and Verify After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Modify Configuration In Owner and Verify After Recover
Documentation:

Modify Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250210 02:36:24.940 / 20250210 02:36:27.444 / 00:00:02.504
Status: PASS
00:00:03.481 TEST Delete Configuration In Owner and Verify After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Delete Configuration In Owner and Verify After Recover
Documentation:

Delete Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250210 02:36:27.444 / 20250210 02:36:30.925 / 00:00:03.481
Status: PASS
00:00:02.520 TEST Add Configuration In Old Owner and Verify After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Add Configuration In Old Owner and Verify After Recover
Documentation:

Add Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250210 02:36:30.925 / 20250210 02:36:33.445 / 00:00:02.520
Status: PASS
00:00:03.562 TEST Modify Configuration In Old Owner and Verify After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Modify Configuration In Old Owner and Verify After Recover
Documentation:

Modify Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250210 02:36:33.445 / 20250210 02:36:37.007 / 00:00:03.562
Status: PASS
00:00:03.403 TEST Delete Configuration In Old Owner and Verify After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Delete Configuration In Old Owner and Verify After Recover
Documentation:

Delete Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250210 02:36:37.007 / 20250210 02:36:40.410 / 00:00:03.403
Status: PASS
00:00:02.871 TEST Send RPC Add to Owner and Verify After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Send RPC Add to Owner and Verify After Recover
Documentation:

Add Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250210 02:36:40.411 / 20250210 02:36:43.282 / 00:00:02.871
Status: PASS
00:00:02.881 TEST Send RPC Delete to Owner and Verify After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Send RPC Delete to Owner and Verify After Recover
Documentation:

Delete Flow in Owner and verify it gets removed from all instances.
Start / End / Elapsed: 20250210 02:36:43.282 / 20250210 02:36:46.163 / 00:00:02.881
Status: PASS
00:00:02.451 TEST Send RPC Add to Old Owner and Verify After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Send RPC Add to Old Owner and Verify After Recover
Documentation:

Add Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250210 02:36:46.163 / 20250210 02:36:48.614 / 00:00:02.451
Status: PASS
00:00:03.558 TEST Send RPC Delete to Old Owner and Verify After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Send RPC Delete to Old Owner and Verify After Recover
Documentation:

Delete Flow in Owner and verify it gets removed from all instances.
Start / End / Elapsed: 20250210 02:36:48.614 / 20250210 02:36:52.172 / 00:00:03.558
Status: PASS
00:00:20.839 TEST Modify Network and Verify After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Modify Network and Verify After Recover
Documentation:

Take a link down and verify port status in all instances.
Start / End / Elapsed: 20250210 02:36:52.172 / 20250210 02:37:13.011 / 00:00:20.839
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 20 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"flow:1","link":[{"link-id":"openflow:2:3","source":{"source-node":"openflow:2","source-tp":"openflow:2:3"},"destination":{"dest-tp":"openflow:1:1","dest-node":"openflow:1"}},{"link-id":"openflow:3:3","source":{"source-node":"openflow:3","source-tp":"openflow:3:3"},"destination":{"dest-tp":"openflow:1:2","dest-node":"openflow:1"}},{"link-id":"openflow:1:1","source":{"source-node":"openflow:1","source-tp":"openflow:1:1"},"destination":{"dest-tp":"openflow:2:3","dest-node":"openflow:2"}},{"link-id":"openflow:1:2","source":{"source-node":"openflow:1","source-tp":"openflow:1:2"},"destination":{"dest-tp":"openflow:3:3","dest-node":"openflow:3"}}],"node":[{"node-id":"openflow:2","opendaylight-topology-inventory:inventory-node-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']","termination-point":[{"tp-id":"openflow:2:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:LOCAL\']"},{"tp-id":"openflow:2:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:1\']"},{"tp-id":"openflow:2:2","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:2\']"},{"tp... [ Message content over the limit has been removed. ] ...ndaylight-inventory:nodes/node[id=\'openflow:3\']","termination-point":[{"tp-id":"openflow:3:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:LOCAL\']"},{"tp-id":"openflow:3:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:1\']"},{"tp-id":"openflow:3:2","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:2\']"},{"tp-id":"openflow:3:3","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:3\']"}]},{"node-id":"openflow:1","opendaylight-topology-inventory:inventory-node-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']","termination-point":[{"tp-id":"openflow:1:2","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:2\']"},{"tp-id":"openflow:1:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:LOCAL\']"},{"tp-id":"openflow:1:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:1\']"}]}]}]}}' contains 'openflow:1' 21 times, not 16 times.
00:00:00.306 TEST Restore Network and Verify After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Restore Network and Verify After Recover
Documentation:

Take the link up and verify port status in all instances.
Start / End / Elapsed: 20250210 02:37:13.011 / 20250210 02:37:13.317 / 00:00:00.306
Status: PASS
00:00:02.587 TEST Stop Mininet and Exit
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Stop Mininet and Exit
Documentation:

Stop mininet and exit connection.
Start / End / Elapsed: 20250210 02:37:13.317 / 20250210 02:37:15.904 / 00:00:02.587
Status: PASS
00:00:20.804 TEST Check No Network Operational Information
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Check No Network Operational Information
Documentation:

Check device is not in operational inventory or topology in all cluster instances.
Start / End / Elapsed: 20250210 02:37:15.905 / 20250210 02:37:36.709 / 00:00:20.804
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 20 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"flow:1","link":[{"link-id":"openflow:2:3","source":{"source-node":"openflow:2","source-tp":"openflow:2:3"},"destination":{"dest-tp":"openflow:1:1","dest-node":"openflow:1"}},{"link-id":"openflow:3:3","source":{"source-node":"openflow:3","source-tp":"openflow:3:3"},"destination":{"dest-tp":"openflow:1:2","dest-node":"openflow:1"}},{"link-id":"openflow:1:1","source":{"source-node":"openflow:1","source-tp":"openflow:1:1"},"destination":{"dest-tp":"openflow:2:3","dest-node":"openflow:2"}},{"link-id":"openflow:1:2","source":{"source-node":"openflow:1","source-tp":"openflow:1:2"},"destination":{"dest-tp":"openflow:3:3","dest-node":"openflow:3"}}]}]}}' contains 'openflow' 20 times, not 0 times.
00:04:00.212 SUITE Cluster HA Owner Restart
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Restart
Documentation:

Test suite for Cluster HA - Device Owner Stop and Start
Source: /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/020__Cluster_HA_Owner_Restart.robot
Start / End / Elapsed: 20250210 02:37:36.713 / 20250210 02:41:36.925 / 00:04:00.212
Status: 51 tests total, 48 passed, 3 failed, 0 skipped
00:05:29.614 SUITE Cluster HA Data Recovery Leader Follower Failover
Full Name: openflowplugin-clustering.txt.Cluster HA Data Recovery Leader Follower Failover
Documentation:

Test suite for Cluster HA - Data Recovery at Leader Follower failover and cluster restart
Source: /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/030__Cluster_HA_Data_Recovery_Leader_Follower_Failover.robot
Start / End / Elapsed: 20250210 02:41:36.925 / 20250210 02:47:06.539 / 00:05:29.614
Status: 21 tests total, 21 passed, 0 failed, 0 skipped
00:09:22.748 SUITE 010 Group Flows
Full Name: openflowplugin-clustering.txt.010 Group Flows
Documentation:

Switch connections and cluster are restarted.
Source: /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustered_Reconciliation/010_Group_Flows.robot
Start / End / Elapsed: 20250210 02:47:06.540 / 20250210 02:56:29.288 / 00:09:22.748
Status: 72 tests total, 65 passed, 7 failed, 0 skipped
00:00:04.918 SUITE 010 Switch Disconnect
Full Name: openflowplugin-clustering.txt.010 Switch Disconnect
Documentation:

Test suite for entity ownership service and openflowplugin. Makes changes on switch side.
Source: /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/010_Switch_Disconnect.robot
Start / End / Elapsed: 20250210 02:56:29.289 / 20250210 02:56:34.207 / 00:00:04.918
Status: 3 tests total, 0 passed, 3 failed, 0 skipped
Message: Suite setup failed: Dictionary does not contain key 's1'.
00:00:04.296 SUITE 020 Cluster Node Failure
Full Name: openflowplugin-clustering.txt.020 Cluster Node Failure
Documentation:

Test suite for entity ownership service and openflowplugin. Makes changes on controller side (restart karaf)
Source: /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/020_Cluster_Node_Failure.robot
Start / End / Elapsed: 20250210 02:56:34.209 / 20250210 02:56:38.505 / 00:00:04.296
Status: 3 tests total, 0 passed, 3 failed, 0 skipped
Message: Suite setup failed: Dictionary does not contain key 's1'.
00:01:23.827 SUITE 030 Cluster Sync Problems
Full Name: openflowplugin-clustering.txt.030 Cluster Sync Problems
Documentation:

Test suite for entity ownership service and openflowplugin. Makes changes on controller side (isolating cluster node)
Source: /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/030_Cluster_Sync_Problems.robot
Start / End / Elapsed: 20250210 02:56:38.506 / 20250210 02:58:02.333 / 00:01:23.827
Status: 5 tests total, 0 passed, 5 failed, 0 skipped
00:00:08.613 SUITE 9145
Full Name: openflowplugin-clustering.txt.9145
Documentation:

Switch connections and cluster are restarted.
Source: /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Bug_Validation/9145.robot
Start / End / Elapsed: 20250210 02:58:02.333 / 20250210 02:58:10.946 / 00:00:08.613
Status: 3 tests total, 3 passed, 0 failed, 0 skipped