Test Statistics

Total Statistics
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
All Tests
20918722000:23:57
Statistics by Tag
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
https://bugs.opendaylight.org/show_bug.cgi?id=6177
303000:01:01
Statistics by Suite
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
openflowplugin-clustering.txt
20918722000:24:28
openflowplugin-clustering . txt . Cluster HA Owner Failover
51474000:04:32
openflowplugin-clustering . txt . Cluster HA Owner Restart
51501000:03:16
openflowplugin-clustering . txt . Cluster HA Data Recovery Leader Follower Failover
21210000:05:37
openflowplugin-clustering . txt . 010 Group Flows
72666000:09:22
openflowplugin-clustering . txt . 010 Switch Disconnect
303000:00:05
openflowplugin-clustering . txt . 020 Cluster Node Failure
303000:00:04
openflowplugin-clustering . txt . 030 Cluster Sync Problems
505000:01:24
openflowplugin-clustering . txt . 9145
330000:00:09

Test Execution Log

00:24:27.746 SUITE openflowplugin-clustering.txt
Full Name: openflowplugin-clustering.txt
Start / End / Elapsed: 20250309 02:28:46.341 / 20250309 02:53:14.087 / 00:24:27.746
Status: 209 tests total, 187 passed, 22 failed, 0 skipped
00:04:31.735 SUITE Cluster HA Owner Failover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover
Documentation:

Test suite for Cluster HA - Device Owner failover
Source: /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/010__Cluster_HA_Owner_Failover.robot
Start / End / Elapsed: 20250309 02:28:46.364 / 20250309 02:33:18.099 / 00:04:31.735
Status: 51 tests total, 47 passed, 4 failed, 0 skipped
00:00:04.811 SETUP SetupUtils . Setup_Utils_For_Setup_And_Teardown  
Documentation:

Prepare both FailFast and Karaf logging, log that the caller suite has started. This keyword is to be used in suite setup, and without it the other keywords may not work properly.
Start / End / Elapsed: 20250309 02:28:46.856 / 20250309 02:28:51.667 / 00:00:04.811
00:00:00.000 TEARDOWN RequestsLibrary . Delete All Sessions  
Documentation:

Removes all the session objects
Start / End / Elapsed: 20250309 02:33:18.098 / 20250309 02:33:18.098 / 00:00:00.000
00:00:09.974 TEST Check Shards Status Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Check Shards Status Before Fail
Documentation:

Check Status for all shards in OpenFlow application.
Start / End / Elapsed: 20250309 02:28:51.667 / 20250309 02:29:01.641 / 00:00:09.974
Status: PASS
00:00:09.752 TEST Start Mininet Multiple Connections
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Start Mininet Multiple Connections
Documentation:

Start mininet tree,2 with connection to all cluster instances.
Start / End / Elapsed: 20250309 02:29:01.641 / 20250309 02:29:11.393 / 00:00:09.752
Status: PASS
00:00:00.191 TEST Check Entity Owner Status And Find Owner and Successor Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Check Entity Owner Status And Find Owner and Successor Before Fail
Documentation:

Check Entity Owner Status and identify owner and successor for first switch s1.
Start / End / Elapsed: 20250309 02:29:11.393 / 20250309 02:29:11.584 / 00:00:00.191
Status: PASS
00:00:04.363 TEST Reconnect Extra Switches To Successors And Check OVS Connections
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Reconnect Extra Switches To Successors And Check OVS Connections
Documentation:

Connect switches s2 and s3 to successor instances.
Start / End / Elapsed: 20250309 02:29:11.583 / 20250309 02:29:15.946 / 00:00:04.363
Status: PASS
00:00:00.469 TEST Check Network Operational Information Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Check Network Operational Information Before Fail
Documentation:

Check devices in operational inventory and topology in all cluster instances.
Start / End / Elapsed: 20250309 02:29:15.946 / 20250309 02:29:16.415 / 00:00:00.469
Status: PASS
00:00:03.577 TEST Add Configuration In Owner and Verify Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Add Configuration In Owner and Verify Before Fail
Documentation:

Add Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250309 02:29:16.416 / 20250309 02:29:19.993 / 00:00:03.577
Status: PASS
00:00:02.489 TEST Modify Configuration In Owner and Verify Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Modify Configuration In Owner and Verify Before Fail
Documentation:

Modify Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250309 02:29:19.993 / 20250309 02:29:22.482 / 00:00:02.489
Status: PASS
00:00:03.384 TEST Delete Configuration In Owner and Verify Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Delete Configuration In Owner and Verify Before Fail
Documentation:

Delete Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250309 02:29:22.482 / 20250309 02:29:25.866 / 00:00:03.384
Status: PASS
00:00:03.569 TEST Add Configuration In Successor and Verify Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Add Configuration In Successor and Verify Before Fail
Documentation:

Add Flow in Successor and verify it gets applied from all instances.
Start / End / Elapsed: 20250309 02:29:25.866 / 20250309 02:29:29.435 / 00:00:03.569
Status: PASS
00:00:02.472 TEST Modify Configuration In Successor and Verify Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Modify Configuration In Successor and Verify Before Fail
Documentation:

Modify Flow in successor and verify it gets applied from all instances.
Start / End / Elapsed: 20250309 02:29:29.436 / 20250309 02:29:31.908 / 00:00:02.472
Status: PASS
00:00:03.379 TEST Delete Configuration In Successor and Verify Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Delete Configuration In Successor and Verify Before Fail
Documentation:

Delete Flow in Successor and verify it gets removed from all instances.
Start / End / Elapsed: 20250309 02:29:31.908 / 20250309 02:29:35.287 / 00:00:03.379
Status: PASS
00:00:02.867 TEST Send RPC Add to Owner and Verify Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Send RPC Add to Owner and Verify Before Fail
Documentation:

Add Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250309 02:29:35.287 / 20250309 02:29:38.154 / 00:00:02.867
Status: PASS
00:00:03.381 TEST Send RPC Delete to Owner and Verify Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Send RPC Delete to Owner and Verify Before Fail
Documentation:

Delete Flow in Owner and verify it gets removed from all instances.
Start / End / Elapsed: 20250309 02:29:38.154 / 20250309 02:29:41.535 / 00:00:03.381
Status: PASS
00:00:02.521 TEST Send RPC Add to Successor and Verify Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Send RPC Add to Successor and Verify Before Fail
Documentation:

Add Flow in Successor and verify it gets applied from all instances.
Start / End / Elapsed: 20250309 02:29:41.536 / 20250309 02:29:44.057 / 00:00:02.521
Status: PASS
00:00:03.571 TEST Send RPC Delete to Successor and Verify Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Send RPC Delete to Successor and Verify Before Fail
Documentation:

Delete Flow in Successor and verify it gets removed from all instances.
Start / End / Elapsed: 20250309 02:29:44.057 / 20250309 02:29:47.628 / 00:00:03.571
Status: PASS
00:00:20.789 TEST Modify Network And Verify Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Modify Network And Verify Before Fail
Documentation:

Take a link down and verify port status in all instances.
Start / End / Elapsed: 20250309 02:29:47.628 / 20250309 02:30:08.417 / 00:00:20.789
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 20 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"flow:1","node":[{"node-id":"openflow:2","opendaylight-topology-inventory:inventory-node-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']","termination-point":[{"tp-id":"openflow:2:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:LOCAL\']"},{"tp-id":"openflow:2:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:1\']"},{"tp-id":"openflow:2:2","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:2\']"},{"tp-id":"openflow:2:3","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:3\']"}]},{"node-id":"openflow:3","opendaylight-topology-inventory:inventory-node-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']","termination-point":[{"tp-id":"openflow:3:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:LOCAL\']"},{"tp-id":"openflow:3:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/nod... [ Message content over the limit has been removed. ] ...3","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:3\']"}]},{"node-id":"openflow:1","opendaylight-topology-inventory:inventory-node-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']","termination-point":[{"tp-id":"openflow:1:2","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:2\']"},{"tp-id":"openflow:1:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:LOCAL\']"},{"tp-id":"openflow:1:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:1\']"}]}],"link":[{"link-id":"openflow:2:3","source":{"source-node":"openflow:2","source-tp":"openflow:2:3"},"destination":{"dest-tp":"openflow:1:1","dest-node":"openflow:1"}},{"link-id":"openflow:3:3","source":{"source-node":"openflow:3","source-tp":"openflow:3:3"},"destination":{"dest-tp":"openflow:1:2","dest-node":"openflow:1"}},{"link-id":"openflow:1:1","source":{"source-node":"openflow:1","source-tp":"openflow:1:1"},"destination":{"dest-tp":"openflow:2:3","dest-node":"openflow:2"}},{"link-id":"openflow:1:2","source":{"source-node":"openflow:1","source-tp":"openflow:1:2"},"destination":{"dest-tp":"openflow:3:3","dest-node":"openflow:3"}}]}]}}' contains 'openflow:1' 21 times, not 16 times.
00:00:20.785 KEYWORD ClusterOpenFlow . Take OpenFlow Device Link Down and Verify  
Documentation:

Take a link down and verify port status in all instances in ${controller_index_list}.
Start / End / Elapsed: 20250309 02:29:47.632 / 20250309 02:30:08.417 / 00:00:20.785
00:00:00.000 KEYWORD ${dictionary} = BuiltIn . Create Dictionary   "link-down":true=1
Documentation:

Creates and returns a dictionary based on the given items.
Start / End / Elapsed: 20250309 02:29:47.634 / 20250309 02:29:47.634 / 00:00:00.000
00:00:00.011 KEYWORD ${ouput} = MininetKeywords . Send Mininet Command   ${mininet_conn_id} link s1 s2 down
Documentation:

Sends Command ${cmd} to Mininet session ${mininet_conn} and returns read buffer response.
Start / End / Elapsed: 20250309 02:29:47.641 / 20250309 02:29:47.652 / 00:00:00.011
00:00:00.141 KEYWORD BuiltIn . Wait Until Keyword Succeeds   5s 1s ClusterManagement.Check_Item_Occurrence_Member_List_Or_All uri=${operational_port_1} dictionary=${dictionary} member_index_list=${controller_index_list}
Documentation:

Runs the specified keyword and retries if it fails.
Start / End / Elapsed: 20250309 02:29:47.652 / 20250309 02:29:47.793 / 00:00:00.141
00:00:00.000 KEYWORD ${dictionary} = BuiltIn . Create Dictionary   openflow:1=16 openflow:2=14 openflow:3=19
Documentation:

Creates and returns a dictionary based on the given items.
Start / End / Elapsed: 20250309 02:29:47.794 / 20250309 02:29:47.794 / 00:00:00.000
00:00:20.623 KEYWORD BuiltIn . Wait Until Keyword Succeeds   20s 2s ClusterManagement.Check_Item_Occurrence_Member_List_Or_All uri=${RFC8040_OPERATIONAL_TOPO_API} dictionary=${dictionary} member_index_list=${controller_index_list}
Documentation:

Runs the specified keyword and retries if it fails.
Start / End / Elapsed: 20250309 02:29:47.794 / 20250309 02:30:08.417 / 00:00:20.623
Message: 10 failing items removed using the --remove-keywords option.
00:00:00.055 KEYWORD ClusterManagement . Check_Item_Occurrence_Member_List_Or_All   uri=${RFC8040_OPERATIONAL_TOPO_API} dictionary=${dictionary} member_index_list=${controller_index_list}
Documentation:

Send a GET with the supplied uri to all or some members defined in ${member_index_list}. Then check received for occurrences of items expressed in a dictionary {'openflow:1': '16', 'openflow:2': '14', 'openflow:3': '19'}.
Start / End / Elapsed: 20250309 02:30:08.362 / 20250309 02:30:08.417 / 00:00:00.055
00:00:00.003 KEYWORD ${index_list} = ClusterManagement . List_Indices_Or_All   given_list=${member_index_list}
Documentation:

Utility to allow ${EMPTY} as default argument value, as the internal list is computed at runtime. This keyword always returns a (shallow) copy of given or default list, so operations with the returned list should not affect other lists. Also note that this keyword does not consider empty list to be ${EMPTY}.
Start / End / Elapsed: 20250309 02:30:08.363 / 20250309 02:30:08.366 / 00:00:00.003
00:00:00.050 FOR ${index} IN @{index_list}  
Start / End / Elapsed: 20250309 02:30:08.367 / 20250309 02:30:08.417 / 00:00:00.050
00:00:00.050 ITERATION ${index} = 1  
Start / End / Elapsed: 20250309 02:30:08.367 / 20250309 02:30:08.417 / 00:00:00.050
00:00:00.043 KEYWORD ${data} = ClusterManagement . Get_From_Member   uri=${uri} member_index=${index}
Documentation:

Send a GET with the supplied uri to member ${member_index}.
Start / End / Elapsed: 20250309 02:30:08.372 / 20250309 02:30:08.415 / 00:00:00.043
00:00:00.001 KEYWORD Utils . Check Item Occurrence   ${data} ${dictionary}
Documentation:

Check string for occurrences of items expressed in a list of dictionaries {item=occurrences}. 0 occurences means item is not present.
Start / End / Elapsed: 20250309 02:30:08.415 / 20250309 02:30:08.416 / 00:00:00.001
00:00:00.001 FOR ${item} IN @{dictionary_item_occurrence}  
Start / End / Elapsed: 20250309 02:30:08.415 / 20250309 02:30:08.416 / 00:00:00.001
00:00:00.001 ITERATION ${item} = openflow:1  
Start / End / Elapsed: 20250309 02:30:08.416 / 20250309 02:30:08.417 / 00:00:00.001
00:00:00.000 KEYWORD BuiltIn . Should Contain X Times   ${string} ${item} ${dictionary_item_occurrence}[${item}]
Documentation:

Fails if container does not contain item count times.
Start / End / Elapsed: 20250309 02:30:08.416 / 20250309 02:30:08.416 / 00:00:00.000
02:30:08.416 INFO Item found from container 21 times.
02:30:08.416 FAIL '{"network-topology:network-topology":{"topology":[{"topology-id":"flow:1","node":[{"node-id":"openflow:2","opendaylight-topology-inventory:inventory-node-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']","termination-point":[{"tp-id":"openflow:2:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:LOCAL\']"},{"tp-id":"openflow:2:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:1\']"},{"tp-id":"openflow:2:2","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:2\']"},{"tp-id":"openflow:2:3","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:3\']"}]},{"node-id":"openflow:3","opendaylight-topology-inventory:inventory-node-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']","termination-point":[{"tp-id":"openflow:3:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:LOCAL\']"},{"tp-id":"openflow:3:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:1\']"},{"tp-id":"openflow:3:2","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:2\']"},{"tp-id":"openflow:3:3","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:3\']"}]},{"node-id":"openflow:1","opendaylight-topology-inventory:inventory-node-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']","termination-point":[{"tp-id":"openflow:1:2","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:2\']"},{"tp-id":"openflow:1:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:LOCAL\']"},{"tp-id":"openflow:1:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:1\']"}]}],"link":[{"link-id":"openflow:2:3","source":{"source-node":"openflow:2","source-tp":"openflow:2:3"},"destination":{"dest-tp":"openflow:1:1","dest-node":"openflow:1"}},{"link-id":"openflow:3:3","source":{"source-node":"openflow:3","source-tp":"openflow:3:3"},"destination":{"dest-tp":"openflow:1:2","dest-node":"openflow:1"}},{"link-id":"openflow:1:1","source":{"source-node":"openflow:1","source-tp":"openflow:1:1"},"destination":{"dest-tp":"openflow:2:3","dest-node":"openflow:2"}},{"link-id":"openflow:1:2","source":{"source-node":"openflow:1","source-tp":"openflow:1:2"},"destination":{"dest-tp":"openflow:3:3","dest-node":"openflow:3"}}]}]}}' contains 'openflow:1' 21 times, not 16 times.
02:30:08.417 FAIL Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 20 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"flow:1","node":[{"node-id":"openflow:2","opendaylight-topology-inventory:inventory-node-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']","termination-point":[{"tp-id":"openflow:2:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:LOCAL\']"},{"tp-id":"openflow:2:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:1\']"},{"tp-id":"openflow:2:2","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:2\']"},{"tp-id":"openflow:2:3","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:3\']"}]},{"node-id":"openflow:3","opendaylight-topology-inventory:inventory-node-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']","termination-point":[{"tp-id":"openflow:3:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:LOCAL\']"},{"tp-id":"openflow:3:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:1\']"},{"tp-id":"openflow:3:2","opendaylight-topology-inventory:inventory-n... [ Message content over the limit has been removed. ] ...3","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:3\']"}]},{"node-id":"openflow:1","opendaylight-topology-inventory:inventory-node-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']","termination-point":[{"tp-id":"openflow:1:2","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:2\']"},{"tp-id":"openflow:1:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:LOCAL\']"},{"tp-id":"openflow:1:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:1\']"}]}],"link":[{"link-id":"openflow:2:3","source":{"source-node":"openflow:2","source-tp":"openflow:2:3"},"destination":{"dest-tp":"openflow:1:1","dest-node":"openflow:1"}},{"link-id":"openflow:3:3","source":{"source-node":"openflow:3","source-tp":"openflow:3:3"},"destination":{"dest-tp":"openflow:1:2","dest-node":"openflow:1"}},{"link-id":"openflow:1:1","source":{"source-node":"openflow:1","source-tp":"openflow:1:1"},"destination":{"dest-tp":"openflow:2:3","dest-node":"openflow:2"}},{"link-id":"openflow:1:2","source":{"source-node":"openflow:1","source-tp":"openflow:1:2"},"destination":{"dest-tp":"openflow:3:3","dest-node":"openflow:3"}}]}]}}' contains 'openflow:1' 21 times, not 16 times.
00:00:00.307 TEST Restore Network And Verify Before Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Restore Network And Verify Before Fail
Documentation:

Take the link up and verify port status in all instances.
Start / End / Elapsed: 20250309 02:30:08.418 / 20250309 02:30:08.725 / 00:00:00.307
Status: PASS
00:00:04.264 TEST Kill Owner Instance
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Kill Owner Instance
Documentation:

Kill Owner Instance and verify it is dead
Start / End / Elapsed: 20250309 02:30:08.725 / 20250309 02:30:12.989 / 00:00:04.264
Status: PASS
00:00:02.238 TEST Check Shards Status After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Check Shards Status After Fail
Documentation:

Create original cluster list and check Status for all shards in OpenFlow application.
Start / End / Elapsed: 20250309 02:30:12.989 / 20250309 02:30:15.227 / 00:00:02.238
Status: PASS
00:00:15.030 TEST Check Entity Owner Status And Find Owner and Successor After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Check Entity Owner Status And Find Owner and Successor After Fail
Documentation:

Check Entity Owner Status and identify owner and successor.
Start / End / Elapsed: 20250309 02:30:15.228 / 20250309 02:30:30.258 / 00:00:15.030
Status: PASS
00:00:00.253 TEST Check Network Operational Information After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Check Network Operational Information After Fail
Documentation:

Check devices in operational inventory and topology in all cluster instances.
Start / End / Elapsed: 20250309 02:30:30.258 / 20250309 02:30:30.511 / 00:00:00.253
Status: PASS
00:00:03.437 TEST Add Configuration In Owner and Verify After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Add Configuration In Owner and Verify After Fail
Documentation:

Add Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250309 02:30:30.511 / 20250309 02:30:33.948 / 00:00:03.437
Status: PASS
00:00:03.443 TEST Modify Configuration In Owner and Verify After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Modify Configuration In Owner and Verify After Fail
Documentation:

Modify Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250309 02:30:33.948 / 20250309 02:30:37.391 / 00:00:03.443
Status: PASS
00:00:02.245 TEST Delete Configuration In Owner and Verify After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Delete Configuration In Owner and Verify After Fail
Documentation:

Delete Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250309 02:30:37.391 / 20250309 02:30:39.636 / 00:00:02.245
Status: PASS
00:00:03.460 TEST Add Configuration In Successor and Verify After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Add Configuration In Successor and Verify After Fail
Documentation:

Add Flow in Successor and verify it gets applied from all instances.
Start / End / Elapsed: 20250309 02:30:39.636 / 20250309 02:30:43.096 / 00:00:03.460
Status: PASS
00:00:02.386 TEST Modify Configuration In Successor and Verify After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Modify Configuration In Successor and Verify After Fail
Documentation:

Modify Flow in Successor and verify it gets applied from all instances.
Start / End / Elapsed: 20250309 02:30:43.097 / 20250309 02:30:45.483 / 00:00:02.386
Status: PASS
00:00:03.275 TEST Delete Configuration In Successor and Verify After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Delete Configuration In Successor and Verify After Fail
Documentation:

Delete Flow in Successor and verify it gets applied from all instances.
Start / End / Elapsed: 20250309 02:30:45.483 / 20250309 02:30:48.758 / 00:00:03.275
Status: PASS
00:00:03.420 TEST Send RPC Add to Owner and Verify After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Send RPC Add to Owner and Verify After Fail
Documentation:

Add Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250309 02:30:48.758 / 20250309 02:30:52.178 / 00:00:03.420
Status: PASS
00:00:02.517 TEST Send RPC Delete to Owner and Verify After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Send RPC Delete to Owner and Verify After Fail
Documentation:

Delete Flow in Owner and verify it gets removed from all instances.
Start / End / Elapsed: 20250309 02:30:52.178 / 20250309 02:30:54.695 / 00:00:02.517
Status: PASS
00:00:03.815 TEST Send RPC Add to Successor and Verify After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Send RPC Add to Successor and Verify After Fail
Documentation:

Add Flow in Successor and verify it gets applied from all instances.
Start / End / Elapsed: 20250309 02:30:54.696 / 20250309 02:30:58.511 / 00:00:03.815
Status: PASS
00:00:02.780 TEST Send RPC Delete to Successor and Verify After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Send RPC Delete to Successor and Verify After Fail
Documentation:

Delete Flow in Successor and verify it gets removed from all instances.
Start / End / Elapsed: 20250309 02:30:58.511 / 20250309 02:31:01.291 / 00:00:02.780
Status: PASS
00:00:20.742 TEST Modify Network and Verify After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Modify Network and Verify After Fail
Documentation:

Take a link down and verify port status in all instances.
Start / End / Elapsed: 20250309 02:31:01.292 / 20250309 02:31:22.034 / 00:00:20.742
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 20 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"flow:1","node":[{"node-id":"openflow:2","opendaylight-topology-inventory:inventory-node-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']","termination-point":[{"tp-id":"openflow:2:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:LOCAL\']"},{"tp-id":"openflow:2:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:1\']"},{"tp-id":"openflow:2:2","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:2\']"},{"tp-id":"openflow:2:3","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:3\']"}]},{"node-id":"openflow:3","opendaylight-topology-inventory:inventory-node-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']","termination-point":[{"tp-id":"openflow:3:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:LOCAL\']"},{"tp-id":"openflow:3:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/nod... [ Message content over the limit has been removed. ] ...ogy-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:2\']"},{"tp-id":"openflow:3:3","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:3\']"}]},{"node-id":"openflow:1","opendaylight-topology-inventory:inventory-node-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']","termination-point":[{"tp-id":"openflow:1:2","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:2\']"},{"tp-id":"openflow:1:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:LOCAL\']"},{"tp-id":"openflow:1:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:1\']"}]}],"link":[{"link-id":"openflow:3:3","source":{"source-node":"openflow:3","source-tp":"openflow:3:3"},"destination":{"dest-tp":"openflow:1:2","dest-node":"openflow:1"}},{"link-id":"openflow:1:1","source":{"source-node":"openflow:1","source-tp":"openflow:1:1"},"destination":{"dest-tp":"openflow:2:3","dest-node":"openflow:2"}},{"link-id":"openflow:1:2","source":{"source-node":"openflow:1","source-tp":"openflow:1:2"},"destination":{"dest-tp":"openflow:3:3","dest-node":"openflow:3"}}]}]}}' contains 'openflow:1' 19 times, not 16 times.
00:00:20.738 KEYWORD ClusterOpenFlow . Take OpenFlow Device Link Down and Verify   ${new_cluster_list}
Documentation:

Take a link down and verify port status in all instances in ${controller_index_list}.
Start / End / Elapsed: 20250309 02:31:01.296 / 20250309 02:31:22.034 / 00:00:20.738
00:00:00.000 KEYWORD ${dictionary} = BuiltIn . Create Dictionary   "link-down":true=1
Documentation:

Creates and returns a dictionary based on the given items.
Start / End / Elapsed: 20250309 02:31:01.296 / 20250309 02:31:01.296 / 00:00:00.000
00:00:00.059 KEYWORD ${ouput} = MininetKeywords . Send Mininet Command   ${mininet_conn_id} link s1 s2 down
Documentation:

Sends Command ${cmd} to Mininet session ${mininet_conn} and returns read buffer response.
Start / End / Elapsed: 20250309 02:31:01.303 / 20250309 02:31:01.362 / 00:00:00.059
00:00:00.080 KEYWORD BuiltIn . Wait Until Keyword Succeeds   5s 1s ClusterManagement.Check_Item_Occurrence_Member_List_Or_All uri=${operational_port_1} dictionary=${dictionary} member_index_list=${controller_index_list}
Documentation:

Runs the specified keyword and retries if it fails.
Start / End / Elapsed: 20250309 02:31:01.363 / 20250309 02:31:01.443 / 00:00:00.080
00:00:00.000 KEYWORD ${dictionary} = BuiltIn . Create Dictionary   openflow:1=16 openflow:2=14 openflow:3=19
Documentation:

Creates and returns a dictionary based on the given items.
Start / End / Elapsed: 20250309 02:31:01.443 / 20250309 02:31:01.443 / 00:00:00.000
00:00:20.590 KEYWORD BuiltIn . Wait Until Keyword Succeeds   20s 2s ClusterManagement.Check_Item_Occurrence_Member_List_Or_All uri=${RFC8040_OPERATIONAL_TOPO_API} dictionary=${dictionary} member_index_list=${controller_index_list}
Documentation:

Runs the specified keyword and retries if it fails.
Start / End / Elapsed: 20250309 02:31:01.443 / 20250309 02:31:22.033 / 00:00:20.590
Message: 10 failing items removed using the --remove-keywords option.
00:00:00.041 KEYWORD ClusterManagement . Check_Item_Occurrence_Member_List_Or_All   uri=${RFC8040_OPERATIONAL_TOPO_API} dictionary=${dictionary} member_index_list=${controller_index_list}
Documentation:

Send a GET with the supplied uri to all or some members defined in ${member_index_list}. Then check received for occurrences of items expressed in a dictionary {'openflow:1': '16', 'openflow:2': '14', 'openflow:3': '19'}.
Start / End / Elapsed: 20250309 02:31:21.992 / 20250309 02:31:22.033 / 00:00:00.041
00:00:00.001 KEYWORD ${index_list} = ClusterManagement . List_Indices_Or_All   given_list=${member_index_list}
Documentation:

Utility to allow ${EMPTY} as default argument value, as the internal list is computed at runtime. This keyword always returns a (shallow) copy of given or default list, so operations with the returned list should not affect other lists. Also note that this keyword does not consider empty list to be ${EMPTY}.
Start / End / Elapsed: 20250309 02:31:21.993 / 20250309 02:31:21.994 / 00:00:00.001
00:00:00.038 FOR ${index} IN @{index_list}  
Start / End / Elapsed: 20250309 02:31:21.995 / 20250309 02:31:22.033 / 00:00:00.038
00:00:00.038 ITERATION ${index} = 2  
Start / End / Elapsed: 20250309 02:31:21.995 / 20250309 02:31:22.033 / 00:00:00.038
00:00:00.032 KEYWORD ${data} = ClusterManagement . Get_From_Member   uri=${uri} member_index=${index}
Documentation:

Send a GET with the supplied uri to member ${member_index}.
Start / End / Elapsed: 20250309 02:31:21.999 / 20250309 02:31:22.031 / 00:00:00.032
00:00:00.001 KEYWORD Utils . Check Item Occurrence   ${data} ${dictionary}
Documentation:

Check string for occurrences of items expressed in a list of dictionaries {item=occurrences}. 0 occurences means item is not present.
Start / End / Elapsed: 20250309 02:31:22.032 / 20250309 02:31:22.033 / 00:00:00.001
00:00:00.001 FOR ${item} IN @{dictionary_item_occurrence}  
Start / End / Elapsed: 20250309 02:31:22.032 / 20250309 02:31:22.033 / 00:00:00.001
00:00:00.001 ITERATION ${item} = openflow:1  
Start / End / Elapsed: 20250309 02:31:22.032 / 20250309 02:31:22.033 / 00:00:00.001
02:31:22.033 FAIL Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 20 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"flow:1","node":[{"node-id":"openflow:2","opendaylight-topology-inventory:inventory-node-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']","termination-point":[{"tp-id":"openflow:2:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:LOCAL\']"},{"tp-id":"openflow:2:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:1\']"},{"tp-id":"openflow:2:2","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:2\']"},{"tp-id":"openflow:2:3","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:3\']"}]},{"node-id":"openflow:3","opendaylight-topology-inventory:inventory-node-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']","termination-point":[{"tp-id":"openflow:3:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:LOCAL\']"},{"tp-id":"openflow:3:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:1\']"},{"tp-id":"openflow:3:2","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:2\']"},{"tp-id":"openflow:3:3","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:3\']"}]},{"node-id":"openflow:1","opendaylight-topology-inventory:inventory-node-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']","termination-point":[{"tp-id":"openflow:1:2","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:2\']"},{"tp-id":"openflow:1:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:LOCAL\']"},{"tp-id":"openflow:1:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:1\']"}]}],"link":[{"link-id":"openflow:3:3","source":{"source-node":"openflow:3","source-tp":"openflow:3:3"},"destination":{"dest-tp":"openflow:1:2","dest-node":"openflow:1"}},{"link-id":"openflow:1:1","source":{"source-node":"openflow:1","source-tp":"openflow:1:1"},"destination":{"dest-tp":"openflow:2:3","dest-node":"openflow:2"}},{"link-id":"openflow:1:2","source":{"source-node":"openflow:1","source-tp":"openflow:1:2"},"destination":{"dest-tp":"openflow:3:3","dest-node":"openflow:3"}}]}]}}' contains 'openflow:1' 19 times, not 16 times.
00:00:02.311 TEST Restore Network and Verify After Fail
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Restore Network and Verify After Fail
Documentation:

Take the link up and verify port status in all instances.
Start / End / Elapsed: 20250309 02:31:22.034 / 20250309 02:31:24.345 / 00:00:02.311
Status: PASS
00:00:34.694 TEST Start Old Owner Instance
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Start Old Owner Instance
Documentation:

Start old Owner Instance and verify it is up
Start / End / Elapsed: 20250309 02:31:24.345 / 20250309 02:31:59.039 / 00:00:34.694
Status: PASS
00:00:03.297 TEST Check Shards Status After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Check Shards Status After Recover
Documentation:

Create original cluster list and check Status for all shards in OpenFlow application.
Start / End / Elapsed: 20250309 02:31:59.039 / 20250309 02:32:02.336 / 00:00:03.297
Status: PASS
00:00:00.196 TEST Check Entity Owner Status After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Check Entity Owner Status After Recover
Documentation:

Check Entity Owner Status and identify owner and successor.
Start / End / Elapsed: 20250309 02:32:02.336 / 20250309 02:32:02.532 / 00:00:00.196
Status: PASS
00:00:00.403 TEST Check Network Operational Information After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Check Network Operational Information After Recover
Documentation:

Check devices in operational inventory and topology in all cluster instances.
Start / End / Elapsed: 20250309 02:32:02.533 / 20250309 02:32:02.936 / 00:00:00.403
Status: PASS
00:00:01.423 TEST Add Configuration In Owner and Verify After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Add Configuration In Owner and Verify After Recover
Documentation:

Add Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250309 02:32:02.936 / 20250309 02:32:04.359 / 00:00:01.423
Status: PASS
00:00:03.516 TEST Modify Configuration In Owner and Verify After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Modify Configuration In Owner and Verify After Recover
Documentation:

Modify Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250309 02:32:04.359 / 20250309 02:32:07.875 / 00:00:03.516
Status: PASS
00:00:03.368 TEST Delete Configuration In Owner and Verify After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Delete Configuration In Owner and Verify After Recover
Documentation:

Delete Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250309 02:32:07.876 / 20250309 02:32:11.244 / 00:00:03.368
Status: PASS
00:00:02.455 TEST Add Configuration In Old Owner and Verify After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Add Configuration In Old Owner and Verify After Recover
Documentation:

Add Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250309 02:32:11.245 / 20250309 02:32:13.700 / 00:00:02.455
Status: PASS
00:00:03.535 TEST Modify Configuration In Old Owner and Verify After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Modify Configuration In Old Owner and Verify After Recover
Documentation:

Modify Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250309 02:32:13.700 / 20250309 02:32:17.235 / 00:00:03.535
Status: PASS
00:00:02.328 TEST Delete Configuration In Old Owner and Verify After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Delete Configuration In Old Owner and Verify After Recover
Documentation:

Delete Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250309 02:32:17.235 / 20250309 02:32:19.563 / 00:00:02.328
Status: PASS
00:00:03.487 TEST Send RPC Add to Owner and Verify After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Send RPC Add to Owner and Verify After Recover
Documentation:

Add Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250309 02:32:19.563 / 20250309 02:32:23.050 / 00:00:03.487
Status: PASS
00:00:02.447 TEST Send RPC Delete to Owner and Verify After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Send RPC Delete to Owner and Verify After Recover
Documentation:

Delete Flow in Owner and verify it gets removed from all instances.
Start / End / Elapsed: 20250309 02:32:23.050 / 20250309 02:32:25.497 / 00:00:02.447
Status: PASS
00:00:03.597 TEST Send RPC Add to Old Owner and Verify After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Send RPC Add to Old Owner and Verify After Recover
Documentation:

Add Flow in Owner and verify it gets applied from all instances.
Start / End / Elapsed: 20250309 02:32:25.497 / 20250309 02:32:29.094 / 00:00:03.597
Status: PASS
00:00:02.448 TEST Send RPC Delete to Old Owner and Verify After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Send RPC Delete to Old Owner and Verify After Recover
Documentation:

Delete Flow in Owner and verify it gets removed from all instances.
Start / End / Elapsed: 20250309 02:32:29.095 / 20250309 02:32:31.543 / 00:00:02.448
Status: PASS
00:00:20.814 TEST Modify Network and Verify After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Modify Network and Verify After Recover
Documentation:

Take a link down and verify port status in all instances.
Start / End / Elapsed: 20250309 02:32:31.543 / 20250309 02:32:52.357 / 00:00:20.814
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 20 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"flow:1","link":[{"link-id":"openflow:3:3","source":{"source-node":"openflow:3","source-tp":"openflow:3:3"},"destination":{"dest-tp":"openflow:1:2","dest-node":"openflow:1"}},{"link-id":"openflow:1:1","source":{"source-node":"openflow:1","source-tp":"openflow:1:1"},"destination":{"dest-tp":"openflow:2:3","dest-node":"openflow:2"}},{"link-id":"openflow:1:2","source":{"source-node":"openflow:1","source-tp":"openflow:1:2"},"destination":{"dest-tp":"openflow:3:3","dest-node":"openflow:3"}}],"node":[{"node-id":"openflow:2","opendaylight-topology-inventory:inventory-node-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']","termination-point":[{"tp-id":"openflow:2:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:LOCAL\']"},{"tp-id":"openflow:2:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:1\']"},{"tp-id":"openflow:2:2","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'openflow:2:2\']"},{"tp-id":"openflow:2:3","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:2\']/node-connector[id=\'... [ Message content over the limit has been removed. ] ...ndaylight-inventory:nodes/node[id=\'openflow:3\']","termination-point":[{"tp-id":"openflow:3:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:LOCAL\']"},{"tp-id":"openflow:3:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:1\']"},{"tp-id":"openflow:3:2","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:2\']"},{"tp-id":"openflow:3:3","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:3\']/node-connector[id=\'openflow:3:3\']"}]},{"node-id":"openflow:1","opendaylight-topology-inventory:inventory-node-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']","termination-point":[{"tp-id":"openflow:1:2","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:2\']"},{"tp-id":"openflow:1:LOCAL","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:LOCAL\']"},{"tp-id":"openflow:1:1","opendaylight-topology-inventory:inventory-node-connector-ref":"/opendaylight-inventory:nodes/node[id=\'openflow:1\']/node-connector[id=\'openflow:1:1\']"}]}]}]}}' contains 'openflow:1' 19 times, not 16 times.
00:00:02.466 TEST Restore Network and Verify After Recover
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Restore Network and Verify After Recover
Documentation:

Take the link up and verify port status in all instances.
Start / End / Elapsed: 20250309 02:32:52.357 / 20250309 02:32:54.823 / 00:00:02.466
Status: PASS
00:00:02.598 TEST Stop Mininet and Exit
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Stop Mininet and Exit
Documentation:

Stop mininet and exit connection.
Start / End / Elapsed: 20250309 02:32:54.823 / 20250309 02:32:57.421 / 00:00:02.598
Status: PASS
00:00:20.676 TEST Check No Network Operational Information
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Failover.Check No Network Operational Information
Documentation:

Check device is not in operational inventory or topology in all cluster instances.
Start / End / Elapsed: 20250309 02:32:57.422 / 20250309 02:33:18.098 / 00:00:20.676
Status: FAIL
Message: Keyword 'ClusterManagement.Check_Item_Occurrence_Member_List_Or_All' failed after retrying for 20 seconds. The last error was: '{"network-topology:network-topology":{"topology":[{"topology-id":"flow:1","link":[{"link-id":"openflow:1:1","source":{"source-node":"openflow:1","source-tp":"openflow:1:1"},"destination":{"dest-tp":"openflow:2:3","dest-node":"openflow:2"}},{"link-id":"openflow:1:2","source":{"source-node":"openflow:1","source-tp":"openflow:1:2"},"destination":{"dest-tp":"openflow:3:3","dest-node":"openflow:3"}}]}]}}' contains 'openflow' 10 times, not 0 times.
00:03:15.976 SUITE Cluster HA Owner Restart
Full Name: openflowplugin-clustering.txt.Cluster HA Owner Restart
Documentation:

Test suite for Cluster HA - Device Owner Stop and Start
Source: /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/020__Cluster_HA_Owner_Restart.robot
Start / End / Elapsed: 20250309 02:33:18.099 / 20250309 02:36:34.075 / 00:03:15.976
Status: 51 tests total, 50 passed, 1 failed, 0 skipped
00:05:36.932 SUITE Cluster HA Data Recovery Leader Follower Failover
Full Name: openflowplugin-clustering.txt.Cluster HA Data Recovery Leader Follower Failover
Documentation:

Test suite for Cluster HA - Data Recovery at Leader Follower failover and cluster restart
Source: /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/030__Cluster_HA_Data_Recovery_Leader_Follower_Failover.robot
Start / End / Elapsed: 20250309 02:36:34.076 / 20250309 02:42:11.008 / 00:05:36.932
Status: 21 tests total, 21 passed, 0 failed, 0 skipped
00:09:21.699 SUITE 010 Group Flows
Full Name: openflowplugin-clustering.txt.010 Group Flows
Documentation:

Switch connections and cluster are restarted.
Source: /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustered_Reconciliation/010_Group_Flows.robot
Start / End / Elapsed: 20250309 02:42:11.009 / 20250309 02:51:32.708 / 00:09:21.699
Status: 72 tests total, 66 passed, 6 failed, 0 skipped
00:00:04.661 SUITE 010 Switch Disconnect
Full Name: openflowplugin-clustering.txt.010 Switch Disconnect
Documentation:

Test suite for entity ownership service and openflowplugin. Makes changes on switch side.
Source: /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/010_Switch_Disconnect.robot
Start / End / Elapsed: 20250309 02:51:32.710 / 20250309 02:51:37.371 / 00:00:04.661
Status: 3 tests total, 0 passed, 3 failed, 0 skipped
Message: Suite setup failed: Dictionary does not contain key 's1'.
00:00:04.277 SUITE 020 Cluster Node Failure
Full Name: openflowplugin-clustering.txt.020 Cluster Node Failure
Documentation:

Test suite for entity ownership service and openflowplugin. Makes changes on controller side (restart karaf)
Source: /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/020_Cluster_Node_Failure.robot
Start / End / Elapsed: 20250309 02:51:37.371 / 20250309 02:51:41.648 / 00:00:04.277
Status: 3 tests total, 0 passed, 3 failed, 0 skipped
Message: Suite setup failed: Dictionary does not contain key 's1'.
00:01:23.871 SUITE 030 Cluster Sync Problems
Full Name: openflowplugin-clustering.txt.030 Cluster Sync Problems
Documentation:

Test suite for entity ownership service and openflowplugin. Makes changes on controller side (isolating cluster node)
Source: /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/030_Cluster_Sync_Problems.robot
Start / End / Elapsed: 20250309 02:51:41.649 / 20250309 02:53:05.520 / 00:01:23.871
Status: 5 tests total, 0 passed, 5 failed, 0 skipped
00:00:08.565 SUITE 9145
Full Name: openflowplugin-clustering.txt.9145
Documentation:

Switch connections and cluster are restarted.
Source: /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Bug_Validation/9145.robot
Start / End / Elapsed: 20250309 02:53:05.521 / 20250309 02:53:14.086 / 00:00:08.565
Status: 3 tests total, 3 passed, 0 failed, 0 skipped