Test Statistics

Total Statistics
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
All Tests
19524171003:38:09
Statistics by Tag
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
carpeople
85085000:00:00
clustering
85085000:00:00
critical
19524171003:38:09
Statistics by Suite
Total
Pass
Fail
Skip
Elapsed
Pass / Fail / Skip
controller-clustering-ask.txt
19524171004:03:40
controller-clustering-ask . txt . Restart Odl With Tell Based False
312000:13:51
controller-clustering-ask . txt . Rpc Provider Precedence
844000:14:40
controller-clustering-ask . txt . Restart Odl With Tell Based False
312000:14:39
controller-clustering-ask . txt . Rpc Provider Partition And Heal
725000:14:00
controller-clustering-ask . txt . Restart Odl With Tell Based False
312000:14:37
controller-clustering-ask . txt . Action Provider Precedence
844000:14:40
controller-clustering-ask . txt . Restart Odl With Tell Based False
312000:14:38
controller-clustering-ask . txt . Action Provider Partition And Heal
725000:14:01
controller-clustering-ask . txt . Restart Odl With Tell Based False
312000:14:26
controller-clustering-ask . txt . Master Stability
606000:09:08
controller-clustering-ask . txt . Restart Odl With Tell Based False
312000:14:38
controller-clustering-ask . txt . Partition And Heal
606000:09:08
controller-clustering-ask . txt . Restart Odl With Tell Based False
312000:13:26
controller-clustering-ask . txt . Chasing The Leader
312000:03:55
controller-clustering-ask . txt . Restart Odl With Tell Based False
312000:13:23
controller-clustering-ask . txt . Global Rpc Kill
11011000:01:03
controller-clustering-ask . txt . Restart Odl With Tell Based False
312000:13:24
controller-clustering-ask . txt . Global Rpc Freeze
10010000:01:04
controller-clustering-ask . txt . Restart Odl With Tell Based False
312000:13:22
controller-clustering-ask . txt . Global Rpc Isolate
11011000:01:04
controller-clustering-ask . txt . Restart Odl With Tell Based False
312000:13:24
controller-clustering-ask . txt . Carpeople Crud
13013000:00:54
controller-clustering-ask . txt . Car Failover Crud
16016000:00:55
controller-clustering-ask . txt . Car Persistence Recovery
707000:00:54
controller-clustering-ask . txt . Buycar Failover
10010000:00:53
controller-clustering-ask . txt . Entity Isolate
505000:00:55
controller-clustering-ask . txt . Buycar Failover Isolation
10010000:00:53
controller-clustering-ask . txt . Car Failover Crud Isolation
16016000:00:53
controller-clustering-ask . txt . Car Outage Corners
808000:00:53

Test Execution Log

04:03:39.596 SUITE controller-clustering-ask.txt
Full Name: controller-clustering-ask.txt
Start / End / Elapsed: 20250209 22:54:08.144 / 20250210 02:57:47.740 / 04:03:39.596
Status: 195 tests total, 24 passed, 171 failed, 0 skipped
00:13:51.293 SUITE Restart Odl With Tell Based False
Full Name: controller-clustering-ask.txt.Restart Odl With Tell Based False
Documentation:

Unset tell-based protocol usage

Copyright (c) 2016 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

Suite stops all odl nodes, outcomment usage of tell-based protocol in config file (means make it false by default) and starts all nodes again.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot
Start / End / Elapsed: 20250209 22:54:08.168 / 20250209 23:07:59.461 / 00:13:51.293
Status: 3 tests total, 1 passed, 2 failed, 0 skipped
00:00:05.221 SETUP SetupUtils . Setup_Utils_For_Setup_And_Teardown   http_timeout=125
Documentation:

Prepare both FailFast and Karaf logging, log that the caller suite has started. This keyword is to be used in suite setup, and without it the other keywords may not work properly.
Start / End / Elapsed: 20250209 22:54:08.732 / 20250209 22:54:13.953 / 00:00:05.221
00:00:00.000 TEARDOWN SSHLibrary . Close All Connections  
Documentation:

Closes all open connections.
Start / End / Elapsed: 20250209 23:07:59.460 / 20250209 23:07:59.460 / 00:00:00.000
00:06:12.865 TEST Stop_All_Members
Full Name: controller-clustering-ask.txt.Restart Odl With Tell Based False.Stop_All_Members
Documentation:

Stop every odl node.
Tags: critical
Start / End / Elapsed: 20250209 22:54:13.953 / 20250209 23:00:26.818 / 00:06:12.865
Status: FAIL
Message: Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
00:00:10.724 TEST Unset_Tell_Based_Protocol_Usage
Full Name: controller-clustering-ask.txt.Restart Odl With Tell Based False.Unset_Tell_Based_Protocol_Usage
Documentation:

Comment out the flag usage in config file. Also clean most data except data/log/.
Tags: critical
Start / End / Elapsed: 20250209 23:00:26.818 / 20250209 23:00:37.542 / 00:00:10.724
Status: PASS
00:07:21.917 TEST Start_All_And_Sync
Full Name: controller-clustering-ask.txt.Restart Odl With Tell Based False.Start_All_And_Sync
Documentation:

Start each member and wait for sync.
Tags: critical
Start / End / Elapsed: 20250209 23:00:37.543 / 20250209 23:07:59.460 / 00:07:21.917
Status: FAIL
Message: Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.170.220', port=8181): Read timed out. (read timeout=125.0)
00:14:40.492 SUITE Rpc Provider Precedence
Full Name: controller-clustering-ask.txt.Rpc Provider Precedence
Documentation:

DOMRpcBroker testing: RPC Provider Precedence

Copyright (c) 2017 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

The aim is to establish that remote RPC implementations have lower priority than local ones, which is to say that any movement of RPCs on remote nodes does not affect routing as long as a local implementation is available.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_precedence.robot
Start / End / Elapsed: 20250209 23:07:59.461 / 20250209 23:22:39.953 / 00:14:40.492
Status: 8 tests total, 4 passed, 4 failed, 0 skipped
00:14:38.998 SUITE Restart Odl With Tell Based False
Full Name: controller-clustering-ask.txt.Restart Odl With Tell Based False
Documentation:

Unset tell-based protocol usage

Copyright (c) 2016 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

Suite stops all odl nodes, outcomment usage of tell-based protocol in config file (means make it false by default) and starts all nodes again.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot
Start / End / Elapsed: 20250209 23:22:39.954 / 20250209 23:37:18.952 / 00:14:38.998
Status: 3 tests total, 1 passed, 2 failed, 0 skipped
00:14:00.080 SUITE Rpc Provider Partition And Heal
Full Name: controller-clustering-ask.txt.Rpc Provider Partition And Heal
Documentation:

DOMRpcBroker testing: RPC Provider Partition And Heal

Copyright (c) 2017 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

This tests establishes that the RPC service operates correctly when faced with node failures. This suite supports more than three node cluster setup too.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_partition_and_heal.robot
Start / End / Elapsed: 20250209 23:37:18.953 / 20250209 23:51:19.033 / 00:14:00.080
Status: 7 tests total, 2 passed, 5 failed, 0 skipped
00:14:36.572 SUITE Restart Odl With Tell Based False
Full Name: controller-clustering-ask.txt.Restart Odl With Tell Based False
Documentation:

Unset tell-based protocol usage

Copyright (c) 2016 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

Suite stops all odl nodes, outcomment usage of tell-based protocol in config file (means make it false by default) and starts all nodes again.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot
Start / End / Elapsed: 20250209 23:51:19.033 / 20250210 00:05:55.605 / 00:14:36.572
Status: 3 tests total, 1 passed, 2 failed, 0 skipped
00:14:40.348 SUITE Action Provider Precedence
Full Name: controller-clustering-ask.txt.Action Provider Precedence
Documentation:

DOMRpcBroker testing: Action Provider Precedence

Copyright (c) 2017 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

The aim is to establish that remote action implementations have lower priority than local ones, which is to say that any movement of actions on remote nodes does not affect routing as long as a local implementation is available.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/action_provider_precedence.robot
Start / End / Elapsed: 20250210 00:05:55.606 / 20250210 00:20:35.954 / 00:14:40.348
Status: 8 tests total, 4 passed, 4 failed, 0 skipped
00:14:37.751 SUITE Restart Odl With Tell Based False
Full Name: controller-clustering-ask.txt.Restart Odl With Tell Based False
Documentation:

Unset tell-based protocol usage

Copyright (c) 2016 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

Suite stops all odl nodes, outcomment usage of tell-based protocol in config file (means make it false by default) and starts all nodes again.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot
Start / End / Elapsed: 20250210 00:20:35.954 / 20250210 00:35:13.705 / 00:14:37.751
Status: 3 tests total, 1 passed, 2 failed, 0 skipped
00:14:01.360 SUITE Action Provider Partition And Heal
Full Name: controller-clustering-ask.txt.Action Provider Partition And Heal
Documentation:

DOMRpcBroker testing: RPC Action Provider Partition And Heal

Copyright (c) 2017 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

This tests establishes that the RPC service for actions operates correctly when faced with node failures. This suite supports more than three node cluster setup too.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/action_provider_partition_and_heal.robot
Start / End / Elapsed: 20250210 00:35:13.706 / 20250210 00:49:15.066 / 00:14:01.360
Status: 7 tests total, 2 passed, 5 failed, 0 skipped
00:14:26.101 SUITE Restart Odl With Tell Based False
Full Name: controller-clustering-ask.txt.Restart Odl With Tell Based False
Documentation:

Unset tell-based protocol usage

Copyright (c) 2016 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

Suite stops all odl nodes, outcomment usage of tell-based protocol in config file (means make it false by default) and starts all nodes again.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot
Start / End / Elapsed: 20250210 00:49:15.067 / 20250210 01:03:41.168 / 00:14:26.101
Status: 3 tests total, 1 passed, 2 failed, 0 skipped
00:09:07.947 SUITE Master Stability
Full Name: controller-clustering-ask.txt.Master Stability
Documentation:

Cluster Singleton testing: Master Stability

Copyright (c) 2017 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

Cluster Singleton service is designed to ensure that only one instance of an application is registered globally in the cluster. The goal is to establish the service operates correctly in face of application registration changing without moving the active instance.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/cluster_singleton/master_stability.robot
Start / End / Elapsed: 20250210 01:03:41.169 / 20250210 01:12:49.116 / 00:09:07.947
Status: 6 tests total, 0 passed, 6 failed, 0 skipped
00:14:38.146 SUITE Restart Odl With Tell Based False
Full Name: controller-clustering-ask.txt.Restart Odl With Tell Based False
Documentation:

Unset tell-based protocol usage

Copyright (c) 2016 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

Suite stops all odl nodes, outcomment usage of tell-based protocol in config file (means make it false by default) and starts all nodes again.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot
Start / End / Elapsed: 20250210 01:12:49.117 / 20250210 01:27:27.263 / 00:14:38.146
Status: 3 tests total, 1 passed, 2 failed, 0 skipped
00:09:07.957 SUITE Partition And Heal
Full Name: controller-clustering-ask.txt.Partition And Heal
Documentation:

Cluster Singleton testing: Partition And Heal

Copyright (c) 2017 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

Cluster Singleton service is designed to ensure that only one instance of an application is registered globally in the cluster. The goal is to establish the service operates correctly in face of node failures.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot
Start / End / Elapsed: 20250210 01:27:27.264 / 20250210 01:36:35.221 / 00:09:07.957
Status: 6 tests total, 0 passed, 6 failed, 0 skipped
00:13:25.549 SUITE Restart Odl With Tell Based False
Full Name: controller-clustering-ask.txt.Restart Odl With Tell Based False
Documentation:

Unset tell-based protocol usage

Copyright (c) 2016 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

Suite stops all odl nodes, outcomment usage of tell-based protocol in config file (means make it false by default) and starts all nodes again.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot
Start / End / Elapsed: 20250210 01:36:35.221 / 20250210 01:50:00.770 / 00:13:25.549
Status: 3 tests total, 1 passed, 2 failed, 0 skipped
00:03:55.229 SUITE Chasing The Leader
Full Name: controller-clustering-ask.txt.Chasing The Leader
Documentation:

Cluster Singleton testing: Chasing the Leader

Copyright (c) 2017 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

This test aims to establish the service operates correctly when faced with rapid application transitions without having a stabilized application.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot
Start / End / Elapsed: 20250210 01:50:00.771 / 20250210 01:53:56.000 / 00:03:55.229
Status: 3 tests total, 1 passed, 2 failed, 0 skipped
00:13:22.587 SUITE Restart Odl With Tell Based False
Full Name: controller-clustering-ask.txt.Restart Odl With Tell Based False
Documentation:

Unset tell-based protocol usage

Copyright (c) 2016 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

Suite stops all odl nodes, outcomment usage of tell-based protocol in config file (means make it false by default) and starts all nodes again.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot
Start / End / Elapsed: 20250210 01:53:56.001 / 20250210 02:07:18.588 / 00:13:22.587
Status: 3 tests total, 1 passed, 2 failed, 0 skipped
00:01:03.329 SUITE Global Rpc Kill
Full Name: controller-clustering-ask.txt.Global Rpc Kill
Documentation:

Controller functional HA testing of global singleton rpcs.

Copyright (c) 2017 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/singleton_service/global_rpc_kill.robot
Start / End / Elapsed: 20250210 02:07:18.589 / 20250210 02:08:21.918 / 00:01:03.329
Status: 11 tests total, 0 passed, 11 failed, 0 skipped
Message: Suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: .
00:13:23.559 SUITE Restart Odl With Tell Based False
Full Name: controller-clustering-ask.txt.Restart Odl With Tell Based False
Documentation:

Unset tell-based protocol usage

Copyright (c) 2016 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

Suite stops all odl nodes, outcomment usage of tell-based protocol in config file (means make it false by default) and starts all nodes again.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot
Start / End / Elapsed: 20250210 02:08:21.918 / 20250210 02:21:45.477 / 00:13:23.559
Status: 3 tests total, 1 passed, 2 failed, 0 skipped
00:01:03.735 SUITE Global Rpc Freeze
Full Name: controller-clustering-ask.txt.Global Rpc Freeze
Documentation:

Controller functional HA testing of global singleton rpcs if jvm frozen.

Copyright (c) 2017 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/singleton_service/global_rpc_freeze.robot
Start / End / Elapsed: 20250210 02:21:45.478 / 20250210 02:22:49.213 / 00:01:03.735
Status: 10 tests total, 0 passed, 10 failed, 0 skipped
Message: Suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: .
00:13:22.064 SUITE Restart Odl With Tell Based False
Full Name: controller-clustering-ask.txt.Restart Odl With Tell Based False
Documentation:

Unset tell-based protocol usage

Copyright (c) 2016 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

Suite stops all odl nodes, outcomment usage of tell-based protocol in config file (means make it false by default) and starts all nodes again.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot
Start / End / Elapsed: 20250210 02:22:49.213 / 20250210 02:36:11.277 / 00:13:22.064
Status: 3 tests total, 1 passed, 2 failed, 0 skipped
00:01:03.721 SUITE Global Rpc Isolate
Full Name: controller-clustering-ask.txt.Global Rpc Isolate
Documentation:

Controller functional HA testing of global singleton rpcs.

Copyright (c) 2017 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/singleton_service/global_rpc_isolate.robot
Start / End / Elapsed: 20250210 02:36:11.278 / 20250210 02:37:14.999 / 00:01:03.721
Status: 11 tests total, 0 passed, 11 failed, 0 skipped
Message: Suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: .
00:13:24.262 SUITE Restart Odl With Tell Based False
Full Name: controller-clustering-ask.txt.Restart Odl With Tell Based False
Documentation:

Unset tell-based protocol usage

Copyright (c) 2016 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

Suite stops all odl nodes, outcomment usage of tell-based protocol in config file (means make it false by default) and starts all nodes again.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot
Start / End / Elapsed: 20250210 02:37:14.999 / 20250210 02:50:39.261 / 00:13:24.262
Status: 3 tests total, 1 passed, 2 failed, 0 skipped
00:00:54.369 SUITE Carpeople Crud
Full Name: controller-clustering-ask.txt.Carpeople Crud
Documentation:

Suite for performing basic car/people CRUD operations on leaders and followers.

Copyright (c) 2016 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

More precisely, Update operation is not executed, but some operations are using specific RPCs which goes beyond "basic CRUD".

Cars are added by one big PUT to datastore on car Leader. People are added in a loop with add-person RPC on a people Follower. Cars are bought by chunks on each member, by loop with buy-car RPC.

All data is deleted at the end of the suite. This suite expects car, people and car-people modules to have separate Shards.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot
Start / End / Elapsed: 20250210 02:50:39.262 / 20250210 02:51:33.631 / 00:00:54.369
Status: 13 tests total, 0 passed, 13 failed, 0 skipped
Message: Suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.24:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
00:00:54.657 SUITE Car Failover Crud
Full Name: controller-clustering-ask.txt.Car Failover Crud
Documentation:

Suite mixing basic operations with restart of car Leader.

Copyright (c) 2016 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

This test stops the current leader of the "car" shard and then executes CRD operations on the new leader and a new follower. The stopped member is brought back. This suite uses 3 different car sets, same size but different starting ID.

Other models and shards (people, car-people) are not accessed by this suite.

All data is deleted at the end of the suite. This suite expects car module to have a separate Shard.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot
Start / End / Elapsed: 20250210 02:51:33.631 / 20250210 02:52:28.288 / 00:00:54.657
Status: 16 tests total, 0 passed, 16 failed, 0 skipped
Message: Suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.24:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
00:00:53.558 SUITE Car Persistence Recovery
Full Name: controller-clustering-ask.txt.Car Persistence Recovery
Documentation:

This test restarts all controllers to verify recovery of car data from persistence.

Copyright (c) 2016 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

Other models and shards (people, car-people) are not accessed by this suite.

All data is deleted at the end of the suite. This suite expects car module to have a separate Shard.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot
Start / End / Elapsed: 20250210 02:52:28.289 / 20250210 02:53:21.847 / 00:00:53.558
Status: 7 tests total, 0 passed, 7 failed, 0 skipped
Message: Suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.24:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
00:00:52.619 SUITE Buycar Failover
Full Name: controller-clustering-ask.txt.Buycar Failover
Documentation:

This test focuses on testing buy-car RPC over 3 Leader reboots.

Copyright (c) 2016 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

All purchases are against the same node, which is the first one to get rebooted.

All data is deleted at the end of the suite. This suite expects car, people and car-people modules to have separate Shards.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot
Start / End / Elapsed: 20250210 02:53:21.848 / 20250210 02:54:14.467 / 00:00:52.619
Status: 10 tests total, 0 passed, 10 failed, 0 skipped
Message: Suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.24:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
00:00:54.569 SUITE Entity Isolate
Full Name: controller-clustering-ask.txt.Entity Isolate
Documentation:

Suite for performing member isolation and rejoin, we do with entity-ownership leader.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot
Start / End / Elapsed: 20250210 02:54:14.468 / 20250210 02:55:09.037 / 00:00:54.569
Status: 5 tests total, 0 passed, 5 failed, 0 skipped
Message: Suite setup failed: Evaluating expression 'json.loads(\'\'\'{\n "error": "javax.management.InstanceNotFoundException : org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "error_type": "javax.management.InstanceNotFoundException",\n "request": {\n "mbean": "org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "type": "read"\n },\n "stacktrace": "javax.management.InstanceNotFoundException: org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1073)\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBeanInfo(DefaultMBeanServerInterceptor.java:1343)\\n\\tat java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.getMBeanInfo(JmxMBeanServer.java:921)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:46)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:41)\\n\\tat org.jolokia.backend.executor.AbstractMBeanServerExecutor.call(AbstractMBeanServerExecutor.java:90)\\n\\tat org.jolokia.handler.ReadHandler.getMBeanInfo(ReadHandler.java:233)\\n\\tat org.jolokia.handler.ReadHandler.getAllAttributesNames(ReadHandler.java:245)\\n\\tat org.jolokia.handler.ReadHandler.resolveAttributes(ReadHandler.java:221)\\n\\tat org.jolokia.handler.ReadHandler.f... [ Message content over the limit has been removed. ] ...lipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\\n\\tat org.eclipse.jetty.server.Server.handle(Server.java:516)\\n\\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)\\n\\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)\\n\\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)\\n\\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)\\n\\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)\\n\\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)\\n\\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)\\n\\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)\\n\\tat java.base/java.lang.Thread.run(Thread.java:1583)\\n",\n "status": 404\n}\n\'\'\')' failed: JSONDecodeError: Invalid control character at: line 8 column 190 (char 619)
00:00:52.655 SUITE Buycar Failover Isolation
Full Name: controller-clustering-ask.txt.Buycar Failover Isolation
Documentation:

This test focuses on testing buy-car RPC over the Leader isolation.

Copyright (c) 2017 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

All purchases are against the same node, which is the first one to get isolated.

All data is deleted at the end of the suite. This suite expects car, people and car-people modules to have separate Shards.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot
Start / End / Elapsed: 20250210 02:55:09.038 / 20250210 02:56:01.693 / 00:00:52.655
Status: 10 tests total, 0 passed, 10 failed, 0 skipped
Message: Suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.24:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
00:00:53.411 SUITE Car Failover Crud Isolation
Full Name: controller-clustering-ask.txt.Car Failover Crud Isolation
Documentation:

Suite mixing basic operations with isolation of car Leader.

Copyright (c) 2017 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

This test isolates the current leader of the "car" shard and then executes CRD operations on the new leader and a new follower. The isolated member is brought back. This suite uses 3 different car sets, same size but different starting ID.

Other models and shards (people, car-people) are not accessed by this suite.

All data is deleted at the end of the suite. This suite expects car module to have a separate Shard.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot
Start / End / Elapsed: 20250210 02:56:01.694 / 20250210 02:56:55.105 / 00:00:53.411
Status: 16 tests total, 0 passed, 16 failed, 0 skipped
Message: Suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.24:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
00:00:52.633 SUITE Car Outage Corners
Full Name: controller-clustering-ask.txt.Car Outage Corners
Documentation:

Cluster suite for testing minimal and sum-minimal member population behavior.

Copyright (c) 2016 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

This test stops majority of the followers and verifies car addition is not possible, then resumes single follower (first from original list) and checks that addition works. Then remaining members are brought up. Leader member is always up and assumed to remain Leading during the whole suite run.

TODO: Use initial data to check more operations. TODO: Perhaps merge with car_failover_crud suite.

Other modules and Shards (people, car-people) are not accessed by this suite.

All data is deleted at the end of the suite. This suite expects car module to have a separate Shard.
Source: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot
Start / End / Elapsed: 20250210 02:56:55.106 / 20250210 02:57:47.739 / 00:00:52.633
Status: 8 tests total, 0 passed, 8 failed, 0 skipped
Message: Suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.24:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig