Started by upstream project "integration-distribution-test-titanium" build number 747
originally caused by:
 Started by upstream project "autorelease-release-titanium-mvn39-openjdk21" build number 642
 originally caused by:
  Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on prd-ubuntu2404-robot-2c-8g-21105 (ubuntu2404-robot-2c-8g) in workspace /w/workspace/openflowplugin-csit-3node-clustering-only-titanium
[ssh-agent] Looking for ssh-agent implementation...
[ssh-agent]   Exec ssh-agent (binary ssh-agent on a remote machine)
$ ssh-agent
SSH_AUTH_SOCK=/tmp/ssh-NZnHk0YRZEuh/agent.3164
SSH_AGENT_PID=3166
[ssh-agent] Started.
Running ssh-add (command line suppressed)
Identity added: /w/workspace/openflowplugin-csit-3node-clustering-only-titanium@tmp/private_key_5652793385451691952.key (/w/workspace/openflowplugin-csit-3node-clustering-only-titanium@tmp/private_key_5652793385451691952.key)
[ssh-agent] Using credentials jenkins (Release Engineering Jenkins Key)
The recommended git tool is: NONE
using credential opendaylight-jenkins-ssh
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository git://devvexx.opendaylight.org/mirror/integration/test
 > git init /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test # timeout=10
Fetching upstream changes from git://devvexx.opendaylight.org/mirror/integration/test
 > git --version # timeout=10
 > git --version # 'git version 2.43.0'
using GIT_SSH to set credentials Release Engineering Jenkins Key
Verifying host key using known hosts file, will automatically accept unseen keys
 > git fetch --tags --force --progress -- git://devvexx.opendaylight.org/mirror/integration/test +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url git://devvexx.opendaylight.org/mirror/integration/test # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url git://devvexx.opendaylight.org/mirror/integration/test # timeout=10
Fetching upstream changes from git://devvexx.opendaylight.org/mirror/integration/test
using GIT_SSH to set credentials Release Engineering Jenkins Key
Verifying host key using known hosts file, will automatically accept unseen keys
 > git fetch --tags --force --progress -- git://devvexx.opendaylight.org/mirror/integration/test master # timeout=10
 > git rev-parse FETCH_HEAD^{commit} # timeout=10
Checking out Revision 901c7e139945b436d95a44b3b592904c3d7a4f9f (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 901c7e139945b436d95a44b3b592904c3d7a4f9f # timeout=10
Commit message: "Adapt updater.py to use new lsp-flags container"
 > git rev-parse FETCH_HEAD^{commit} # timeout=10
 > git rev-list --no-walk f65244a2a093016c71f9a7a564be7d086941e15c # timeout=10
No emails were triggered.
provisioning config files...
copy managed file [npmrc] to file:/home/jenkins/.npmrc
copy managed file [pipconf] to file:/home/jenkins/.config/pip/pip.conf
copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
[openflowplugin-csit-3node-clustering-only-titanium] $ /bin/bash /tmp/jenkins16668822352953193861.sh
---> python-tools-install.sh
Setup pyenv:
* system (set by /opt/pyenv/version)
* 3.8.20 (set by /opt/pyenv/version)
* 3.9.20 (set by /opt/pyenv/version)
  3.10.15
  3.11.10
lf-activate-venv(): INFO: Creating python3 venv at /tmp/venv-QOlV
lf-activate-venv(): INFO: Save venv in file: /tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv)
lf-activate-venv(): INFO: Attempting to install with network-safe options...
lf-activate-venv(): INFO: Base packages installed successfully
lf-activate-venv(): INFO: Installing additional packages: lftools 
lf-activate-venv(): INFO: Adding /tmp/venv-QOlV/bin to PATH
Generating Requirements File
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
httplib2 0.30.2 requires pyparsing<4,>=3.0.4, but you have pyparsing 2.4.7 which is incompatible.
Python 3.11.10
pip 26.1.1 from /tmp/venv-QOlV/lib/python3.11/site-packages/pip (python 3.11)
aiohappyeyeballs==2.6.2
aiohttp==3.13.5
aiosignal==1.4.0
appdirs==1.4.4
argcomplete==3.6.3
aspy.yaml==1.3.0
attrs==26.1.0
autopage==0.6.0
beautifulsoup4==4.14.3
boto3==1.43.17
botocore==1.43.17
bs4==0.0.2
certifi==2026.5.20
cffi==2.0.0
cfgv==3.5.0
chardet==7.4.3
charset-normalizer==3.4.7
click==8.4.1
cliff==4.14.0
cmd2==3.5.1
cryptography==3.3.2
debtcollector==3.1.0
decorator==5.3.1
defusedxml==0.7.1
Deprecated==1.3.1
distlib==0.4.0
dnspython==2.8.0
docker==7.1.0
dogpile.cache==1.5.0
durationpy==0.10
email-validator==2.3.0
filelock==3.29.0
frozenlist==1.8.0
future==1.0.0
gitdb==4.0.12
GitPython==3.1.50
httplib2==0.30.2
identify==2.6.19
idna==3.17
importlib-resources==1.5.0
iso8601==2.1.0
Jinja2==3.1.6
jmespath==1.1.0
jsonpatch==1.33
jsonpointer==3.1.1
jsonschema==4.26.0
jsonschema-specifications==2025.9.1
keystoneauth1==5.14.0
kubernetes==36.0.1
lftools==0.37.22
lxml==6.1.1
markdown-it-py==4.2.0
MarkupSafe==3.0.3
mdurl==0.1.2
msgpack==1.1.2
multi_key_dict==2.0.3
multidict==6.7.1
munch==4.0.0
netaddr==1.3.0
niet==1.4.2
nodeenv==1.10.0
oauth2client==4.1.3
oauthlib==3.3.1
openstacksdk==4.14.0
os-service-types==1.8.2
osc-lib==4.6.0
oslo.config==10.4.0
oslo.context==6.4.0
oslo.i18n==6.8.0
oslo.log==8.2.0
oslo.serialization==5.10.0
oslo.utils==10.1.0
packaging==26.2
pbr==7.0.3
platformdirs==4.10.0
prettytable==3.17.0
propcache==0.5.2
psutil==7.2.2
pyasn1==0.6.3
pyasn1_modules==0.4.2
pycparser==3.0
pygerrit2==2.0.15
PyGithub==2.9.1
Pygments==2.20.0
PyJWT==2.13.0
PyNaCl==1.6.2
pyparsing==2.4.7
pyperclip==1.11.0
pyrsistent==0.20.0
python-cinderclient==9.9.0
python-dateutil==2.9.0.post0
python-discovery==1.4.0
python-heatclient==5.2.0
python-jenkins==1.8.3
python-keystoneclient==5.8.0
python-magnumclient==4.10.0
python-openstackclient==10.0.0
python-swiftclient==4.10.0
PyYAML==6.0.3
referencing==0.37.0
requests==2.34.2
requests-oauthlib==2.0.0
rfc3986==2.0.0
rich==15.0.0
rich-argparse==1.8.0
rpds-py==2026.5.1
rsa==4.9.1
ruamel.yaml==0.19.1
ruamel.yaml.clib==0.2.15
s3transfer==0.18.0
simplejson==4.1.1
six==1.17.0
smmap==5.0.3
soupsieve==2.8.4
stevedore==5.8.0
tabulate==0.10.0
toml==0.10.2
tomlkit==0.15.0
tqdm==4.67.3
typing_extensions==4.15.0
urllib3==1.26.20
virtualenv==21.4.1
wcwidth==0.7.0
websocket-client==1.9.0
wrapt==2.2.1
xdg==6.0.0
xmltodict==1.0.4
yarl==1.24.2
yq==3.4.3
[openflowplugin-csit-3node-clustering-only-titanium] $ /bin/sh /tmp/jenkins6093847446786066138.sh
---> uv-install.sh
Installing uv/uvx (latest) using shell installer
2026-05-29 07:00:01 URL:https://releases.astral.sh/installers/uv/latest/uv-installer.sh [71233/71233] -> "/tmp/uv-install-RPIK6F.sh" [1]
downloading uv 0.11.17 x86_64-unknown-linux-gnu
installing to /home/jenkins/.local/bin
  uv
  uvx
everything's installed!

To add $HOME/.local/bin to your PATH, either restart your shell or run:

    source $HOME/.local/bin/env (sh, bash, zsh)
    source $HOME/.local/bin/env.fish (fish)
Adding install location to PATH
---> Validating uv/uvx install
uvx 0.11.17 (x86_64-unknown-linux-gnu)
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties content 
OS_STACK_TEMPLATE=csit-2-instance-type.yaml
OS_CLOUD=vex
OS_STACK_NAME=releng-openflowplugin-csit-3node-clustering-only-titanium-733
OS_STACK_TEMPLATE_DIR=openstack-hot

[EnvInject] - Variables injected successfully.
provisioning config files...
copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
[openflowplugin-csit-3node-clustering-only-titanium] $ /bin/bash /tmp/jenkins6606448781527309067.sh
---> Create parameters file for OpenStack HOT
OpenStack Heat parameters generated
-----------------------------------
parameters:
    vm_0_count: '3'
    vm_0_flavor: 'v3-standard-4'
    vm_0_image: 'ZZCI - Ubuntu 22.04 - builder - x86_64 - 20260401-010115.623'
    vm_1_count: '1'
    vm_1_flavor: 'v3-standard-2'
    vm_1_image: 'ZZCI - Ubuntu 22.04 - mininet-ovs-217 - x86_64 - 20250917-133034.654'
    
    job_name: '62057-733'
    silo: 'releng'
[openflowplugin-csit-3node-clustering-only-titanium] $ /bin/bash -l /tmp/jenkins2911198171147201132.sh
---> Create HEAT stack
+ source /home/jenkins/lf-env.sh
+ lf-activate-venv --python python3 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient urllib3~=1.26.15 yq
++ mktemp -d /tmp/venv-XXXX
+ lf_venv=/tmp/venv-sxSJ
+ local venv_file=/tmp/.os_lf_venv
+ local python=python3
+ local options
+ local set_path=true
+ local install_args=
++ getopt -o np:v: -l no-path,system-site-packages,python:,venv-file: -n lf-activate-venv -- --python python3 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient urllib3~=1.26.15 yq
+ options=' --python '\''python3'\'' -- '\''lftools[openstack]'\'' '\''kubernetes'\'' '\''niet'\'' '\''python-heatclient'\'' '\''python-openstackclient'\'' '\''python-magnumclient'\'' '\''urllib3~=1.26.15'\'' '\''yq'\'''
+ eval set -- ' --python '\''python3'\'' -- '\''lftools[openstack]'\'' '\''kubernetes'\'' '\''niet'\'' '\''python-heatclient'\'' '\''python-openstackclient'\'' '\''python-magnumclient'\'' '\''urllib3~=1.26.15'\'' '\''yq'\'''
++ set -- --python python3 -- 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient urllib3~=1.26.15 yq
+ true
+ case $1 in
+ python=python3
+ shift 2
+ true
+ case $1 in
+ shift
+ break
+ case $python in
+ local pkg_list=
+ [[ -d /opt/pyenv ]]
+ echo 'Setup pyenv:'
Setup pyenv:
+ export PYENV_ROOT=/opt/pyenv
+ PYENV_ROOT=/opt/pyenv
+ export PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
+ PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
+ pyenv versions
  system
  3.8.20
  3.9.20
  3.10.15
* 3.11.10 (set by /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/.python-version)
+ command -v pyenv
++ pyenv init - --no-rehash
+ eval 'PATH="$(bash --norc -ec '\''IFS=:; paths=($PATH); 
for i in ${!paths[@]}; do 
if [[ ${paths[i]} == "'\'''\''/opt/pyenv/shims'\'''\''" ]]; then unset '\''\'\'''\''paths[i]'\''\'\'''\''; 
fi; done; 
echo "${paths[*]}"'\'')"
export PATH="/opt/pyenv/shims:${PATH}"
export PYENV_SHELL=bash
source '\''/opt/pyenv/libexec/../completions/pyenv.bash'\''
pyenv() {
  local command
  command="${1:-}"
  if [ "$#" -gt 0 ]; then
    shift
  fi

  case "$command" in
  rehash|shell)
    eval "$(pyenv "sh-$command" "$@")"
    ;;
  *)
    command pyenv "$command" "$@"
    ;;
  esac
}'
+++ bash --norc -ec 'IFS=:; paths=($PATH); 
for i in ${!paths[@]}; do 
if [[ ${paths[i]} == "/opt/pyenv/shims" ]]; then unset '\''paths[i]'\''; 
fi; done; 
echo "${paths[*]}"'
++ PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
++ export PATH=/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
++ PATH=/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
++ export PYENV_SHELL=bash
++ PYENV_SHELL=bash
++ source /opt/pyenv/libexec/../completions/pyenv.bash
+++ complete -F _pyenv pyenv
++ lf-pyver python3
++ local py_version_xy=python3
++ local py_version_xyz=
++ pyenv versions
++ local command
++ command=versions
++ '[' 1 -gt 0 ']'
++ shift
++ case "$command" in
++ command pyenv versions
++ sed 's/^[ *]* //'
++ awk '{ print $1 }'
++ grep -E '^[0-9.]*[0-9]$'
++ [[ ! -s /tmp/.pyenv_versions ]]
+++ grep '^3' /tmp/.pyenv_versions
+++ sort -V
+++ tail -n 1
++ py_version_xyz=3.11.10
++ [[ -z 3.11.10 ]]
++ echo 3.11.10
++ return 0
+ pyenv local 3.11.10
+ local command
+ command=local
+ '[' 2 -gt 0 ']'
+ shift
+ case "$command" in
+ command pyenv local 3.11.10
+ for arg in "$@"
+ case $arg in
+ pkg_list+='lftools[openstack] '
+ for arg in "$@"
+ case $arg in
+ pkg_list+='kubernetes '
+ for arg in "$@"
+ case $arg in
+ pkg_list+='niet '
+ for arg in "$@"
+ case $arg in
+ pkg_list+='python-heatclient '
+ for arg in "$@"
+ case $arg in
+ pkg_list+='python-openstackclient '
+ for arg in "$@"
+ case $arg in
+ pkg_list+='python-magnumclient '
+ for arg in "$@"
+ case $arg in
+ pkg_list+='urllib3~=1.26.15 '
+ for arg in "$@"
+ case $arg in
+ pkg_list+='yq '
+ [[ -f /tmp/.os_lf_venv ]]
++ cat /tmp/.os_lf_venv
+ lf_venv=/tmp/venv-QOlV
+ echo 'lf-activate-venv(): INFO: Reuse venv:/tmp/venv-QOlV from' file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-QOlV from file:/tmp/.os_lf_venv
+ echo 'lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv)'
lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv)
+ local 'pip_opts=--upgrade --quiet'
+ pip_opts='--upgrade --quiet --trusted-host pypi.org'
+ pip_opts='--upgrade --quiet --trusted-host pypi.org --trusted-host files.pythonhosted.org'
+ pip_opts='--upgrade --quiet --trusted-host pypi.org --trusted-host files.pythonhosted.org --trusted-host pypi.python.org'
+ [[ -n '' ]]
+ [[ -n '' ]]
+ echo 'lf-activate-venv(): INFO: Attempting to install with network-safe options...'
lf-activate-venv(): INFO: Attempting to install with network-safe options...
+ /tmp/venv-QOlV/bin/python3 -m pip install --upgrade --quiet --trusted-host pypi.org --trusted-host files.pythonhosted.org --trusted-host pypi.python.org pip 'setuptools<66' virtualenv
+ echo 'lf-activate-venv(): INFO: Base packages installed successfully'
lf-activate-venv(): INFO: Base packages installed successfully
+ [[ -z lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient urllib3~=1.26.15 yq  ]]
+ echo 'lf-activate-venv(): INFO: Installing additional packages: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient urllib3~=1.26.15 yq '
lf-activate-venv(): INFO: Installing additional packages: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient urllib3~=1.26.15 yq 
+ /tmp/venv-QOlV/bin/python3 -m pip install --upgrade --quiet --trusted-host pypi.org --trusted-host files.pythonhosted.org --trusted-host pypi.python.org --upgrade-strategy eager 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient urllib3~=1.26.15 yq
+ type python3
+ true
+ echo 'lf-activate-venv(): INFO: Adding /tmp/venv-QOlV/bin to PATH'
lf-activate-venv(): INFO: Adding /tmp/venv-QOlV/bin to PATH
+ PATH=/tmp/venv-QOlV/bin:/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
+ return 0
+ openstack --os-cloud vex limits show --absolute
+--------------------------+---------+
| Name                     |   Value |
+--------------------------+---------+
| maxTotalInstances        |      -1 |
| maxTotalCores            |     450 |
| maxTotalRAMSize          | 1000000 |
| maxServerMeta            |     128 |
| maxImageMeta             |     128 |
| maxPersonality           |       5 |
| maxPersonalitySize       |   10240 |
| maxTotalKeypairs         |     100 |
| maxServerGroups          |      10 |
| maxServerGroupMembers    |      10 |
| maxTotalFloatingIps      |      -1 |
| maxSecurityGroups        |      -1 |
| maxSecurityGroupRules    |      -1 |
| totalRAMUsed             |  582656 |
| totalCoresUsed           |     143 |
| totalInstancesUsed       |      53 |
| totalFloatingIpsUsed     |       0 |
| totalSecurityGroupsUsed  |       0 |
| totalServerGroupsUsed    |       0 |
| maxTotalVolumes          |      -1 |
| maxTotalSnapshots        |      10 |
| maxTotalVolumeGigabytes  |    4096 |
| maxTotalBackups          |      10 |
| maxTotalBackupGigabytes  |    1000 |
| totalVolumesUsed         |       1 |
| totalGigabytesUsed       |      20 |
| totalSnapshotsUsed       |       0 |
| totalBackupsUsed         |       0 |
| totalBackupGigabytesUsed |       0 |
+--------------------------+---------+
+ pushd /opt/ciman/openstack-hot
/opt/ciman/openstack-hot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium
+ lftools openstack --os-cloud vex stack create releng-openflowplugin-csit-3node-clustering-only-titanium-733 csit-2-instance-type.yaml /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/stack-parameters.yaml
Creating stack releng-openflowplugin-csit-3node-clustering-only-titanium-733
Waiting to initialize infrastructure...
Waiting to initialize infrastructure...
Waiting to initialize infrastructure...
Waiting to initialize infrastructure...
Waiting to initialize infrastructure...
Waiting to initialize infrastructure...
Waiting to initialize infrastructure...
Waiting to initialize infrastructure...
Waiting to initialize infrastructure...
WARN: Failed to initialize stack. Reason: Resource CREATE failed: ResourceInError: resources.vm_0_group.resources[2].resources.instance: Went to status ERROR due to "Message: No valid host was found. There are not enough hosts available., Code: 500"
Deleting stack 0a866940-ae54-45dd-bb0c-90a99e0afd42
Successfully deleted stack 0a866940-ae54-45dd-bb0c-90a99e0afd42
Waiting to initialize infrastructure...
Waiting to initialize infrastructure...
Waiting to initialize infrastructure...
Waiting to initialize infrastructure...
Waiting to initialize infrastructure...
Waiting to initialize infrastructure...
Waiting to initialize infrastructure...
Waiting to initialize infrastructure...
WARN: Failed to initialize stack. Reason: Resource CREATE failed: ResourceInError: resources.vm_0_group.resources[1].resources.instance: Went to status ERROR due to "Message: No valid host was found. There are not enough hosts available., Code: 500"
Deleting stack 88875672-c5f9-4b65-a5d4-6c797a0cb8e0
Successfully deleted stack 88875672-c5f9-4b65-a5d4-6c797a0cb8e0
------------------------------------
Stack Details
------------------------------------
{'added': None,
 'capabilities': [],
 'created_at': '2026-05-29T07:02:30Z',
 'deleted': None,
 'deleted_at': None,
 'description': 'No description',
 'environment': None,
 'environment_files': None,
 'files': None,
 'files_container': None,
 'id': '88875672-c5f9-4b65-a5d4-6c797a0cb8e0',
 'is_rollback_disabled': True,
 'links': [{'href': 'https://orchestration.public.mtl1.vexxhost.net/v1/12c36e260d8e4bb2913965203b1b491f/stacks/releng-openflowplugin-csit-3node-clustering-only-titanium-733/88875672-c5f9-4b65-a5d4-6c797a0cb8e0',
            'rel': 'self'}],
 'location': Munch({'cloud': 'vex', 'region_name': 'ca-ymq-1', 'zone': None, 'project': Munch({'id': '12c36e260d8e4bb2913965203b1b491f', 'name': '61975f2c-7c17-4d69-82fa-c3ae420ad6fd', 'domain_id': None, 'domain_name': 'Default'})}),
 'name': 'releng-openflowplugin-csit-3node-clustering-only-titanium-733',
 'notification_topics': [],
 'outputs': [{'description': 'IP addresses of the 1st vm types',
              'output_key': 'vm_0_ips',
              'output_value': ['10.30.170.139', None, '10.30.171.59']},
             {'description': 'IP addresses of the 2nd vm types',
              'output_key': 'vm_1_ips',
              'output_value': ['10.30.170.87']}],
 'owner_id': ****,
 'parameters': {'OS::project_id': '12c36e260d8e4bb2913965203b1b491f',
                'OS::stack_id': '88875672-c5f9-4b65-a5d4-6c797a0cb8e0',
                'OS::stack_name': 'releng-openflowplugin-csit-3node-clustering-only-titanium-733',
                'job_name': '62057-733',
                'silo': 'releng',
                'vm_0_count': '3',
                'vm_0_flavor': 'v3-standard-4',
                'vm_0_image': 'ZZCI - Ubuntu 22.04 - builder - x86_64 - '
                              '20260401-010115.623',
                'vm_1_count': '1',
                'vm_1_flavor': 'v3-standard-2',
                'vm_1_image': 'ZZCI - Ubuntu 22.04 - mininet-ovs-217 - x86_64 '
                              '- 20250917-133034.654'},
 'parent_id': None,
 'replaced': None,
 'status': 'CREATE_FAILED',
 'status_reason': 'Resource CREATE failed: ResourceInError: '
                  'resources.vm_0_group.resources[1].resources.instance: Went '
                  'to status ERROR due to "Message: No valid host was found. '
                  'There are not enough hosts available., Code: 500"',
 'tags': [],
 'template': None,
 'template_description': 'No description',
 'template_url': None,
 'timeout_mins': 15,
 'unchanged': None,
 'updated': None,
 'updated_at': None,
 'user_project_id': '68fa16c6eb9c407b80f5428af1b222bd'}
------------------------------------
+ popd
/w/workspace/openflowplugin-csit-3node-clustering-only-titanium
[openflowplugin-csit-3node-clustering-only-titanium] $ /bin/bash -l /tmp/jenkins4497450415399471787.sh
---> Copy SSH public keys to CSIT lab
Setup pyenv:
  system
  3.8.20
  3.9.20
  3.10.15
* 3.11.10 (set by /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-QOlV from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv)
lf-activate-venv(): INFO: Attempting to install with network-safe options...
lf-activate-venv(): INFO: Base packages installed successfully
lf-activate-venv(): INFO: Installing additional packages: lftools[openstack] kubernetes python-heatclient python-openstackclient urllib3~=1.26.15 
lf-activate-venv(): INFO: Adding /tmp/venv-QOlV/bin to PATH
Stack not found: releng-openflowplugin-csit-3node-clustering-only-titanium-733
SSH ready on all stack servers.
[openflowplugin-csit-3node-clustering-only-titanium] $ /bin/bash -l /tmp/jenkins12755886483311151307.sh
Setup pyenv:
  system
  3.8.20
  3.9.20
  3.10.15
* 3.11.10 (set by /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/.python-version)
lf-activate-venv(): INFO: Creating python3 venv at /tmp/venv-cA4f
lf-activate-venv(): INFO: Save venv in file: /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/.robot_venv
lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv)
lf-activate-venv(): INFO: Attempting to install with network-safe options...
lf-activate-venv(): INFO: Base packages installed successfully
lf-activate-venv(): INFO: Installing additional packages: setuptools wheel 
lf-activate-venv(): INFO: Adding /tmp/venv-cA4f/bin to PATH
+ echo 'Installing Python Requirements'
Installing Python Requirements
+ cat
+ python -m pip install -r requirements.txt
Looking in indexes: https://nexus3.opendaylight.org/repository/PyPi/simple
Collecting docker-py (from -r requirements.txt (line 1))
  Downloading docker_py-1.10.6-py2.py3-none-any.whl (50 kB)
Collecting ipaddr (from -r requirements.txt (line 2))
  Downloading ipaddr-2.2.0.tar.gz (26 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting netaddr (from -r requirements.txt (line 3))
  Using cached netaddr-1.3.0-py3-none-any.whl (2.3 MB)
Collecting netifaces (from -r requirements.txt (line 4))
  Downloading netifaces-0.11.0.tar.gz (30 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting pyhocon (from -r requirements.txt (line 5))
  Downloading pyhocon-0.3.63.tar.gz (307 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting requests (from -r requirements.txt (line 6))
  Using cached requests-2.34.2-py3-none-any.whl (73 kB)
Collecting robotframework (from -r requirements.txt (line 7))
  Downloading robotframework-7.4.2-py3-none-any.whl (807 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 807.1/807.1 kB 24.5 MB/s  0:00:00
Collecting robotframework-httplibrary (from -r requirements.txt (line 8))
  Downloading robotframework-httplibrary-0.4.2.tar.gz (9.1 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting robotframework-requests==0.9.7 (from -r requirements.txt (line 9))
  Downloading robotframework_requests-0.9.7-py3-none-any.whl (21 kB)
Collecting robotframework-selenium2library (from -r requirements.txt (line 10))
  Downloading robotframework_selenium2library-3.0.0-py2.py3-none-any.whl (6.2 kB)
Collecting robotframework-sshlibrary==3.8.0 (from -r requirements.txt (line 11))
  Downloading robotframework-sshlibrary-3.8.0.tar.gz (51 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting scapy (from -r requirements.txt (line 12))
  Downloading scapy-2.7.0-py3-none-any.whl (2.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.6/2.6 MB 47.4 MB/s  0:00:00
Collecting jsonpath-rw (from -r requirements.txt (line 15))
  Downloading jsonpath-rw-1.4.0.tar.gz (13 kB)
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting elasticsearch (from -r requirements.txt (line 18))
  Downloading elasticsearch-9.4.1-py3-none-any.whl (993 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 993.6/993.6 kB 36.6 MB/s  0:00:00
Collecting elasticsearch-dsl (from -r requirements.txt (line 19))
  Downloading elasticsearch_dsl-8.18.0-py3-none-any.whl (10 kB)
Collecting pyangbind (from -r requirements.txt (line 22))
  Downloading pyangbind-0.8.7-py3-none-any.whl (52 kB)
Collecting isodate (from -r requirements.txt (line 25))
  Downloading isodate-0.7.2-py3-none-any.whl (22 kB)
Collecting jmespath (from -r requirements.txt (line 28))
  Using cached jmespath-1.1.0-py3-none-any.whl (20 kB)
Collecting jsonpatch (from -r requirements.txt (line 31))
  Using cached jsonpatch-1.33-py2.py3-none-any.whl (12 kB)
Collecting paramiko>=1.15.3 (from robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
  Downloading paramiko-5.0.0-py3-none-any.whl (208 kB)
Collecting scp>=0.13.0 (from robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
  Downloading scp-0.15.0-py2.py3-none-any.whl (8.8 kB)
Collecting docker-pycreds>=0.2.1 (from docker-py->-r requirements.txt (line 1))
  Downloading docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)
Collecting six>=1.4.0 (from docker-py->-r requirements.txt (line 1))
  Using cached six-1.17.0-py2.py3-none-any.whl (11 kB)
Collecting websocket-client>=0.32.0 (from docker-py->-r requirements.txt (line 1))
  Using cached websocket_client-1.9.0-py3-none-any.whl (82 kB)
Collecting pyparsing>=3.0.0 (from pyhocon->-r requirements.txt (line 5))
  Using cached pyparsing-3.3.2-py3-none-any.whl (122 kB)
Collecting charset_normalizer<4,>=2 (from requests->-r requirements.txt (line 6))
  Using cached charset_normalizer-3.4.7-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (214 kB)
Collecting idna<4,>=2.5 (from requests->-r requirements.txt (line 6))
  Using cached idna-3.17-py3-none-any.whl (65 kB)
Collecting urllib3<3,>=1.26 (from requests->-r requirements.txt (line 6))
  Using cached urllib3-2.7.0-py3-none-any.whl (131 kB)
Collecting certifi>=2023.5.7 (from requests->-r requirements.txt (line 6))
  Using cached certifi-2026.5.20-py3-none-any.whl (134 kB)
Collecting webtest>=2.0 (from robotframework-httplibrary->-r requirements.txt (line 8))
  Downloading webtest-3.0.7-py3-none-any.whl (32 kB)
Collecting jsonpointer (from robotframework-httplibrary->-r requirements.txt (line 8))
  Using cached jsonpointer-3.1.1-py3-none-any.whl (7.7 kB)
Collecting robotframework-seleniumlibrary>=3.0.0 (from robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading robotframework_seleniumlibrary-6.9.0-py3-none-any.whl (105 kB)
Collecting ply (from jsonpath-rw->-r requirements.txt (line 15))
  Downloading ply-3.11-py2.py3-none-any.whl (49 kB)
Collecting decorator (from jsonpath-rw->-r requirements.txt (line 15))
  Using cached decorator-5.3.1-py3-none-any.whl (10 kB)
Collecting anyio (from elasticsearch->-r requirements.txt (line 18))
  Downloading anyio-4.13.0-py3-none-any.whl (114 kB)
Collecting elastic-transport<10,>=9.4.1 (from elasticsearch->-r requirements.txt (line 18))
  Downloading elastic_transport-9.4.1-py3-none-any.whl (66 kB)
Collecting python-dateutil (from elasticsearch->-r requirements.txt (line 18))
  Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Collecting sniffio (from elasticsearch->-r requirements.txt (line 18))
  Downloading sniffio-1.3.1-py3-none-any.whl (10 kB)
Collecting typing-extensions (from elasticsearch->-r requirements.txt (line 18))
  Using cached typing_extensions-4.15.0-py3-none-any.whl (44 kB)
Collecting elasticsearch (from -r requirements.txt (line 18))
  Downloading elasticsearch-8.19.3-py3-none-any.whl (952 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 952.8/952.8 kB 32.4 MB/s  0:00:00
INFO: pip is looking at multiple versions of elasticsearch-dsl to determine which version is compatible with other requirements. This could take a while.
Collecting elasticsearch-dsl (from -r requirements.txt (line 19))
  Downloading elasticsearch_dsl-8.17.1-py3-none-any.whl (158 kB)
  Downloading elasticsearch_dsl-8.17.0-py3-none-any.whl (158 kB)
  Downloading elasticsearch_dsl-8.16.0-py3-none-any.whl (158 kB)
  Downloading elasticsearch_dsl-8.15.4-py3-none-any.whl (104 kB)
Collecting elastic-transport<9,>=8.15.1 (from elasticsearch->-r requirements.txt (line 18))
  Downloading elastic_transport-8.17.1-py3-none-any.whl (64 kB)
Collecting pyang (from pyangbind->-r requirements.txt (line 22))
  Downloading pyang-2.7.1-py2.py3-none-any.whl (598 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 598.5/598.5 kB 27.0 MB/s  0:00:00
Collecting lxml (from pyangbind->-r requirements.txt (line 22))
  Using cached lxml-6.1.1-cp311-cp311-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl (5.2 MB)
Collecting regex (from pyangbind->-r requirements.txt (line 22))
  Downloading regex-2026.5.9-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (799 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 799.8/799.8 kB 22.9 MB/s  0:00:00
Collecting enum34 (from pyangbind->-r requirements.txt (line 22))
  Downloading enum34-1.1.10-py3-none-any.whl (11 kB)
Collecting bcrypt>=3.2 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
  Downloading bcrypt-5.0.0-cp39-abi3-manylinux_2_34_x86_64.whl (278 kB)
Collecting cryptography>=3.3 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
  Using cached cryptography-48.0.0-cp311-abi3-manylinux_2_34_x86_64.whl (4.7 MB)
Collecting invoke>=2.0 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
  Downloading invoke-3.0.3-py3-none-any.whl (160 kB)
Collecting pynacl>=1.5 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
  Using cached pynacl-1.6.2-cp38-abi3-manylinux_2_34_x86_64.whl (1.4 MB)
Collecting cffi>=2.0.0 (from cryptography>=3.3->paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
  Using cached cffi-2.0.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (215 kB)
Collecting pycparser (from cffi>=2.0.0->cryptography>=3.3->paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
  Using cached pycparser-3.0-py3-none-any.whl (48 kB)
Collecting selenium>=4.25.0 (from robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading selenium-4.44.0-py3-none-any.whl (9.7 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.7/9.7 MB 66.7 MB/s  0:00:00
Collecting robotframework-pythonlibcore>=4.4.1 (from robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading robotframework_pythonlibcore-4.6.0-py3-none-any.whl (20 kB)
Collecting click>=8.0 (from robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Using cached click-8.4.1-py3-none-any.whl (116 kB)
Collecting trio<1.0,>=0.31.0 (from selenium>=4.25.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading trio-0.33.0-py3-none-any.whl (510 kB)
Collecting trio-websocket<1.0,>=0.12.2 (from selenium>=4.25.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading trio_websocket-0.12.2-py3-none-any.whl (21 kB)
Collecting attrs>=23.2.0 (from trio<1.0,>=0.31.0->selenium>=4.25.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Using cached attrs-26.1.0-py3-none-any.whl (67 kB)
Collecting sortedcontainers (from trio<1.0,>=0.31.0->selenium>=4.25.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting outcome (from trio<1.0,>=0.31.0->selenium>=4.25.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading outcome-1.3.0.post0-py2.py3-none-any.whl (10 kB)
Collecting wsproto>=0.14 (from trio-websocket<1.0,>=0.12.2->selenium>=4.25.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading wsproto-1.3.2-py3-none-any.whl (24 kB)
Collecting pysocks!=1.5.7,<2.0,>=1.5.6 (from urllib3[socks]<3.0,>=2.6.3->selenium>=4.25.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading PySocks-1.7.1-py3-none-any.whl (16 kB)
Collecting WebOb>=1.2 (from webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8))
  Downloading WebOb-1.8.9-py2.py3-none-any.whl (115 kB)
Collecting waitress>=3.0.2 (from webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8))
  Downloading waitress-3.0.2-py3-none-any.whl (56 kB)
Collecting beautifulsoup4 (from webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8))
  Using cached beautifulsoup4-4.14.3-py3-none-any.whl (107 kB)
Collecting h11<1,>=0.16.0 (from wsproto>=0.14->trio-websocket<1.0,>=0.12.2->selenium>=4.25.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading h11-0.16.0-py3-none-any.whl (37 kB)
Collecting soupsieve>=1.6.1 (from beautifulsoup4->webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8))
  Using cached soupsieve-2.8.4-py3-none-any.whl (37 kB)
Building wheels for collected packages: robotframework-sshlibrary, ipaddr, netifaces, pyhocon, robotframework-httplibrary, jsonpath-rw
  Building wheel for robotframework-sshlibrary (pyproject.toml): started
  Building wheel for robotframework-sshlibrary (pyproject.toml): finished with status 'done'
  Created wheel for robotframework-sshlibrary: filename=robotframework_sshlibrary-3.8.0-py3-none-any.whl size=55205 sha256=72e82550e24e40a6178e034406cf19effc4a17fa8fdfd4ad63ff31780fd130f7
  Stored in directory: /home/jenkins/.cache/pip/wheels/f7/c9/b3/a977b7bcc410d45ae27d240df3d00a12585509180e373ecccc
  Building wheel for ipaddr (pyproject.toml): started
  Building wheel for ipaddr (pyproject.toml): finished with status 'done'
  Created wheel for ipaddr: filename=ipaddr-2.2.0-py3-none-any.whl size=18355 sha256=be3f2685cfa59bb0fee352125c18eb650dfcaa5850791222b00e365b94224903
  Stored in directory: /home/jenkins/.cache/pip/wheels/dc/6c/04/da2d847fa8d45c59af3e1d83e2acc29cb8adcbaf04c0898dbf
  Building wheel for netifaces (pyproject.toml): started
  Building wheel for netifaces (pyproject.toml): finished with status 'done'
  Created wheel for netifaces: filename=netifaces-0.11.0-cp311-cp311-linux_x86_64.whl size=34976 sha256=158866d8638fd2f7616e19ad9925b75a56d6cef1e2a85a6a3981d383ac1bc29f
  Stored in directory: /home/jenkins/.cache/pip/wheels/f8/18/88/e61d54b995bea304bdb1d040a92b72228a1bf72ca2a3eba7c9
  Building wheel for pyhocon (pyproject.toml): started
  Building wheel for pyhocon (pyproject.toml): finished with status 'done'
  Created wheel for pyhocon: filename=pyhocon-0.3.63-py3-none-any.whl size=24721 sha256=7ffdefafdf74ce7981d6517259f69f737d6e96f1f95bb96e556f206d18521d1d
  Stored in directory: /home/jenkins/.cache/pip/wheels/ce/9e/92/cb17e17d79fafd46a767074b3ed1dca8f78036163f9bea914d
  Building wheel for robotframework-httplibrary (pyproject.toml): started
  Building wheel for robotframework-httplibrary (pyproject.toml): finished with status 'done'
  Created wheel for robotframework-httplibrary: filename=robotframework_httplibrary-0.4.2-py3-none-any.whl size=10014 sha256=f36a9f311a4f8f05bf4505d3e3c9212c3814b3db178acecba6cbe4f508a77a0b
  Stored in directory: /home/jenkins/.cache/pip/wheels/aa/bc/0d/9a20dd51effef392aae2733cb4c7b66c6fa29fca33d88b57ed
  Building wheel for jsonpath-rw (pyproject.toml): started
  Building wheel for jsonpath-rw (pyproject.toml): finished with status 'done'
  Created wheel for jsonpath-rw: filename=jsonpath_rw-1.4.0-py3-none-any.whl size=15176 sha256=47738487c18f6ff6f02ce20da94c4397739fb946f3ec0778901c844b6f63346a
  Stored in directory: /home/jenkins/.cache/pip/wheels/f1/54/63/9a8da38cefae13755097b36cc852decc25d8ef69c37d58d4eb
Successfully built robotframework-sshlibrary ipaddr netifaces pyhocon robotframework-httplibrary jsonpath-rw
Installing collected packages: sortedcontainers, ply, netifaces, ipaddr, enum34, websocket-client, WebOb, waitress, urllib3, typing-extensions, soupsieve, sniffio, six, scapy, robotframework-pythonlibcore, robotframework, regex, pysocks, pyparsing, pycparser, netaddr, lxml, jsonpointer, jmespath, isodate, invoke, idna, h11, decorator, click, charset_normalizer, certifi, bcrypt, attrs, wsproto, requests, python-dateutil, pyhocon, pyang, outcome, jsonpath-rw, jsonpatch, elastic-transport, docker-pycreds, cffi, beautifulsoup4, webtest, trio, robotframework-requests, pynacl, pyangbind, elasticsearch, docker-py, cryptography, trio-websocket, robotframework-httplibrary, paramiko, elasticsearch-dsl, selenium, scp, robotframework-sshlibrary, robotframework-seleniumlibrary, robotframework-selenium2library

Successfully installed WebOb-1.8.9 attrs-26.1.0 bcrypt-5.0.0 beautifulsoup4-4.14.3 certifi-2026.5.20 cffi-2.0.0 charset_normalizer-3.4.7 click-8.4.1 cryptography-48.0.0 decorator-5.3.1 docker-py-1.10.6 docker-pycreds-0.4.0 elastic-transport-8.17.1 elasticsearch-8.19.3 elasticsearch-dsl-8.15.4 enum34-1.1.10 h11-0.16.0 idna-3.17 invoke-3.0.3 ipaddr-2.2.0 isodate-0.7.2 jmespath-1.1.0 jsonpatch-1.33 jsonpath-rw-1.4.0 jsonpointer-3.1.1 lxml-6.1.1 netaddr-1.3.0 netifaces-0.11.0 outcome-1.3.0.post0 paramiko-5.0.0 ply-3.11 pyang-2.7.1 pyangbind-0.8.7 pycparser-3.0 pyhocon-0.3.63 pynacl-1.6.2 pyparsing-3.3.2 pysocks-1.7.1 python-dateutil-2.9.0.post0 regex-2026.5.9 requests-2.34.2 robotframework-7.4.2 robotframework-httplibrary-0.4.2 robotframework-pythonlibcore-4.6.0 robotframework-requests-0.9.7 robotframework-selenium2library-3.0.0 robotframework-seleniumlibrary-6.9.0 robotframework-sshlibrary-3.8.0 scapy-2.7.0 scp-0.15.0 selenium-4.44.0 six-1.17.0 sniffio-1.3.1 sortedcontainers-2.4.0 soupsieve-2.8.4 trio-0.33.0 trio-websocket-0.12.2 typing-extensions-4.15.0 urllib3-2.7.0 waitress-3.0.2 websocket-client-1.9.0 webtest-3.0.7 wsproto-1.3.2
+ pip freeze
attrs==26.1.0
bcrypt==5.0.0
beautifulsoup4==4.14.3
certifi==2026.5.20
cffi==2.0.0
charset-normalizer==3.4.7
click==8.4.1
cryptography==48.0.0
decorator==5.3.1
distlib==0.4.0
docker-py==1.10.6
docker-pycreds==0.4.0
elastic-transport==8.17.1
elasticsearch==8.19.3
elasticsearch-dsl==8.15.4
enum34==1.1.10
filelock==3.29.0
h11==0.16.0
idna==3.17
invoke==3.0.3
ipaddr==2.2.0
isodate==0.7.2
jmespath==1.1.0
jsonpatch==1.33
jsonpath-rw==1.4.0
jsonpointer==3.1.1
lxml==6.1.1
netaddr==1.3.0
netifaces==0.11.0
outcome==1.3.0.post0
packaging==26.2
paramiko==5.0.0
platformdirs==4.10.0
ply==3.11
pyang==2.7.1
pyangbind==0.8.7
pycparser==3.0
pyhocon==0.3.63
PyNaCl==1.6.2
pyparsing==3.3.2
PySocks==1.7.1
python-dateutil==2.9.0.post0
python-discovery==1.4.0
regex==2026.5.9
requests==2.34.2
robotframework==7.4.2
robotframework-httplibrary==0.4.2
robotframework-pythonlibcore==4.6.0
robotframework-requests==0.9.7
robotframework-selenium2library==3.0.0
robotframework-seleniumlibrary==6.9.0
robotframework-sshlibrary==3.8.0
scapy==2.7.0
scp==0.15.0
selenium==4.44.0
six==1.17.0
sniffio==1.3.1
sortedcontainers==2.4.0
soupsieve==2.8.4
trio==0.33.0
trio-websocket==0.12.2
typing_extensions==4.15.0
urllib3==2.7.0
virtualenv==21.4.1
waitress==3.0.2
WebOb==1.8.9
websocket-client==1.9.0
WebTest==3.0.7
wsproto==1.3.2
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties file path 'env.properties'
[EnvInject] - Variables injected successfully.
[openflowplugin-csit-3node-clustering-only-titanium] $ /bin/bash -l /tmp/jenkins17185333501527671817.sh
Setup pyenv:
  system
  3.8.20
  3.9.20
  3.10.15
* 3.11.10 (set by /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-QOlV from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv)
lf-activate-venv(): INFO: Attempting to install with network-safe options...
lf-activate-venv(): INFO: Base packages installed successfully
lf-activate-venv(): INFO: Installing additional packages: python-heatclient python-openstackclient yq 
lf-activate-venv(): INFO: Adding /tmp/venv-QOlV/bin to PATH
+ ODL_SYSTEM=()
+ TOOLS_SYSTEM=()
+ OPENSTACK_SYSTEM=()
+ OPENSTACK_CONTROLLERS=()
+ mapfile -t ADDR
++ openstack stack show -f json -c outputs releng-openflowplugin-csit-3node-clustering-only-titanium-733
++ jq -r '.outputs[] | select(.output_key | match("^vm_[0-9]+_ips$")) | .output_value | .[]'
Stack not found: releng-openflowplugin-csit-3node-clustering-only-titanium-733
+ for i in "${ADDR[@]}"
++ ssh '' hostname -s
ssh: Could not resolve hostname : Device or resource busy
+ REMHOST=
+ case ${REMHOST} in
+ TOOLS_SYSTEM=("${TOOLS_SYSTEM[@]}" "${i}")
+ echo NUM_ODL_SYSTEM=0
+ echo NUM_TOOLS_SYSTEM=1
+ '[' '' == yes ']'
+ NUM_OPENSTACK_SYSTEM=0
+ echo NUM_OPENSTACK_SYSTEM=0
+ '[' 0 -eq 2 ']'
+ echo ODL_SYSTEM_IP=
++ seq 0 -1
+ echo TOOLS_SYSTEM_IP=
++ seq 0 0
+ for i in $(seq 0 $(( ${#TOOLS_SYSTEM[@]} - 1 )))
+ echo TOOLS_SYSTEM_1_IP=
+ openstack_index=0
+ NUM_OPENSTACK_CONTROL_NODES=1
+ echo NUM_OPENSTACK_CONTROL_NODES=1
++ seq 0 0
+ for i in $(seq 0 $((NUM_OPENSTACK_CONTROL_NODES - 1)))
+ echo OPENSTACK_CONTROL_NODE_1_IP=
+ NUM_OPENSTACK_COMPUTE_NODES=-1
+ echo NUM_OPENSTACK_COMPUTE_NODES=-1
+ '[' -1 -ge 2 ']'
++ seq 0 -2
+ NUM_OPENSTACK_HAPROXY_NODES=0
+ echo NUM_OPENSTACK_HAPROXY_NODES=0
++ seq 0 -1
+ echo 'Contents of slave_addresses.txt:'
Contents of slave_addresses.txt:
+ cat slave_addresses.txt
NUM_ODL_SYSTEM=0
NUM_TOOLS_SYSTEM=1
NUM_OPENSTACK_SYSTEM=0
ODL_SYSTEM_IP=
TOOLS_SYSTEM_IP=
TOOLS_SYSTEM_1_IP=
NUM_OPENSTACK_CONTROL_NODES=1
OPENSTACK_CONTROL_NODE_1_IP=
NUM_OPENSTACK_COMPUTE_NODES=-1
NUM_OPENSTACK_HAPROXY_NODES=0
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties file path 'slave_addresses.txt'
[EnvInject] - Variables injected successfully.
[openflowplugin-csit-3node-clustering-only-titanium] $ /bin/sh /tmp/jenkins12138187179540454445.sh
Preparing for JRE Version 21
Karaf artifact is karaf
Karaf project is integration
Java home is /usr/lib/jvm/java-21-openjdk-amd64
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties file path 'set_variables.env'
[EnvInject] - Variables injected successfully.
[openflowplugin-csit-3node-clustering-only-titanium] $ /bin/bash /tmp/jenkins8582223191477488960.sh
2026-05-29 07:05:30 URL:https://raw.githubusercontent.com/opendaylight/integration-distribution/stable/titanium/pom.xml [2619/2619] -> "pom.xml" [1]
Bundle version is 0.22.3-SNAPSHOT
--2026-05-29 07:05:31--  https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.22.3-SNAPSHOT/maven-metadata.xml
Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6
Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1410 (1.4K) [application/xml]
Saving to: ‘maven-metadata.xml’

     0K .                                                     100% 1.17G=0s

2026-05-29 07:05:31 (1.17 GB/s) - ‘maven-metadata.xml’ saved [1410/1410]

<?xml version="1.0" encoding="UTF-8"?>
<metadata modelVersion="1.1.0">
  <groupId>org.opendaylight.integration</groupId>
  <artifactId>karaf</artifactId>
  <version>0.22.3-SNAPSHOT</version>
  <versioning>
    <snapshot>
      <timestamp>20260529.050331</timestamp>
      <buildNumber>217</buildNumber>
    </snapshot>
    <lastUpdated>20260529050331</lastUpdated>
    <snapshotVersions>
      <snapshotVersion>
        <extension>pom</extension>
        <value>0.22.3-20260529.050331-217</value>
        <updated>20260529050331</updated>
      </snapshotVersion>
      <snapshotVersion>
        <extension>tar.gz</extension>
        <value>0.22.3-20260529.050331-217</value>
        <updated>20260529050331</updated>
      </snapshotVersion>
      <snapshotVersion>
        <extension>zip</extension>
        <value>0.22.3-20260529.050331-217</value>
        <updated>20260529050331</updated>
      </snapshotVersion>
      <snapshotVersion>
        <classifier>cyclonedx</classifier>
        <extension>xml</extension>
        <value>0.22.3-20260529.050331-217</value>
        <updated>20260529050331</updated>
      </snapshotVersion>
      <snapshotVersion>
        <classifier>cyclonedx</classifier>
        <extension>json</extension>
        <value>0.22.3-20260529.050331-217</value>
        <updated>20260529050331</updated>
      </snapshotVersion>
    </snapshotVersions>
  </versioning>
</metadata>
Nexus timestamp is 0.22.3-20260529.050331-217
Distribution bundle URL is https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.22.3-SNAPSHOT/karaf-0.22.3-20260529.050331-217.zip
Distribution bundle is karaf-0.22.3-20260529.050331-217.zip
Distribution bundle version is 0.22.3-SNAPSHOT
Distribution folder is karaf-0.22.3-SNAPSHOT
Nexus prefix is https://nexus.opendaylight.org
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties file path 'detect_variables.env'
[EnvInject] - Variables injected successfully.
[openflowplugin-csit-3node-clustering-only-titanium] $ /bin/bash -l /tmp/jenkins16683620125738259554.sh
Setup pyenv:
  system
  3.8.20
  3.9.20
  3.10.15
* 3.11.10 (set by /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-QOlV from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv)
lf-activate-venv(): INFO: Attempting to install with network-safe options...
lf-activate-venv(): INFO: Base packages installed successfully
lf-activate-venv(): INFO: Installing additional packages: python-heatclient python-openstackclient 
lf-activate-venv(): INFO: Adding /tmp/venv-QOlV/bin to PATH
Copying common-functions.sh to /tmp
Stack not found: releng-openflowplugin-csit-3node-clustering-only-titanium-733
Copying common-functions.sh to :/tmp
cp: cannot create regular file ':/tmp': No such file or directory
[openflowplugin-csit-3node-clustering-only-titanium] $ /bin/bash /tmp/jenkins11326474497332536621.sh
common-functions.sh is being sourced
common-functions environment:
MAVENCONF: /tmp/karaf-0.22.3-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
ACTUALFEATURES: 
FEATURESCONF: /tmp/karaf-0.22.3-SNAPSHOT/etc/org.apache.karaf.features.cfg
CUSTOMPROP: /tmp/karaf-0.22.3-SNAPSHOT/etc/custom.properties
LOGCONF: /tmp/karaf-0.22.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
MEMCONF: /tmp/karaf-0.22.3-SNAPSHOT/bin/setenv
CONTROLLERMEM: 2048m
AKKACONF: /tmp/karaf-0.22.3-SNAPSHOT/configuration/initial/pekko.conf
MODULESCONF: /tmp/karaf-0.22.3-SNAPSHOT/configuration/initial/modules.conf
MODULESHARDSCONF: /tmp/karaf-0.22.3-SNAPSHOT/configuration/initial/module-shards.conf
SUITES: 

#################################################
##         Configure Cluster and Start         ##
#################################################
ACTUALFEATURES: odl-infrautils-ready,odl-jolokia,odl-openflowplugin-flow-services-rest,odl-openflowplugin-app-table-miss-enforcer
SPACE_SEPARATED_FEATURES: odl-infrautils-ready odl-jolokia odl-openflowplugin-flow-services-rest odl-openflowplugin-app-table-miss-enforcer
Locating script plan to use...
Finished running script plans
Locating config plan to use...
config plan exists!!!
Changing the config plan path...
# Place the suites in run order:
/w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/scripts/set_akka_debug.sh
Executing /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/scripts/set_akka_debug.sh...
Copying config files to ODL Controller folder
Finished running config plans
[openflowplugin-csit-3node-clustering-only-titanium] $ /bin/bash /tmp/jenkins6459451004856972121.sh
common-functions.sh is being sourced
common-functions environment:
MAVENCONF: /tmp/karaf-0.22.3-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
ACTUALFEATURES: 
FEATURESCONF: /tmp/karaf-0.22.3-SNAPSHOT/etc/org.apache.karaf.features.cfg
CUSTOMPROP: /tmp/karaf-0.22.3-SNAPSHOT/etc/custom.properties
LOGCONF: /tmp/karaf-0.22.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
MEMCONF: /tmp/karaf-0.22.3-SNAPSHOT/bin/setenv
CONTROLLERMEM: 2048m
AKKACONF: /tmp/karaf-0.22.3-SNAPSHOT/configuration/initial/pekko.conf
MODULESCONF: /tmp/karaf-0.22.3-SNAPSHOT/configuration/initial/modules.conf
MODULESHARDSCONF: /tmp/karaf-0.22.3-SNAPSHOT/configuration/initial/module-shards.conf
SUITES: 

+ echo '#################################################'
#################################################
+ echo '##         Verify Cluster is UP                ##'
##         Verify Cluster is UP                ##
+ echo '#################################################'
#################################################
+ create_post_startup_script
+ cat
+ copy_and_run_post_startup_script
+ seed_index=1
++ seq 1 0
+ dump_controller_threads
++ seq 1 0
+ '[' 0 -gt 0 ']'
+ echo 'Generating controller variables...'
Generating controller variables...
++ seq 1 0
+ echo 'Generating mininet variables...'
Generating mininet variables...
++ seq 1 1
+ for i in $(seq 1 "${NUM_TOOLS_SYSTEM}")
+ MININETIP=TOOLS_SYSTEM_1_IP
+ tools_variables=' -v TOOLS_SYSTEM_1_IP:'
+ get_test_suites SUITES
+ local __suite_list=SUITES
+ echo 'Locating test plan to use...'
Locating test plan to use...
+ testplan_filepath=/w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/testplans/openflowplugin-clustering-titanium.txt
+ '[' '!' -f /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/testplans/openflowplugin-clustering-titanium.txt ']'
+ testplan_filepath=/w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/testplans/openflowplugin-clustering.txt
+ '[' disabled '!=' disabled ']'
+ echo 'Changing the testplan path...'
Changing the testplan path...
+ sed s:integration:/w/workspace/openflowplugin-csit-3node-clustering-only-titanium: /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/testplans/openflowplugin-clustering.txt
+ cat testplan.txt
# Place the suites in run order:
/w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/010__Cluster_HA_Owner_Failover.robot
/w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/020__Cluster_HA_Owner_Restart.robot
/w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/030__Cluster_HA_Data_Recovery_Leader_Follower_Failover.robot
/w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustered_Reconciliation/010_Group_Flows.robot
/w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/010_Switch_Disconnect.robot
/w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/020_Cluster_Node_Failure.robot
/w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/030_Cluster_Sync_Problems.robot
/w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Bug_Validation/9145.robot
+ '[' -z '' ']'
++ tr '\012' ' '
++ grep -E -v '(^[[:space:]]*#|^[[:space:]]*$)' testplan.txt
+ suite_list='/w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/010__Cluster_HA_Owner_Failover.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/020__Cluster_HA_Owner_Restart.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/030__Cluster_HA_Data_Recovery_Leader_Follower_Failover.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustered_Reconciliation/010_Group_Flows.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/010_Switch_Disconnect.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/020_Cluster_Node_Failure.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/030_Cluster_Sync_Problems.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Bug_Validation/9145.robot '
+ eval 'SUITES='\''/w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/010__Cluster_HA_Owner_Failover.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/020__Cluster_HA_Owner_Restart.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/030__Cluster_HA_Data_Recovery_Leader_Follower_Failover.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustered_Reconciliation/010_Group_Flows.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/010_Switch_Disconnect.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/020_Cluster_Node_Failure.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/030_Cluster_Sync_Problems.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Bug_Validation/9145.robot '\'''
++ SUITES='/w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/010__Cluster_HA_Owner_Failover.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/020__Cluster_HA_Owner_Restart.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/030__Cluster_HA_Data_Recovery_Leader_Follower_Failover.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustered_Reconciliation/010_Group_Flows.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/010_Switch_Disconnect.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/020_Cluster_Node_Failure.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/030_Cluster_Sync_Problems.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Bug_Validation/9145.robot '
+ echo 'Starting Robot test suites /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/010__Cluster_HA_Owner_Failover.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/020__Cluster_HA_Owner_Restart.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/030__Cluster_HA_Data_Recovery_Leader_Follower_Failover.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustered_Reconciliation/010_Group_Flows.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/010_Switch_Disconnect.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/020_Cluster_Node_Failure.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/030_Cluster_Sync_Problems.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Bug_Validation/9145.robot  ...'
Starting Robot test suites /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/010__Cluster_HA_Owner_Failover.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/020__Cluster_HA_Owner_Restart.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/030__Cluster_HA_Data_Recovery_Leader_Follower_Failover.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustered_Reconciliation/010_Group_Flows.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/010_Switch_Disconnect.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/020_Cluster_Node_Failure.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/030_Cluster_Sync_Problems.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Bug_Validation/9145.robot  ...
+ robot -N openflowplugin-clustering.txt --removekeywords wuks -e exclude -e skip_if_titanium -v BUNDLEFOLDER:karaf-0.22.3-SNAPSHOT -v BUNDLE_URL:https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.22.3-SNAPSHOT/karaf-0.22.3-20260529.050331-217.zip -v CONTROLLER: -v CONTROLLER1: -v CONTROLLER2: -v CONTROLLER_USER:jenkins -v JAVA_HOME:/usr/lib/jvm/java-21-openjdk-amd64 -v JDKVERSION:openjdk21 -v JENKINS_WORKSPACE:/w/workspace/openflowplugin-csit-3node-clustering-only-titanium -v MININET: -v MININET1: -v MININET2: -v MININET_USER:jenkins -v NEXUSURL_PREFIX:https://nexus.opendaylight.org -v NUM_ODL_SYSTEM:0 -v NUM_TOOLS_SYSTEM:1 -v ODL_STREAM:titanium -v ODL_SYSTEM_IP: -v ODL_SYSTEM_USER:jenkins -v TOOLS_SYSTEM_IP: -v TOOLS_SYSTEM_1_IP: -v TOOLS_SYSTEM_USER:jenkins -v USER_HOME:/home/jenkins -v IS_KARAF_APPL:True -v WORKSPACE:/tmp -v ODL_OF_PLUGIN:lithium /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/010__Cluster_HA_Owner_Failover.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/020__Cluster_HA_Owner_Restart.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustering/030__Cluster_HA_Data_Recovery_Leader_Follower_Failover.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Clustered_Reconciliation/010_Group_Flows.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/010_Switch_Disconnect.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/020_Cluster_Node_Failure.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/EntityOwnership/030_Cluster_Sync_Problems.robot /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/suites/openflowplugin/Bug_Validation/9145.robot
==============================================================================
openflowplugin-clustering.txt                                                 
==============================================================================
openflowplugin-clustering.txt.Cluster HA Owner Failover :: Test suite for C...
==============================================================================
Check Shards Status Before Fail :: Check Status for all shards in ... | FAIL |
No leader found.
------------------------------------------------------------------------------
Start Mininet Multiple Connections :: Start mininet tree,2 with co... | FAIL |
OSError: [Errno 16] Device or resource busy
------------------------------------------------------------------------------
Check Entity Owner Status And Find Owner and Successor Before Fail... | FAIL |
Keyword 'ClusterManagement.Verify_Owner_And_Successors_For_Device' failed after retrying for 30 seconds. The last error was: Could not parse owner and candidates for device openflow:1
------------------------------------------------------------------------------
Reconnect Extra Switches To Successors And Check OVS Connections :... | FAIL |
Variable '@{original_successor_list}' not found.
------------------------------------------------------------------------------
Check Network Operational Information Before Fail :: Check devices... | PASS |
------------------------------------------------------------------------------
Add Configuration In Owner and Verify Before Fail :: Add Flow in O... | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Modify Configuration In Owner and Verify Before Fail :: Modify Flo... | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Delete Configuration In Owner and Verify Before Fail :: Delete Flo... | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Add Configuration In Successor and Verify Before Fail :: Add Flow ... | FAIL |
Variable '${original_successor}' not found.
------------------------------------------------------------------------------
Modify Configuration In Successor and Verify Before Fail :: Modify... | FAIL |
Variable '${original_successor}' not found.
------------------------------------------------------------------------------
Delete Configuration In Successor and Verify Before Fail :: Delete... | FAIL |
Variable '${original_successor}' not found.
------------------------------------------------------------------------------
Send RPC Add to Owner and Verify Before Fail :: Add Flow in Owner ... | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Send RPC Delete to Owner and Verify Before Fail :: Delete Flow in ... | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Send RPC Add to Successor and Verify Before Fail :: Add Flow in Su... | FAIL |
Variable '${original_successor}' not found.
------------------------------------------------------------------------------
Send RPC Delete to Successor and Verify Before Fail :: Delete Flow... | FAIL |
Variable '${original_successor}' not found.
------------------------------------------------------------------------------
Modify Network And Verify Before Fail :: Take a link down and veri... | FAIL |
Variable '${mininet_conn_id}' not found.
------------------------------------------------------------------------------
Restore Network And Verify Before Fail :: Take the link up and ver... | FAIL |
Variable '${mininet_conn_id}' not found.
------------------------------------------------------------------------------
Kill Owner Instance :: Kill Owner Instance and verify it is dead      | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Check Shards Status After Fail :: Create original cluster list and... | FAIL |
Variable '${new_cluster_list}' not found.
------------------------------------------------------------------------------
Check Entity Owner Status And Find Owner and Successor After Fail ... | FAIL |
Variable '${original_successor}' not found.
------------------------------------------------------------------------------
Check Network Operational Information After Fail :: Check devices ... | FAIL |
Variable '${new_cluster_list}' not found.
------------------------------------------------------------------------------
Add Configuration In Owner and Verify After Fail :: Add Flow in Ow... | FAIL |
Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Modify Configuration In Owner and Verify After Fail :: Modify Flow... | FAIL |
Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Delete Configuration In Owner and Verify After Fail :: Delete Flow... | FAIL |
Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Add Configuration In Successor and Verify After Fail :: Add Flow i... | FAIL |
Variable '${new_successor}' not found.
------------------------------------------------------------------------------
Modify Configuration In Successor and Verify After Fail :: Modify ... | FAIL |
Variable '${new_successor}' not found.
------------------------------------------------------------------------------
Delete Configuration In Successor and Verify After Fail :: Delete ... | FAIL |
Variable '${new_successor}' not found.
------------------------------------------------------------------------------
Send RPC Add to Owner and Verify After Fail :: Add Flow in Owner a... | FAIL |
Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Send RPC Delete to Owner and Verify After Fail :: Delete Flow in O... | FAIL |
Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Send RPC Add to Successor and Verify After Fail :: Add Flow in Suc... | FAIL |
Variable '${new_successor}' not found.
------------------------------------------------------------------------------
Send RPC Delete to Successor and Verify After Fail :: Delete Flow ... | FAIL |
Variable '${new_successor}' not found.
------------------------------------------------------------------------------
Modify Network and Verify After Fail :: Take a link down and verif... | FAIL |
Variable '${new_cluster_list}' not found.
------------------------------------------------------------------------------
Restore Network and Verify After Fail :: Take the link up and veri... | FAIL |
Variable '${new_cluster_list}' not found.
------------------------------------------------------------------------------
Start Old Owner Instance :: Start old Owner Instance and verify it... | FAIL |
This test fails due to https://jira.opendaylight.org/browse/CONTROLLER-1849

Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Check Shards Status After Recover :: Create original cluster list ... | FAIL |
Keyword 'ClusterOpenFlow.Check OpenFlow Shards Status' failed after retrying for 1 minute 30 seconds. The last error was: No leader found.
------------------------------------------------------------------------------
Check Entity Owner Status After Recover :: Check Entity Owner Stat... | FAIL |
Keyword 'ClusterManagement.Verify_Owner_And_Successors_For_Device' failed after retrying for 30 seconds. The last error was: Could not parse owner and candidates for device openflow:1
------------------------------------------------------------------------------
Check Network Operational Information After Recover :: Check devic... | PASS |
------------------------------------------------------------------------------
Add Configuration In Owner and Verify After Recover :: Add Flow in... | FAIL |
Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Modify Configuration In Owner and Verify After Recover :: Modify F... | FAIL |
Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Delete Configuration In Owner and Verify After Recover :: Delete F... | FAIL |
Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Add Configuration In Old Owner and Verify After Recover :: Add Flo... | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Modify Configuration In Old Owner and Verify After Recover :: Modi... | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Delete Configuration In Old Owner and Verify After Recover :: Dele... | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Send RPC Add to Owner and Verify After Recover :: Add Flow in Owne... | FAIL |
Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Send RPC Delete to Owner and Verify After Recover :: Delete Flow i... | FAIL |
Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Send RPC Add to Old Owner and Verify After Recover :: Add Flow in ... | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Send RPC Delete to Old Owner and Verify After Recover :: Delete Fl... | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Modify Network and Verify After Recover :: Take a link down and ve... | FAIL |
Variable '${mininet_conn_id}' not found.
------------------------------------------------------------------------------
Restore Network and Verify After Recover :: Take the link up and v... | FAIL |
Variable '${mininet_conn_id}' not found.
------------------------------------------------------------------------------
Stop Mininet and Exit :: Stop mininet and exit connection.            | FAIL |
Variable '${mininet_conn_id}' not found.
------------------------------------------------------------------------------
Check No Network Operational Information :: Check device is not in... | PASS |
------------------------------------------------------------------------------
openflowplugin-clustering.txt.Cluster HA Owner Failover :: Test su... | FAIL |
51 tests, 3 passed, 48 failed
==============================================================================
openflowplugin-clustering.txt.Cluster HA Owner Restart :: Test suite for Cl...
==============================================================================
Check Shards Status Before Stop :: Check Status for all shards in ... | FAIL |
No leader found.
------------------------------------------------------------------------------
Start Mininet Multiple Connections :: Start mininet tree,2 with co... | FAIL |
OSError: [Errno 16] Device or resource busy
------------------------------------------------------------------------------
Check Entity Owner Status And Find Owner and Successor Before Stop... | FAIL |
Keyword 'ClusterManagement.Verify_Owner_And_Successors_For_Device' failed after retrying for 30 seconds. The last error was: Could not parse owner and candidates for device openflow:1
------------------------------------------------------------------------------
Reconnect Extra Switches To Successors And Check OVS Connections :... | FAIL |
Variable '@{original_successor_list}' not found.
------------------------------------------------------------------------------
Check Network Operational Information Before Stop :: Check devices... | PASS |
------------------------------------------------------------------------------
Add Configuration In Owner and Verify Before Stop :: Add Flow in O... | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Modify Configuration In Owner and Verify Before Stop :: Modify Flo... | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Delete Configuration In Owner and Verify Before Stop :: Delete Flo... | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Add Configuration In Successor and Verify Before Stop :: Add Flow ... | FAIL |
Variable '${original_successor}' not found.
------------------------------------------------------------------------------
Modify Configuration In Successor and Verify Before Stop :: Modify... | FAIL |
Variable '${original_successor}' not found.
------------------------------------------------------------------------------
Delete Configuration In Successor and Verify Before Stop :: Delete... | FAIL |
Variable '${original_successor}' not found.
------------------------------------------------------------------------------
Send RPC Add to Owner and Verify Before Stop :: Add Flow in Owner ... | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Send RPC Delete to Owner and Verify Before Stop :: Delete Flow in ... | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Send RPC Add to Successor and Verify Before Stop :: Add Flow in Su... | FAIL |
Variable '${original_successor}' not found.
------------------------------------------------------------------------------
Send RPC Delete to Successor and Verify Before Stop :: Delete Flow... | FAIL |
Variable '${original_successor}' not found.
------------------------------------------------------------------------------
Modify Network And Verify Before Stop :: Take a link down and veri... | FAIL |
Variable '${mininet_conn_id}' not found.
------------------------------------------------------------------------------
Restore Network And Verify Before Stop :: Take the link up and ver... | FAIL |
Variable '${mininet_conn_id}' not found.
------------------------------------------------------------------------------
Stop Owner Instance :: Stop Owner Instance and verify it is dead      | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Check Shards Status After Stop :: Create original cluster list and... | FAIL |
Variable '${new_cluster_list}' not found.
------------------------------------------------------------------------------
Check Entity Owner Status And Find Owner and Successor After Stop ... | FAIL |
Variable '${original_successor}' not found.
------------------------------------------------------------------------------
Check Network Operational Information After Stop :: Check devices ... | FAIL |
Variable '${new_cluster_list}' not found.
------------------------------------------------------------------------------
Add Configuration In Owner and Verify After Stop :: Add Flow in Ow... | FAIL |
Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Modify Configuration In Owner and Verify After Stop :: Modify Flow... | FAIL |
Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Delete Configuration In Owner and Verify After Stop :: Delete Flow... | FAIL |
Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Add Configuration In Successor and Verify After Stop :: Add Flow i... | FAIL |
Variable '${new_successor}' not found.
------------------------------------------------------------------------------
Modify Configuration In Successor and Verify After Stop :: Modify ... | FAIL |
Variable '${new_successor}' not found.
------------------------------------------------------------------------------
Delete Configuration In Successor and Verify After Stop :: Delete ... | FAIL |
Variable '${new_successor}' not found.
------------------------------------------------------------------------------
Send RPC Add to Owner and Verify After Stop :: Add Flow in Owner a... | FAIL |
Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Send RPC Delete to Owner and Verify After Stop :: Delete Flow in O... | FAIL |
Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Send RPC Add to Successor and Verify After Stop :: Add Flow in Suc... | FAIL |
Variable '${new_successor}' not found.
------------------------------------------------------------------------------
Send RPC Delete to Successor and Verify After Stop :: Delete Flow ... | FAIL |
Variable '${new_successor}' not found.
------------------------------------------------------------------------------
Modify Network and Verify After Stop :: Take a link down and verif... | FAIL |
Variable '${new_cluster_list}' not found.
------------------------------------------------------------------------------
Restore Network and Verify After Stop :: Take the link up and veri... | FAIL |
Variable '${new_cluster_list}' not found.
------------------------------------------------------------------------------
Start Old Owner Instance :: Start old Owner Instance and verify it... | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Check Shards Status After Start :: Create original cluster list an... | FAIL |
Keyword 'ClusterOpenFlow.Check OpenFlow Shards Status' failed after retrying for 1 minute 30 seconds. The last error was: No leader found.
------------------------------------------------------------------------------
Check Entity Owner Status After Start :: Check Entity Owner Status... | FAIL |
Keyword 'ClusterManagement.Verify_Owner_And_Successors_For_Device' failed after retrying for 30 seconds. The last error was: Could not parse owner and candidates for device openflow:1
------------------------------------------------------------------------------
Check Network Operational Information After Start :: Check devices... | PASS |
------------------------------------------------------------------------------
Add Configuration In Owner and Verify After Start :: Add Flow in O... | FAIL |
Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Modify Configuration In Owner and Verify After Start :: Modify Flo... | FAIL |
Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Delete Configuration In Owner and Verify After Start :: Delete Flo... | FAIL |
Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Add Configuration In Old Owner and Verify After Start :: Add Flow ... | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Modify Configuration In Old Owner and Verify After Start :: Modify... | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Delete Configuration In Old Owner and Verify After Start :: Delete... | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Send RPC Add to Owner and Verify After Start :: Add Flow in Owner ... | FAIL |
Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Send RPC Delete to Owner and Verify After Start :: Delete Flow in ... | FAIL |
Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Send RPC Add to Old Owner and Verify After Start :: Add Flow in Ow... | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Send RPC Delete to Old Owner and Verify After Start :: Delete Flow... | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Modify Network and Verify After Start :: Take a link down and veri... | FAIL |
Variable '${mininet_conn_id}' not found.
------------------------------------------------------------------------------
Restore Network and Verify After Start :: Take the link up and ver... | FAIL |
Variable '${mininet_conn_id}' not found.
------------------------------------------------------------------------------
Stop Mininet and Exit :: Stop mininet and exit connection.            | FAIL |
Variable '${mininet_conn_id}' not found.
------------------------------------------------------------------------------
Check No Network Operational Information :: Check device is not in... | PASS |
------------------------------------------------------------------------------
openflowplugin-clustering.txt.Cluster HA Owner Restart :: Test sui... | FAIL |
51 tests, 3 passed, 48 failed
==============================================================================
openflowplugin-clustering.txt.Cluster HA Data Recovery Leader Follower Fail...
==============================================================================
Check Shards Status Before Leader Restart :: Check Status for all ... | FAIL |
No leader found.
------------------------------------------------------------------------------
Get inventory Leader Before Leader Restart :: Find leader in the i... | FAIL |
Keyword 'ClusterManagement.Get_Leader_And_Followers_For_Shard' failed after retrying for 10 seconds. The last error was: No leader found.
------------------------------------------------------------------------------
Start Mininet Connect To Follower Node1 :: Start mininet with conn... | FAIL |
Variable '${follower_node_1}' not found.
------------------------------------------------------------------------------
Add Flows In Follower Node2 and Verify Before Leader Restart :: Ad... | FAIL |
Variable '${follower_node_2}' not found.
------------------------------------------------------------------------------
Stop Mininet Connected To Follower Node1 and Exit :: Stop mininet ... | FAIL |
Variable '${mininet_conn_id}' not found.
------------------------------------------------------------------------------
Restart Leader From Cluster Node :: Stop Leader Node and Start it ... | FAIL |
Variable '${inventory_leader}' not found.
------------------------------------------------------------------------------
Get inventory Follower After Leader Restart :: Find new Followers ... | FAIL |
Keyword 'ClusterManagement.Get_Leader_And_Followers_For_Shard' failed after retrying for 10 seconds. The last error was: No leader found.
------------------------------------------------------------------------------
Start Mininet Connect To Old Leader :: Start mininet with connecti... | FAIL |
Variable '${inventory_leader_old}' not found.
------------------------------------------------------------------------------
Verify Flows In Switch After Leader Restart :: Verify flows are in... | PASS |
------------------------------------------------------------------------------
Stop Mininet Connected To Old Leader and Exit :: Stop mininet and ... | FAIL |
Variable '${mininet_conn_id}' not found.
------------------------------------------------------------------------------
Restart Follower Node2 :: Stop Follower Node2 and Start it Up, Ver... | FAIL |
Variable '${follower_node_2}' not found.
------------------------------------------------------------------------------
Get inventory Follower After Follower Restart :: Find Followers an... | FAIL |
Keyword 'ClusterManagement.Get_Leader_And_Followers_For_Shard' failed after retrying for 10 seconds. The last error was: No leader found.
------------------------------------------------------------------------------
Start Mininet Connect To Leader :: Start mininet with connection t... | FAIL |
Variable '${inventory_leader}' not found.
------------------------------------------------------------------------------
Verify Flows In Switch After Follower Restart :: Verify flows are ... | PASS |
------------------------------------------------------------------------------
Stop Mininet Connected To Leader and Exit :: Stop mininet Connecte... | FAIL |
Variable '${mininet_conn_id}' not found.
------------------------------------------------------------------------------
Restart Full Cluster :: Stop all Cluster Nodes and Start it Up All.   | PASS |
------------------------------------------------------------------------------
Get inventory Status After Cluster Restart :: Find New Followers a... | FAIL |
Keyword 'ClusterManagement.Get_Leader_And_Followers_For_Shard' failed after retrying for 10 seconds. The last error was: No leader found.
------------------------------------------------------------------------------
Start Mininet Connect To Follower Node2 After Cluster Restart :: S... | FAIL |
Variable '${follower_node_2}' not found.
------------------------------------------------------------------------------
Verify Flows In Switch After Cluster Restart :: Verify flows are i... | PASS |
------------------------------------------------------------------------------
Delete Flows In Follower Node1 and Verify After Leader Restart :: ... | FAIL |
Variable '${follower_node_1}' not found.
------------------------------------------------------------------------------
Stop Mininet Connected To Follower Node2 and Exit After Cluster Re... | FAIL |
Variable '${mininet_conn_id}' not found.
------------------------------------------------------------------------------
openflowplugin-clustering.txt.Cluster HA Data Recovery Leader Foll... | FAIL |
21 tests, 4 passed, 17 failed
==============================================================================
/w/workspace/openflowplugin-csit-3node-clustering-only-titanium/test/csit/libraries/VsctlListParser.py:61: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  if ctl_ref is not "":
openflowplugin-clustering.txt.010 Group Flows :: Switch connections and clu...
==============================================================================
Add Groups And Flows :: Add 100 groups 1&2 and flows in every switch. | FAIL |
Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Start Mininet Multiple Connections :: Start mininet linear with co... | FAIL |
OSError: [Errno 16] Device or resource busy
------------------------------------------------------------------------------
Check Linear Topology :: Check Linear Topology.                       | FAIL |
Keyword 'ClusterOpenFlow.Check Linear Topology On Member' failed after retrying for 30 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Stats Are Not Frozen :: Check that duration flow stat is inc... | FAIL |
Keyword 'Check Flow Stats Are Not Frozen' failed after retrying for 30 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Flows In Operational DS :: Check Flows in operational DS.       | FAIL |
Keyword 'ClusterOpenFlow.Check Number Of Flows On Member' failed after retrying for 10 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Groups In Operational DS :: Check Groups in operational DS.     | FAIL |
Keyword 'ClusterOpenFlow.Check Number Of Groups On Member' failed after retrying for 10 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Flows In Switch :: Check Flows in switch.                       | FAIL |
Variable '${mininet_conn_id}' not found.
------------------------------------------------------------------------------
Check Entity Owner Status And Find Owner and Successor Before Fail... | FAIL |
Keyword 'ClusterManagement.Verify_Owner_And_Successors_For_Device' failed after retrying for 30 seconds. The last error was: Could not parse owner and candidates for device openflow:1
------------------------------------------------------------------------------
Disconnect Mininet From Owner :: Disconnect mininet from the owner    | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Check Entity Owner Status And Find Owner and Successor After Fail ... | FAIL |
Keyword 'ClusterOpenFlow.Get OpenFlow Entity Owner Status For One Device' failed after retrying for 10 seconds. The last error was: Variable '${new_cluster_list}' not found.
------------------------------------------------------------------------------
Check Switch Moves To New Master :: Check switch s1 is connected t... | FAIL |
Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Check Linear Topology After Disconnect :: Check Linear Topology.      | FAIL |
Keyword 'ClusterOpenFlow.Check Linear Topology On Member' failed after retrying for 30 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Stats Are Not Frozen After Disconnect :: Check that duration... | FAIL |
Keyword 'Check Flow Stats Are Not Frozen' failed after retrying for 30 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Remove Flows And Groups After Mininet Is Disconnected :: Remove 1 ... | FAIL |
Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Flows In Operational DS After Mininet Is Disconnected :: Che... | FAIL |
Keyword 'ClusterOpenFlow.Check Number Of Flows On Member' failed after retrying for 30 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Groups In Operational DS After Mininet Is Disconnected :: Ch... | FAIL |
Keyword 'ClusterOpenFlow.Check Number Of Groups On Member' failed after retrying for 10 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Flows In Switch After Mininet Is Disconnected :: Check Flows... | FAIL |
Variable '${mininet_conn_id}' not found.
------------------------------------------------------------------------------
Reconnect Mininet To Owner :: Reconnect mininet to switch 1 owner.    | FAIL |
Variable '${original_owner_list}' not found.
------------------------------------------------------------------------------
Check Entity Owner Status And Find Owner and Successor After Recon... | FAIL |
Keyword 'ClusterOpenFlow.Get OpenFlow Entity Owner Status For One Device' failed after retrying for 10 seconds. The last error was: Keyword 'ClusterManagement.Verify_Owner_And_Successors_For_Device' failed after retrying for 30 seconds. The last error was: Could not parse owner and candidates for device openflow:1
------------------------------------------------------------------------------
Add Flows And Groups After Owner Reconnect :: Add 1 group type 1&2... | FAIL |
Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Stats Are Not Frozen After Owner Reconnect :: Check that dur... | FAIL |
Keyword 'Check Flow Stats Are Not Frozen' failed after retrying for 30 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Flows After Owner Reconnect In Operational DS :: Check Flows... | FAIL |
Keyword 'ClusterOpenFlow.Check Number Of Flows On Member' failed after retrying for 30 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Groups After Owner Reconnect In Operational DS :: Check Grou... | FAIL |
Keyword 'ClusterOpenFlow.Check Number Of Groups On Member' failed after retrying for 10 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Flows After Owner Reconnect In Switch :: Check Flows in switch. | FAIL |
Variable '${mininet_conn_id}' not found.
------------------------------------------------------------------------------
Check Switches Generate Slave Connection :: Check switches are con... | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Disconnect Mininet From Successor :: Disconnect mininet from the S... | FAIL |
Variable '${new_successor_list}' not found.
------------------------------------------------------------------------------
Check Entity Owner Status And Find New Owner and Successor After D... | FAIL |
Keyword 'ClusterOpenFlow.Get OpenFlow Entity Owner Status For One Device' failed after retrying for 10 seconds. The last error was: Variable '${owner_list}' not found.
------------------------------------------------------------------------------
Disconnect Mininet From Current Owner :: Disconnect mininet from t... | FAIL |
Variable '${current_owner}' not found.
------------------------------------------------------------------------------
Check Entity Owner Status And Find Current Owner and Successor Aft... | FAIL |
Keyword 'ClusterOpenFlow.Get OpenFlow Entity Owner Status For One Device' failed after retrying for 10 seconds. The last error was: Variable '${original_owner_list}' not found.
------------------------------------------------------------------------------
Check Switch Moves To Current Master :: Check switch s1 is connect... | FAIL |
Variable '${current_new_owner}' not found.
------------------------------------------------------------------------------
Check Linear Topology After Owner Disconnect :: Check Linear Topol... | FAIL |
Keyword 'ClusterOpenFlow.Check Linear Topology On Member' failed after retrying for 30 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Stats Are Not Frozen After Owner Disconnect :: Check that du... | FAIL |
Keyword 'Check Flow Stats Are Not Frozen' failed after retrying for 30 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Remove Flows And Groups After Owner Disconnected :: Remove 1 group... | FAIL |
Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Flows In Operational DS After Owner Disconnected :: Check Fl... | FAIL |
Keyword 'ClusterOpenFlow.Check Number Of Flows On Member' failed after retrying for 30 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Groups In Operational DS After Owner Disconnected :: Check G... | FAIL |
Keyword 'ClusterOpenFlow.Check Number Of Groups On Member' failed after retrying for 10 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Flows In Switch After Owner Disconnected :: Check Flows in s... | FAIL |
Variable '${mininet_conn_id}' not found.
------------------------------------------------------------------------------
Disconnect Mininet From Cluster :: Disconnect Mininet from Cluster.   | FAIL |
Variable '${original_owner_list}' not found.
------------------------------------------------------------------------------
Check No Switches After Disconnect :: Check no switches in topology.  | FAIL |
Keyword 'ClusterOpenFlow.Check No Switches On Member' failed after retrying for 30 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Switch Is Not Connected :: Check switch s1 is not connected ... | FAIL |
Variable '@{cluster_index_list}' not found.
------------------------------------------------------------------------------
Reconnect Mininet To Cluster :: Reconnect mininet to cluster by re... | PASS |
------------------------------------------------------------------------------
Check Linear Topology After Mininet Reconnects :: Check Linear Top... | FAIL |
Keyword 'ClusterOpenFlow.Check Linear Topology On Member' failed after retrying for 10 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Add Flows And Groups After Mininet Reconnects :: Add 1 group type ... | FAIL |
Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Flows In Operational DS After Mininet Reconnects :: Check Fl... | FAIL |
Keyword 'ClusterOpenFlow.Check Number Of Flows On Member' failed after retrying for 30 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Groups In Operational DS After Mininet Reconnects :: Check G... | FAIL |
Keyword 'ClusterOpenFlow.Check Number Of Groups On Member' failed after retrying for 10 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Flows In Switch After Mininet Reconnects :: Check Flows in s... | FAIL |
Variable '${mininet_conn_id}' not found.
------------------------------------------------------------------------------
Check Entity Owner Status And Find Owner and Successor Before Owne... | FAIL |
Keyword 'ClusterManagement.Verify_Owner_And_Successors_For_Device' failed after retrying for 30 seconds. The last error was: Could not parse owner and candidates for device openflow:1
------------------------------------------------------------------------------
Check Switch Generates Slave Connection Before Owner Stop :: Check... | FAIL |
Variable '${original_successor}' not found.
------------------------------------------------------------------------------
Check Shards Status Before Owner Stop :: Check Status for all shar... | FAIL |
No leader found.
------------------------------------------------------------------------------
Stop Owner Instance :: Stop Owner Instance and verify it is shutdown  | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Check Shards Status After Stop :: Check Status for all shards in O... | FAIL |
Variable '${new_cluster_list}' not found.
------------------------------------------------------------------------------
Check Entity Owner Status And Find Owner and Successor After Stop ... | FAIL |
Variable '${original_successor}' not found.
------------------------------------------------------------------------------
Check Stats Are Not Frozen After Owner Stop :: Check that duration... | FAIL |
Keyword 'Check Flow Stats Are Not Frozen' failed after retrying for 30 seconds. The last error was: Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Remove Configuration In Owner and Verify After Owner Stop :: Remov... | FAIL |
Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Check Flows After Owner Stop In Operational DS :: Check Flows in O... | FAIL |
Keyword 'ClusterOpenFlow.Check Number Of Flows On Member' failed after retrying for 30 seconds. The last error was: Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Check Groups After Owner Stop In Operational DS :: Check Groups in... | FAIL |
Keyword 'ClusterOpenFlow.Check Number Of Groups On Member' failed after retrying for 10 seconds. The last error was: Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Check Flows In Switch After Owner Stop :: Check Flows in switch.      | FAIL |
Variable '${mininet_conn_id}' not found.
------------------------------------------------------------------------------
Start Old Owner Instance :: Start old Owner Instance and verify it... | FAIL |
Variable '${original_owner}' not found.
------------------------------------------------------------------------------
Check Entity Owner Status And Find Owner and Successor After Start... | FAIL |
Keyword 'ClusterOpenFlow.Get OpenFlow Entity Owner Status For One Device' failed after retrying for 10 seconds. The last error was: Keyword 'ClusterManagement.Verify_Owner_And_Successors_For_Device' failed after retrying for 30 seconds. The last error was: Could not parse owner and candidates for device openflow:1
------------------------------------------------------------------------------
Check Linear Topology After Owner Restart :: Check Linear Topology.   | FAIL |
Keyword 'ClusterOpenFlow.Check Linear Topology On Member' failed after retrying for 10 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Add Configuration In Owner and Verify After Owner Restart :: Add 1... | FAIL |
Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Check Stats Are Not Frozen After Owner Restart :: Check that durat... | FAIL |
Variable '${new_owner}' not found.
------------------------------------------------------------------------------
Check Flows In Operational DS After Owner Restart :: Check Flows i... | FAIL |
Keyword 'ClusterOpenFlow.Check Number Of Flows On Member' failed after retrying for 30 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Groups In Operational DS After Owner Restart :: Check Groups... | FAIL |
Keyword 'ClusterOpenFlow.Check Number Of Groups On Member' failed after retrying for 10 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Flows In Switch After Owner Restart :: Check Flows in switch.   | FAIL |
Variable '${mininet_conn_id}' not found.
------------------------------------------------------------------------------
Restart Cluster :: Stop and Start cluster.                            | PASS |
------------------------------------------------------------------------------
Check Linear Topology After Controller Restarts :: Check Linear To... | FAIL |
Keyword 'ClusterOpenFlow.Check Linear Topology On Member' failed after retrying for 5 minutes. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Stats Are Not Frozen After Cluster Restart :: Check that dur... | FAIL |
Keyword 'Check Flow Stats Are Not Frozen' failed after retrying for 30 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Flows In Operational DS After Controller Restarts :: Check F... | FAIL |
Keyword 'ClusterOpenFlow.Check Number Of Flows On Member' failed after retrying for 30 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Groups In Operational DS After Controller Restarts :: Check ... | FAIL |
Keyword 'ClusterOpenFlow.Check Number Of Groups On Member' failed after retrying for 10 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
Check Flows In Switch After Controller Restarts :: Check Flows in ... | FAIL |
Variable '${mininet_conn_id}' not found.
------------------------------------------------------------------------------
Stop Mininet :: Stop Mininet.                                         | FAIL |
No open connection.
------------------------------------------------------------------------------
Check No Switches :: Check no switches in topology.                   | FAIL |
Keyword 'ClusterOpenFlow.Check No Switches On Member' failed after retrying for 5 seconds. The last error was: Non-existing index or alias 'ClusterManagement__session_1'.
------------------------------------------------------------------------------
openflowplugin-clustering.txt.010 Group Flows :: Switch connection... | FAIL |
72 tests, 2 passed, 70 failed
==============================================================================
openflowplugin-clustering.txt.010 Switch Disconnect :: Test suite for entit...
==============================================================================
Switches To Be Connected To All Nodes :: Initial check for correct... | FAIL |
Parent suite setup failed:
OSError: [Errno 16] Device or resource busy
------------------------------------------------------------------------------
Reconnecting Switch s1                                                | FAIL |
Parent suite setup failed:
OSError: [Errno 16] Device or resource busy
------------------------------------------------------------------------------
Switches Still Be Connected To All Nodes                              | FAIL |
Parent suite setup failed:
OSError: [Errno 16] Device or resource busy
------------------------------------------------------------------------------
openflowplugin-clustering.txt.010 Switch Disconnect :: Test suite ... | FAIL |
Suite setup failed:
OSError: [Errno 16] Device or resource busy

Also suite teardown failed:
Several failures occurred:

1) Cannot open session, you need to establish a connection first.

2) Cannot open session, you need to establish a connection first.

3) Cannot open session, you need to establish a connection first.

3 tests, 0 passed, 3 failed
==============================================================================
openflowplugin-clustering.txt.020 Cluster Node Failure :: Test suite for en...
==============================================================================
Switches To Be Connected To All Nodes :: Initial check for correct... | FAIL |
Parent suite setup failed:
OSError: [Errno 16] Device or resource busy
------------------------------------------------------------------------------
Restarting Owner Of Switch s1                                         | FAIL |
Parent suite setup failed:
OSError: [Errno 16] Device or resource busy
------------------------------------------------------------------------------
Switches Still Be Connected To All Nodes                              | FAIL |
Parent suite setup failed:
OSError: [Errno 16] Device or resource busy
------------------------------------------------------------------------------
openflowplugin-clustering.txt.020 Cluster Node Failure :: Test sui... | FAIL |
Suite setup failed:
OSError: [Errno 16] Device or resource busy

Also suite teardown failed:
Several failures occurred:

1) Cannot open session, you need to establish a connection first.

2) Cannot open session, you need to establish a connection first.

3) Cannot open session, you need to establish a connection first.

3 tests, 0 passed, 3 failed
==============================================================================
openflowplugin-clustering.txt.030 Cluster Sync Problems :: Test suite for e...
==============================================================================
Start Mininet To All Nodes                                            | FAIL |
OSError: [Errno 16] Device or resource busy
------------------------------------------------------------------------------
Switches To Be Connected To All Nodes :: Initial check for correct... | FAIL |
Keyword 'Check All Switches Connected To All Cluster Nodes' failed after retrying 15 times. The last error was: Cannot open session, you need to establish a connection first.
------------------------------------------------------------------------------
Isolating Owner Of Switch s1                                          | FAIL |
This test fails due to https://bugs.opendaylight.org/show_bug.cgi?id=6177

Variable '${active_member}' not found.
------------------------------------------------------------------------------
Switches Still Be Connected To All Nodes                              | FAIL |
This test fails due to https://bugs.opendaylight.org/show_bug.cgi?id=6177

Keyword 'Check All Switches Connected To All Cluster Nodes' failed after retrying 15 times. The last error was: Cannot open session, you need to establish a connection first.
------------------------------------------------------------------------------
Stop Mininet And Verify No Owners                                     | FAIL |
This test fails due to https://bugs.opendaylight.org/show_bug.cgi?id=6177

Cannot open session, you need to establish a connection first.
------------------------------------------------------------------------------
openflowplugin-clustering.txt.030 Cluster Sync Problems :: Test su... | FAIL |
5 tests, 0 passed, 5 failed
==============================================================================
openflowplugin-clustering.txt.9145 :: Switch connections and cluster are re...
==============================================================================
Start Mininet Multiple Connections :: Start mininet linear with co... <string>:1: SyntaxWarning: "is not" with a literal. Did you mean "!="?
| FAIL |
OSError: [Errno 16] Device or resource busy
------------------------------------------------------------------------------
Check Entity Owner Status And Find Owner and Successor :: Check En... | FAIL |
This test fails due to https://bugs.opendaylight.org/show_bug.cgi?id=9145

Keyword 'ClusterManagement.Verify_Owner_And_Successors_For_Device' failed after retrying for 30 seconds. The last error was: Could not parse owner and candidates for device openflow:1
------------------------------------------------------------------------------
Stop Mininet :: Stop Mininet.                                         | FAIL |
Variable '${mininet_conn_id}' not found.
------------------------------------------------------------------------------
openflowplugin-clustering.txt.9145 :: Switch connections and clust... | FAIL |
3 tests, 0 passed, 3 failed
==============================================================================
openflowplugin-clustering.txt                                         | FAIL |
209 tests, 12 passed, 197 failed
==============================================================================
Output:  /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/output.xml
Log:     /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/log.html
Report:  /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/report.html
+ true
+ echo 'Examining the files in data/log and checking filesize'
Examining the files in data/log and checking filesize
+ ssh '' 'ls -altr /tmp/karaf-0.22.3-SNAPSHOT/data/log/'
ssh: Could not resolve hostname : Device or resource busy
Build step 'Execute shell' marked build as failure
$ ssh-agent -k
unset SSH_AUTH_SOCK;
unset SSH_AGENT_PID;
echo Agent pid 3166 killed;
[ssh-agent] Stopped.
Recording plot data
Robot results publisher started...
INFO: Checking test criticality is deprecated and will be dropped in a future release!
-Parsing output xml:
Done!
-Copying log files to build dir:
Done!
-Assigning results to build:
Done!
-Checking thresholds:
Done!
Done publishing Robot results.
[PostBuildScript] - [INFO] Executing post build scripts.
[openflowplugin-csit-3node-clustering-only-titanium] $ /bin/bash /tmp/jenkins1812670550718342987.sh
Archiving csit artifacts
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 1014k    0 1014k    0     0  1555k      0 --:--:-- --:--:-- --:--:-- 1555k
Archive:  robot-plugin.zip
  inflating: ./archives/robot-plugin/log.html  
  inflating: ./archives/robot-plugin/output.xml  
  inflating: ./archives/robot-plugin/report.html  
mv: cannot stat '*.log': No such file or directory
mv: cannot stat '*.log.gz': No such file or directory
mv: cannot stat '*.csv': No such file or directory
mv: cannot stat '*.png': No such file or directory
[PostBuildScript] - [INFO] Executing post build scripts.
[openflowplugin-csit-3node-clustering-only-titanium] $ /bin/bash /tmp/jenkins12756216850382189253.sh
[PostBuildScript] - [INFO] Executing post build scripts.
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties content 
OS_CLOUD=vex
OS_STACK_NAME=releng-openflowplugin-csit-3node-clustering-only-titanium-733

[EnvInject] - Variables injected successfully.
provisioning config files...
copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
[openflowplugin-csit-3node-clustering-only-titanium] $ /bin/bash /tmp/jenkins9116502617863897921.sh
---> openstack-stack-delete.sh
Setup pyenv:
  system
  3.8.20
  3.9.20
  3.10.15
* 3.11.10 (set by /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-QOlV from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv)
lf-activate-venv(): INFO: Attempting to install with network-safe options...
lf-activate-venv(): INFO: Base packages installed successfully
lf-activate-venv(): INFO: Installing additional packages: lftools[openstack] kubernetes python-heatclient python-openstackclient urllib3~=1.26.15 
lf-activate-venv(): INFO: Adding /tmp/venv-QOlV/bin to PATH
INFO: Stack cost retrieval disabled, setting cost to 0
INFO: Deleting stack releng-openflowplugin-csit-3node-clustering-only-titanium-733
Successfully deleted stack releng-openflowplugin-csit-3node-clustering-only-titanium-733
[PostBuildScript] - [INFO] Executing post build scripts.
[openflowplugin-csit-3node-clustering-only-titanium] $ /bin/bash /tmp/jenkins17287191158307511097.sh
---> sysstat.sh
/tmp/jenkins17287191158307511097.sh: line 19: facter: command not found
[openflowplugin-csit-3node-clustering-only-titanium] $ /bin/bash /tmp/jenkins13461033110776633735.sh
---> package-listing.sh
++ facter osfamily
/tmp/jenkins13461033110776633735.sh: line 19: facter: command not found
++ tr '[:upper:]' '[:lower:]'
+ OS_FAMILY=
[openflowplugin-csit-3node-clustering-only-titanium] $ /bin/bash /tmp/jenkins8012821070786731983.sh
---> capture-instance-metadata.sh
Setup pyenv:
  system
  3.8.20
  3.9.20
  3.10.15
* 3.11.10 (set by /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-QOlV from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv)
lf-activate-venv(): INFO: Attempting to install with network-safe options...
lf-activate-venv(): INFO: Base packages installed successfully
lf-activate-venv(): INFO: Installing additional packages: lftools 
lf-activate-venv(): INFO: Adding /tmp/venv-QOlV/bin to PATH
INFO: Running in OpenStack, capturing instance metadata
[openflowplugin-csit-3node-clustering-only-titanium] $ /bin/bash /tmp/jenkins9737271760304058655.sh
provisioning config files...
Could not find credentials [logs] for openflowplugin-csit-3node-clustering-only-titanium #733
copy managed file [jenkins-log-archives-settings] to file:/w/workspace/openflowplugin-csit-3node-clustering-only-titanium@tmp/config2006940071018549950tmp
Regular expression run condition: Expression=[^.*logs-s3.*], Label=[odl-logs-s3-cloudfront-index]
Run condition [Regular expression match] enabling perform for step [Provide Configuration files]
provisioning config files...
copy managed file [jenkins-s3-log-ship] to file:/home/jenkins/.aws/credentials
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties content 
SERVER_ID=logs

[EnvInject] - Variables injected successfully.
[openflowplugin-csit-3node-clustering-only-titanium] $ /bin/bash /tmp/jenkins1191745611794888482.sh
---> create-netrc.sh
WARN: Log server credential not found.
[openflowplugin-csit-3node-clustering-only-titanium] $ /bin/bash /tmp/jenkins14981709672173261057.sh
---> python-tools-install.sh
Setup pyenv:
  system
  3.8.20
  3.9.20
  3.10.15
* 3.11.10 (set by /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-QOlV from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv)
lf-activate-venv(): INFO: Attempting to install with network-safe options...
lf-activate-venv(): INFO: Base packages installed successfully
lf-activate-venv(): INFO: Installing additional packages: lftools 
lf-activate-venv(): INFO: Adding /tmp/venv-QOlV/bin to PATH
[openflowplugin-csit-3node-clustering-only-titanium] $ /bin/sh /tmp/jenkins9117752167254786907.sh
---> uv-install.sh
uv 0.11.17 is already installed
uvx 0.11.17 (x86_64-unknown-linux-gnu)
[openflowplugin-csit-3node-clustering-only-titanium] $ /bin/bash /tmp/jenkins8556443041111531045.sh
---> sudo-logs.sh
Archiving 'sudo' log..
/tmp/jenkins8556443041111531045.sh: line 41: facter: command not found
[openflowplugin-csit-3node-clustering-only-titanium] $ /bin/bash /tmp/jenkins18384262172777230867.sh
---> job-cost.sh
INFO: Activating Python virtual environment...
Setup pyenv:
  system
  3.8.20
  3.9.20
  3.10.15
* 3.11.10 (set by /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-QOlV from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv)
lf-activate-venv(): INFO: Attempting to install with network-safe options...
lf-activate-venv(): INFO: Base packages installed successfully
lf-activate-venv(): INFO: Installing additional packages: zipp==1.1.0 python-openstackclient urllib3~=1.26.15 
lf-activate-venv(): INFO: Adding /tmp/venv-QOlV/bin to PATH
DEBUG: Contents of stack-cost file:
total: 0
INFO: Retrieving Stack Cost...
INFO: Stack cost: 0
INFO: Instance uptime: 2030s
INFO: Fetching instance metadata (attempt 1 of 3)...
DEBUG: URL: http://169.254.169.254/latest/meta-data/instance-type
INFO: Successfully fetched instance metadata
INFO: Instance type: v3-standard-2
INFO: Retrieving pricing info for: v3-standard-2
INFO: Fetching Vexxhost pricing API (attempt 1 of 3)...
DEBUG: URL: https://pricing.vexxhost.net/v1/pricing/v3-standard-2/cost?seconds=2030
INFO: Successfully fetched Vexxhost pricing API
INFO: Retrieved cost: 0.06
INFO: Retrieved resource: v3-standard-2
INFO: Creating archive directory: /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/archives/cost
INFO: Archiving costs to: /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/archives/cost.csv
INFO: Successfully archived job cost data
DEBUG: Cost data: openflowplugin-csit-3node-clustering-only-titanium,733,2026-05-29 07:31:56,v3-standard-2,2030,0.06,0.00,FAILURE
[openflowplugin-csit-3node-clustering-only-titanium] $ /bin/bash -l /tmp/jenkins2357077180074757345.sh
---> logs-deploy.sh
Setup pyenv:
  system
  3.8.20
  3.9.20
  3.10.15
* 3.11.10 (set by /w/workspace/openflowplugin-csit-3node-clustering-only-titanium/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-QOlV from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv)
lf-activate-venv(): INFO: Attempting to install with network-safe options...
lf-activate-venv(): INFO: Base packages installed successfully
lf-activate-venv(): INFO: Installing additional packages: lftools urllib3~=1.26.15 
lf-activate-venv(): INFO: Adding /tmp/venv-QOlV/bin to PATH
WARNING: Nexus logging server not set
INFO: S3 path logs/releng/vex-yul-odl-jenkins-1/openflowplugin-csit-3node-clustering-only-titanium/733/
INFO: archiving logs to S3
---> uname -a:
Linux prd-ubuntu2404-robot-2c-8g-21105 6.8.0-90-generic #91-Ubuntu SMP PREEMPT_DYNAMIC Tue Nov 18 14:14:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux


---> lscpu:
Architecture:                         x86_64
CPU op-mode(s):                       32-bit, 64-bit
Address sizes:                        40 bits physical, 48 bits virtual
Byte Order:                           Little Endian
CPU(s):                               2
On-line CPU(s) list:                  0,1
Vendor ID:                            AuthenticAMD
Model name:                           AMD EPYC-Rome Processor
CPU family:                           23
Model:                                49
Thread(s) per core:                   1
Core(s) per socket:                   1
Socket(s):                            2
Stepping:                             0
BogoMIPS:                             5599.99
Flags:                                fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 clzero xsaveerptr wbnoinvd arat npt nrip_save umip rdpid arch_capabilities
Virtualization:                       AMD-V
Hypervisor vendor:                    KVM
Virtualization type:                  full
L1d cache:                            64 KiB (2 instances)
L1i cache:                            64 KiB (2 instances)
L2 cache:                             1 MiB (2 instances)
L3 cache:                             32 MiB (2 instances)
NUMA node(s):                         1
NUMA node0 CPU(s):                    0,1
Vulnerability Gather data sampling:   Not affected
Vulnerability Itlb multihit:          Not affected
Vulnerability L1tf:                   Not affected
Vulnerability Mds:                    Not affected
Vulnerability Meltdown:               Not affected
Vulnerability Mmio stale data:        Not affected
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed:               Mitigation; untrained return thunk; SMT disabled
Vulnerability Spec rstack overflow:   Vulnerable: Safe RET, no microcode
Vulnerability Spec store bypass:      Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:             Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2:             Mitigation; Retpolines; IBPB conditional; STIBP disabled; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
Vulnerability Srbds:                  Not affected
Vulnerability Tsx async abort:        Not affected
Vulnerability Vmscape:                Not affected


---> nproc:
2


---> df -h:
Filesystem      Size  Used Avail Use% Mounted on
tmpfs           795M  1.1M  794M   1% /run
/dev/vda1        38G  8.1G   30G  22% /
tmpfs           3.9G     0  3.9G   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
/dev/vda16      881M  117M  703M  15% /boot
/dev/vda15      105M  6.2M   99M   6% /boot/efi
tmpfs           795M   16K  795M   1% /run/user/1001


---> free -m:
               total        used        free      shared  buff/cache   available
Mem:            7941         804        4958           4        2479        7137
Swap:           1023           0        1023


---> ip addr:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: ens3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1458 qdisc mq state UP group default qlen 1000
    link/ether fa:16:3e:19:be:aa brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    inet 10.30.171.210/23 metric 100 brd 10.30.171.255 scope global dynamic ens3
       valid_lft 84368sec preferred_lft 84368sec
    inet6 fe80::f816:3eff:fe19:beaa/64 scope link 
       valid_lft forever preferred_lft forever


---> sar -b -r -n DEV:
Linux 6.8.0-90-generic (prd-ubuntu2404-robot-2c-8g-21105) 	05/29/26 	_x86_64_	(2 CPU)

06:58:19     LINUX RESTART	(2 CPU)

07:00:00          tps      rtps      wtps      dtps   bread/s   bwrtn/s   bdscd/s
07:10:07        16.80      1.77     13.25      1.77    128.78   2630.36    590.38
07:20:08         1.85      0.26      1.58      0.01     31.00     94.50      0.53
07:30:01         1.59      0.01      1.57      0.02      1.06     31.39      0.28
Average:         6.80      0.69      5.51      0.61     54.06    927.71    199.09

07:00:00    kbmemfree   kbavail kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit  kbactive   kbinact   kbdirty
07:10:07      5233632   7401272    366840      4.51     86176   2242804    610348      6.65    588784   2014524       452
07:20:08      5153052   7352784    414772      5.10     86476   2274464    814976      8.88    603480   2037752       268
07:30:01      5153548   7357476    410224      5.04     86628   2278452    611176      6.66    598640   2041604       156
Average:      5180077   7370511    397279      4.89     86427   2265240    678833      7.39    596968   2031293       292

07:00:00        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
07:10:07           lo      1.24      1.24      0.07      0.07      0.00      0.00      0.00      0.00
07:10:07         ens3     15.76     10.33    139.90      2.39      0.00      0.00      0.00      0.00
07:20:08           lo      1.02      1.02      0.05      0.05      0.00      0.00      0.00      0.00
07:20:08         ens3      1.95      1.11      3.49      0.31      0.00      0.00      0.00      0.00
07:30:01           lo      1.00      1.00      0.05      0.05      0.00      0.00      0.00      0.00
07:30:01         ens3      1.51      0.59      3.19      0.27      0.00      0.00      0.00      0.00
Average:           lo      1.09      1.09      0.06      0.06      0.00      0.00      0.00      0.00
Average:         ens3      6.46      4.04     49.33      1.00      0.00      0.00      0.00      0.00


---> sar -P ALL:
Linux 6.8.0-90-generic (prd-ubuntu2404-robot-2c-8g-21105) 	05/29/26 	_x86_64_	(2 CPU)

06:58:19     LINUX RESTART	(2 CPU)

07:00:00        CPU     %user     %nice   %system   %iowait    %steal     %idle
07:10:07        all      8.77      0.00      1.13      0.51      0.05     89.54
07:10:07          0      6.89      0.00      0.98      0.54      0.05     91.55
07:10:07          1     10.66      0.00      1.28      0.48      0.05     87.53
07:20:08        all      1.77      0.00      0.20      0.03      0.06     97.94
07:20:08          0      2.11      0.00      0.17      0.02      0.07     97.63
07:20:08          1      1.44      0.00      0.22      0.05      0.05     98.25
07:30:01        all      1.15      0.00      0.24      0.01      0.06     98.54
07:30:01          0      1.00      0.00      0.21      0.01      0.04     98.74
07:30:01          1      1.31      0.00      0.28      0.01      0.07     98.33
Average:        all      3.93      0.00      0.53      0.19      0.05     95.31
Average:          0      3.35      0.00      0.46      0.19      0.05     95.95
Average:          1      4.50      0.00      0.60      0.18      0.06     94.66