00:18:29 Started by upstream project "integration-distribution-mri-test-titanium" build number 96 00:18:29 originally caused by: 00:18:29 Started by timer 00:18:29 Running as SYSTEM 00:18:29 [EnvInject] - Loading node environment variables. 00:18:29 Building remotely on prd-ubuntu2404-robot-2c-8g-38843 (ubuntu2404-robot-2c-8g) in workspace /w/workspace/controller-csit-3node-clustering-tell-all-titanium 00:18:37 [ssh-agent] Looking for ssh-agent implementation... 00:18:37 [ssh-agent] Exec ssh-agent (binary ssh-agent on a remote machine) 00:18:37 $ ssh-agent 00:18:37 SSH_AUTH_SOCK=/tmp/ssh-UvYtns7Gvrzs/agent.3166 00:18:37 SSH_AGENT_PID=3168 00:18:37 [ssh-agent] Started. 00:18:37 Running ssh-add (command line suppressed) 00:18:37 Identity added: /w/workspace/controller-csit-3node-clustering-tell-all-titanium@tmp/private_key_15839803154023398953.key (/w/workspace/controller-csit-3node-clustering-tell-all-titanium@tmp/private_key_15839803154023398953.key) 00:18:37 [ssh-agent] Using credentials jenkins (Release Engineering Jenkins Key) 00:18:37 The recommended git tool is: NONE 00:18:44 using credential opendaylight-jenkins-ssh 00:18:44 Wiping out workspace first. 00:18:44 Cloning the remote Git repository 00:18:44 Cloning repository git://devvexx.opendaylight.org/mirror/integration/test 00:18:44 > git init /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test # timeout=10 00:18:44 Fetching upstream changes from git://devvexx.opendaylight.org/mirror/integration/test 00:18:44 > git --version # timeout=10 00:18:44 > git --version # 'git version 2.43.0' 00:18:44 using GIT_SSH to set credentials Release Engineering Jenkins Key 00:18:44 Verifying host key using known hosts file, will automatically accept unseen keys 00:18:44 > git fetch --tags --force --progress -- git://devvexx.opendaylight.org/mirror/integration/test +refs/heads/*:refs/remotes/origin/* # timeout=10 00:18:50 > git config remote.origin.url git://devvexx.opendaylight.org/mirror/integration/test # timeout=10 00:18:50 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 00:18:50 > git config remote.origin.url git://devvexx.opendaylight.org/mirror/integration/test # timeout=10 00:18:50 Fetching upstream changes from git://devvexx.opendaylight.org/mirror/integration/test 00:18:50 using GIT_SSH to set credentials Release Engineering Jenkins Key 00:18:50 Verifying host key using known hosts file, will automatically accept unseen keys 00:18:50 > git fetch --tags --force --progress -- git://devvexx.opendaylight.org/mirror/integration/test master # timeout=10 00:18:50 > git rev-parse FETCH_HEAD^{commit} # timeout=10 00:18:50 Checking out Revision 901c7e139945b436d95a44b3b592904c3d7a4f9f (origin/master) 00:18:50 > git config core.sparsecheckout # timeout=10 00:18:50 > git checkout -f 901c7e139945b436d95a44b3b592904c3d7a4f9f # timeout=10 00:18:51 Commit message: "Adapt updater.py to use new lsp-flags container" 00:18:51 > git rev-parse FETCH_HEAD^{commit} # timeout=10 00:18:51 > git rev-list --no-walk f65244a2a093016c71f9a7a564be7d086941e15c # timeout=10 00:18:51 No emails were triggered. 00:18:51 provisioning config files... 00:18:51 copy managed file [npmrc] to file:/home/jenkins/.npmrc 00:18:51 copy managed file [pipconf] to file:/home/jenkins/.config/pip/pip.conf 00:18:51 copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml 00:18:51 [controller-csit-3node-clustering-tell-all-titanium] $ /bin/bash /tmp/jenkins1961695356055222814.sh 00:18:51 ---> python-tools-install.sh 00:18:51 Setup pyenv: 00:18:51 * system (set by /opt/pyenv/version) 00:18:51 * 3.8.20 (set by /opt/pyenv/version) 00:18:51 * 3.9.20 (set by /opt/pyenv/version) 00:18:51 3.10.15 00:18:51 3.11.10 00:18:57 lf-activate-venv(): INFO: Creating python3 venv at /tmp/venv-1T4P 00:18:57 lf-activate-venv(): INFO: Save venv in file: /tmp/.os_lf_venv 00:18:57 lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv) 00:18:57 lf-activate-venv(): INFO: Attempting to install with network-safe options... 00:19:02 lf-activate-venv(): INFO: Base packages installed successfully 00:19:02 lf-activate-venv(): INFO: Installing additional packages: lftools 00:19:31 lf-activate-venv(): INFO: Adding /tmp/venv-1T4P/bin to PATH 00:19:31 Generating Requirements File 00:19:59 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 00:19:59 httplib2 0.30.2 requires pyparsing<4,>=3.0.4, but you have pyparsing 2.4.7 which is incompatible. 00:19:59 Python 3.11.10 00:19:59 pip 26.1.2 from /tmp/venv-1T4P/lib/python3.11/site-packages/pip (python 3.11) 00:20:00 aiohappyeyeballs==2.7.1 00:20:00 aiohttp==3.14.3 00:20:00 aiosignal==1.4.0 00:20:00 appdirs==1.4.4 00:20:00 argcomplete==3.7.0 00:20:00 aspy.yaml==1.3.0 00:20:00 attrs==26.1.0 00:20:00 autopage==0.6.0 00:20:00 beautifulsoup4==4.15.0 00:20:00 boto3==1.43.56 00:20:00 botocore==1.43.56 00:20:00 bs4==0.0.2 00:20:00 certifi==2026.7.22 00:20:00 cffi==2.1.0 00:20:00 cfgv==3.5.0 00:20:00 chardet==7.4.3 00:20:00 charset-normalizer==3.4.9 00:20:00 click==8.4.2 00:20:00 cliff==4.15.0 00:20:00 cmd2==4.1.2 00:20:00 cryptography==3.3.2 00:20:00 debtcollector==3.1.0 00:20:00 decorator==5.3.1 00:20:00 defusedxml==0.7.1 00:20:00 Deprecated==1.3.1 00:20:00 distlib==0.4.3 00:20:00 dnspython==2.8.0 00:20:00 docker==7.2.0 00:20:00 dogpile.cache==1.5.0 00:20:00 durationpy==0.10 00:20:00 email-validator==2.3.0 00:20:00 filelock==3.32.0 00:20:00 frozenlist==1.8.0 00:20:00 future==1.0.0 00:20:00 gitdb==4.0.12 00:20:00 GitPython==3.1.56 00:20:00 httplib2==0.30.2 00:20:00 identify==2.6.19 00:20:00 idna==3.18 00:20:00 importlib-resources==1.5.0 00:20:00 iso8601==2.1.0 00:20:00 Jinja2==3.1.6 00:20:00 jmespath==1.1.0 00:20:00 jsonpatch==1.33 00:20:00 jsonpointer==3.1.1 00:20:00 jsonschema==4.26.0 00:20:00 jsonschema-specifications==2025.9.1 00:20:00 keystoneauth1==5.15.0 00:20:00 kubernetes==36.0.3 00:20:00 lftools==0.37.22 00:20:00 lxml==6.1.1 00:20:00 markdown-it-py==4.2.0 00:20:00 MarkupSafe==3.0.3 00:20:00 mdurl==0.1.2 00:20:00 msgpack==1.2.1 00:20:00 multi_key_dict==2.0.3 00:20:00 multidict==6.7.1 00:20:00 munch==4.0.0 00:20:00 netaddr==1.3.0 00:20:00 niet==1.4.2 00:20:00 nodeenv==1.10.0 00:20:00 oauth2client==4.1.3 00:20:00 oauthlib==3.3.1 00:20:00 openstacksdk==4.17.0 00:20:00 os-service-types==1.8.2 00:20:00 osc-lib==4.7.0 00:20:00 oslo.config==10.6.0 00:20:00 oslo.context==6.5.0 00:20:00 oslo.i18n==6.9.0 00:20:00 oslo.log==8.3.0 00:20:00 oslo.serialization==5.11.0 00:20:00 oslo.utils==10.1.1 00:20:00 packaging==26.2 00:20:00 pbr==7.0.3 00:20:00 platformdirs==4.11.0 00:20:00 prettytable==3.18.0 00:20:00 prompt_toolkit==3.0.52 00:20:00 propcache==0.5.2 00:20:00 psutil==7.2.2 00:20:00 pyasn1==0.6.4 00:20:00 pyasn1_modules==0.4.2 00:20:00 pycparser==3.0 00:20:00 pygerrit2==2.0.15 00:20:00 PyGithub==2.9.1 00:20:00 Pygments==2.20.0 00:20:00 PyJWT==2.13.0 00:20:00 PyNaCl==1.6.2 00:20:00 pyparsing==2.4.7 00:20:00 pyperclip==1.11.0 00:20:00 pyrsistent==0.20.0 00:20:00 python-dateutil==2.9.0.post0 00:20:00 python-discovery==1.5.0 00:20:00 python-heatclient==5.2.0 00:20:00 python-jenkins==1.8.3 00:20:00 python-keystoneclient==5.8.0 00:20:00 python-magnumclient==4.11.0 00:20:00 python-manilaclient==6.2.0 00:20:00 python-openstackclient==10.2.1 00:20:00 python-swiftclient==4.10.0 00:20:00 PyYAML==6.0.3 00:20:00 referencing==0.37.0 00:20:00 requests==2.34.2 00:20:00 requests-oauthlib==2.0.0 00:20:00 rfc3986==2.0.0 00:20:00 rich==15.0.0 00:20:00 rich-argparse==1.8.0 00:20:00 rpds-py==2026.6.3 00:20:00 rsa==4.9.1 00:20:00 ruamel.yaml==0.19.1 00:20:00 ruamel.yaml.clib==0.2.15 00:20:00 s3transfer==0.19.2 00:20:00 simplejson==4.1.1 00:20:00 six==1.17.0 00:20:00 smmap==5.0.3 00:20:00 soupsieve==2.9.1 00:20:00 stevedore==5.9.0 00:20:00 tabulate==0.10.0 00:20:00 toml==0.10.2 00:20:00 tomlkit==0.15.1 00:20:00 tqdm==4.69.1 00:20:00 typing_extensions==4.16.0 00:20:00 urllib3==1.26.20 00:20:00 virtualenv==21.7.0 00:20:00 wcwidth==0.8.2 00:20:00 websocket-client==1.9.0 00:20:00 wrapt==2.2.2 00:20:00 xdg==6.0.0 00:20:00 xmltodict==1.0.4 00:20:00 yarl==1.24.5 00:20:00 yq==4.1.2 00:20:00 [controller-csit-3node-clustering-tell-all-titanium] $ /bin/sh /tmp/jenkins17945601891139951556.sh 00:20:00 ---> uv-install.sh 00:20:00 Installing uv/uvx (latest) using shell installer 00:20:00 2026-07-26 00:20:00 URL:https://releases.astral.sh/installers/uv/latest/uv-installer.sh [71233/71233] -> "/tmp/uv-install-SNKrwF.sh" [1] 00:20:00 downloading uv 0.11.32 x86_64-unknown-linux-gnu 00:20:01 installing to /home/jenkins/.local/bin 00:20:01 uv 00:20:01 uvx 00:20:01 everything's installed! 00:20:01 00:20:01 To add $HOME/.local/bin to your PATH, either restart your shell or run: 00:20:01 00:20:01 source $HOME/.local/bin/env (sh, bash, zsh) 00:20:01 source $HOME/.local/bin/env.fish (fish) 00:20:01 Adding install location to PATH 00:20:01 ---> Validating uv/uvx install 00:20:01 uvx 0.11.32 (x86_64-unknown-linux-gnu) 00:20:01 [EnvInject] - Injecting environment variables from a build step. 00:20:01 [EnvInject] - Injecting as environment variables the properties content 00:20:01 OS_STACK_TEMPLATE=csit-2-instance-type.yaml 00:20:01 OS_CLOUD=vex 00:20:01 OS_STACK_NAME=releng-controller-csit-3node-clustering-tell-all-titanium-96 00:20:01 OS_STACK_TEMPLATE_DIR=openstack-hot 00:20:01 00:20:01 [EnvInject] - Variables injected successfully. 00:20:01 provisioning config files... 00:20:01 copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml 00:20:01 [controller-csit-3node-clustering-tell-all-titanium] $ /bin/bash /tmp/jenkins929722414190576958.sh 00:20:01 ---> Create parameters file for OpenStack HOT 00:20:01 OpenStack Heat parameters generated 00:20:01 ----------------------------------- 00:20:01 parameters: 00:20:01 vm_0_count: '3' 00:20:01 vm_0_flavor: 'v3-standard-4' 00:20:01 vm_0_image: 'ZZCI - Ubuntu 22.04 - builder - x86_64 - 20260401-010115.623' 00:20:01 vm_1_count: '0' 00:20:01 vm_1_flavor: 'v3-standard-2' 00:20:01 vm_1_image: 'ZZCI - Ubuntu 22.04 - mininet-ovs-217 - x86_64 - 20250917-133034.654' 00:20:01 00:20:01 job_name: '14749-96' 00:20:01 silo: 'releng' 00:20:01 [controller-csit-3node-clustering-tell-all-titanium] $ /bin/bash -l /tmp/jenkins10615872936082438647.sh 00:20:01 ---> Create HEAT stack 00:20:01 + source /home/jenkins/lf-env.sh 00:20:01 + lf-activate-venv --python python3 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient urllib3~=1.26.15 yq 00:20:01 ++ mktemp -d /tmp/venv-XXXX 00:20:01 + lf_venv=/tmp/venv-wKOV 00:20:01 + local venv_file=/tmp/.os_lf_venv 00:20:01 + local python=python3 00:20:01 + local options 00:20:01 + local set_path=true 00:20:01 + local install_args= 00:20:01 ++ getopt -o np:v: -l no-path,system-site-packages,python:,venv-file: -n lf-activate-venv -- --python python3 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient urllib3~=1.26.15 yq 00:20:01 + options=' --python '\''python3'\'' -- '\''lftools[openstack]'\'' '\''kubernetes'\'' '\''niet'\'' '\''python-heatclient'\'' '\''python-openstackclient'\'' '\''python-magnumclient'\'' '\''urllib3~=1.26.15'\'' '\''yq'\''' 00:20:01 + eval set -- ' --python '\''python3'\'' -- '\''lftools[openstack]'\'' '\''kubernetes'\'' '\''niet'\'' '\''python-heatclient'\'' '\''python-openstackclient'\'' '\''python-magnumclient'\'' '\''urllib3~=1.26.15'\'' '\''yq'\''' 00:20:01 ++ set -- --python python3 -- 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient urllib3~=1.26.15 yq 00:20:01 + true 00:20:01 + case $1 in 00:20:01 + python=python3 00:20:01 + shift 2 00:20:01 + true 00:20:01 + case $1 in 00:20:01 + shift 00:20:01 + break 00:20:01 + case $python in 00:20:01 + local pkg_list= 00:20:01 + [[ -d /opt/pyenv ]] 00:20:01 + echo 'Setup pyenv:' 00:20:01 Setup pyenv: 00:20:01 + export PYENV_ROOT=/opt/pyenv 00:20:01 + PYENV_ROOT=/opt/pyenv 00:20:01 + export PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin 00:20:01 + PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin 00:20:01 + pyenv versions 00:20:01 system 00:20:01 3.8.20 00:20:01 3.9.20 00:20:01 3.10.15 00:20:01 * 3.11.10 (set by /w/workspace/controller-csit-3node-clustering-tell-all-titanium/.python-version) 00:20:01 + command -v pyenv 00:20:01 ++ pyenv init - --no-rehash 00:20:01 + eval 'PATH="$(bash --norc -ec '\''IFS=:; paths=($PATH); 00:20:01 for i in ${!paths[@]}; do 00:20:01 if [[ ${paths[i]} == "'\'''\''/opt/pyenv/shims'\'''\''" ]]; then unset '\''\'\'''\''paths[i]'\''\'\'''\''; 00:20:01 fi; done; 00:20:01 echo "${paths[*]}"'\'')" 00:20:01 export PATH="/opt/pyenv/shims:${PATH}" 00:20:01 export PYENV_SHELL=bash 00:20:01 source '\''/opt/pyenv/libexec/../completions/pyenv.bash'\'' 00:20:01 pyenv() { 00:20:01 local command 00:20:01 command="${1:-}" 00:20:01 if [ "$#" -gt 0 ]; then 00:20:01 shift 00:20:01 fi 00:20:01 00:20:01 case "$command" in 00:20:01 rehash|shell) 00:20:01 eval "$(pyenv "sh-$command" "$@")" 00:20:01 ;; 00:20:01 *) 00:20:01 command pyenv "$command" "$@" 00:20:01 ;; 00:20:01 esac 00:20:01 }' 00:20:01 +++ bash --norc -ec 'IFS=:; paths=($PATH); 00:20:01 for i in ${!paths[@]}; do 00:20:01 if [[ ${paths[i]} == "/opt/pyenv/shims" ]]; then unset '\''paths[i]'\''; 00:20:01 fi; done; 00:20:01 echo "${paths[*]}"' 00:20:01 ++ PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin 00:20:01 ++ export PATH=/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin 00:20:01 ++ PATH=/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin 00:20:01 ++ export PYENV_SHELL=bash 00:20:01 ++ PYENV_SHELL=bash 00:20:01 ++ source /opt/pyenv/libexec/../completions/pyenv.bash 00:20:01 +++ complete -F _pyenv pyenv 00:20:01 ++ lf-pyver python3 00:20:01 ++ local py_version_xy=python3 00:20:01 ++ local py_version_xyz= 00:20:01 ++ pyenv versions 00:20:01 ++ sed 's/^[ *]* //' 00:20:01 ++ grep -E '^[0-9.]*[0-9]$' 00:20:01 ++ local command 00:20:01 ++ command=versions 00:20:01 ++ '[' 1 -gt 0 ']' 00:20:01 ++ shift 00:20:01 ++ case "$command" in 00:20:01 ++ command pyenv versions 00:20:01 ++ awk '{ print $1 }' 00:20:02 ++ [[ ! -s /tmp/.pyenv_versions ]] 00:20:02 +++ grep '^3' /tmp/.pyenv_versions 00:20:02 +++ sort -V 00:20:02 +++ tail -n 1 00:20:02 ++ py_version_xyz=3.11.10 00:20:02 ++ [[ -z 3.11.10 ]] 00:20:02 ++ echo 3.11.10 00:20:02 ++ return 0 00:20:02 + pyenv local 3.11.10 00:20:02 + local command 00:20:02 + command=local 00:20:02 + '[' 2 -gt 0 ']' 00:20:02 + shift 00:20:02 + case "$command" in 00:20:02 + command pyenv local 3.11.10 00:20:02 + for arg in "$@" 00:20:02 + case $arg in 00:20:02 + pkg_list+='lftools[openstack] ' 00:20:02 + for arg in "$@" 00:20:02 + case $arg in 00:20:02 + pkg_list+='kubernetes ' 00:20:02 + for arg in "$@" 00:20:02 + case $arg in 00:20:02 + pkg_list+='niet ' 00:20:02 + for arg in "$@" 00:20:02 + case $arg in 00:20:02 + pkg_list+='python-heatclient ' 00:20:02 + for arg in "$@" 00:20:02 + case $arg in 00:20:02 + pkg_list+='python-openstackclient ' 00:20:02 + for arg in "$@" 00:20:02 + case $arg in 00:20:02 + pkg_list+='python-magnumclient ' 00:20:02 + for arg in "$@" 00:20:02 + case $arg in 00:20:02 + pkg_list+='urllib3~=1.26.15 ' 00:20:02 + for arg in "$@" 00:20:02 + case $arg in 00:20:02 + pkg_list+='yq ' 00:20:02 + [[ -f /tmp/.os_lf_venv ]] 00:20:02 ++ cat /tmp/.os_lf_venv 00:20:02 + lf_venv=/tmp/venv-1T4P 00:20:02 + echo 'lf-activate-venv(): INFO: Reuse venv:/tmp/venv-1T4P from' file:/tmp/.os_lf_venv 00:20:02 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-1T4P from file:/tmp/.os_lf_venv 00:20:02 + echo 'lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv)' 00:20:02 lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv) 00:20:02 + local 'pip_opts=--upgrade --quiet' 00:20:02 + pip_opts='--upgrade --quiet --trusted-host pypi.org' 00:20:02 + pip_opts='--upgrade --quiet --trusted-host pypi.org --trusted-host files.pythonhosted.org' 00:20:02 + pip_opts='--upgrade --quiet --trusted-host pypi.org --trusted-host files.pythonhosted.org --trusted-host pypi.python.org' 00:20:02 + [[ -n '' ]] 00:20:02 + [[ -n '' ]] 00:20:02 + echo 'lf-activate-venv(): INFO: Attempting to install with network-safe options...' 00:20:02 lf-activate-venv(): INFO: Attempting to install with network-safe options... 00:20:02 + /tmp/venv-1T4P/bin/python3 -m pip install --upgrade --quiet --trusted-host pypi.org --trusted-host files.pythonhosted.org --trusted-host pypi.python.org pip 'setuptools<66' virtualenv 00:20:04 + echo 'lf-activate-venv(): INFO: Base packages installed successfully' 00:20:04 lf-activate-venv(): INFO: Base packages installed successfully 00:20:04 + [[ -z lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient urllib3~=1.26.15 yq ]] 00:20:04 + echo 'lf-activate-venv(): INFO: Installing additional packages: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient urllib3~=1.26.15 yq ' 00:20:04 lf-activate-venv(): INFO: Installing additional packages: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient urllib3~=1.26.15 yq 00:20:04 + /tmp/venv-1T4P/bin/python3 -m pip install --upgrade --quiet --trusted-host pypi.org --trusted-host files.pythonhosted.org --trusted-host pypi.python.org --upgrade-strategy eager 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient urllib3~=1.26.15 yq 00:20:25 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 00:20:25 python-manilaclient 6.2.0 requires osc-lib>=3.2.0, but you have osc-lib 2.2.1 which is incompatible. 00:20:25 + type python3 00:20:25 + true 00:20:25 + echo 'lf-activate-venv(): INFO: Adding /tmp/venv-1T4P/bin to PATH' 00:20:25 lf-activate-venv(): INFO: Adding /tmp/venv-1T4P/bin to PATH 00:20:25 + PATH=/tmp/venv-1T4P/bin:/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin 00:20:25 + return 0 00:20:25 + openstack --os-cloud vex limits show --absolute 00:20:27 +--------------------------+---------+ 00:20:27 | Name | Value | 00:20:27 +--------------------------+---------+ 00:20:27 | maxTotalInstances | -1 | 00:20:27 | maxTotalCores | 450 | 00:20:27 | maxTotalRAMSize | 1000000 | 00:20:27 | maxServerMeta | 128 | 00:20:27 | maxImageMeta | 128 | 00:20:27 | maxPersonality | 5 | 00:20:27 | maxPersonalitySize | 10240 | 00:20:27 | maxTotalKeypairs | 100 | 00:20:27 | maxServerGroups | 10 | 00:20:27 | maxServerGroupMembers | 10 | 00:20:27 | maxTotalFloatingIps | -1 | 00:20:27 | maxSecurityGroups | -1 | 00:20:27 | maxSecurityGroupRules | -1 | 00:20:27 | totalRAMUsed | 992256 | 00:20:27 | totalCoresUsed | 243 | 00:20:27 | totalInstancesUsed | 87 | 00:20:27 | totalFloatingIpsUsed | 0 | 00:20:27 | totalSecurityGroupsUsed | 0 | 00:20:27 | totalServerGroupsUsed | 0 | 00:20:27 | maxTotalVolumes | -1 | 00:20:27 | maxTotalSnapshots | 10 | 00:20:27 | maxTotalVolumeGigabytes | 4096 | 00:20:27 | maxTotalBackups | 10 | 00:20:27 | maxTotalBackupGigabytes | 1000 | 00:20:27 | totalVolumesUsed | 1 | 00:20:27 | totalGigabytesUsed | 20 | 00:20:27 | totalSnapshotsUsed | 0 | 00:20:27 | totalBackupsUsed | 0 | 00:20:27 | totalBackupGigabytesUsed | 0 | 00:20:27 +--------------------------+---------+ 00:20:28 + pushd /opt/ciman/openstack-hot 00:20:28 /opt/ciman/openstack-hot /w/workspace/controller-csit-3node-clustering-tell-all-titanium 00:20:28 + lftools openstack --os-cloud vex stack create releng-controller-csit-3node-clustering-tell-all-titanium-96 csit-2-instance-type.yaml /w/workspace/controller-csit-3node-clustering-tell-all-titanium/stack-parameters.yaml 00:27:03 Creating stack releng-controller-csit-3node-clustering-tell-all-titanium-96 00:27:03 Waiting to initialize infrastructure... 00:27:03 WARN: Failed to initialize stack. Reason: Resource CREATE failed: Forbidden: resources.vm_0_group.resources[2].resources.instance: Quota exceeded for ram: Requested 16384, but already used 992256 of 1000000 ram (HTTP 403) (Request-ID: req-4c63a5e2-05ac-4f0a-b820-6f212de14a29) 00:27:03 Deleting stack cd396959-4d70-4040-bc7f-c02e41b96c6f 00:27:03 Waiting for stack to delete... 00:27:03 Waiting for stack to delete... 00:27:03 Successfully deleted stack cd396959-4d70-4040-bc7f-c02e41b96c6f 00:27:03 Waiting to initialize infrastructure... 00:27:03 WARN: Failed to initialize stack. Reason: Resource CREATE failed: StackValidationFailed: resources.vm_0_group.resources[2].resources.instance: Property error: instance.Properties.flavor: Unexpected API Error. Please report this at http: //bugs.launchpad.net/nova/ and attach the Nova API log if possible. 00:27:03 (HTTP 500) (Request-ID: req-6b2d5def-9aa8-4952-8dcd-c8ef7fe8b707) 00:27:03 Deleting stack b8528a94-61f1-451d-936e-be651e8fb9c1 00:27:03 Waiting for stack to delete... 00:27:03 Waiting for stack to delete... 00:27:03 Waiting for stack to delete... 00:27:03 Waiting for stack to delete... 00:27:03 WARN: Failed to delete $STACK_NAME. Reason: Resource DELETE failed: ClientException: resources.vm_0_group.resources[0].resources.instance: Unexpected API Error. Please report this at http: //bugs.launchpad.net/nova/ and attach the Nova API log if possible. 00:27:03 (HTTP 500) (Request-ID: req-bed57dcf-6221-4c89-8e6d-b1873b99ad6d) 00:27:03 Retrying delete... 00:27:03 Waiting for stack to delete... 00:27:03 Waiting for stack to delete... 00:27:03 Waiting for stack to delete... 00:27:03 WARN: Failed to delete $STACK_NAME. Reason: Resource DELETE failed: ClientException: resources.vm_0_group.resources[0].resources.instance: Unexpected API Error. Please report this at http: //bugs.launchpad.net/nova/ and attach the Nova API log if possible. 00:27:03 (HTTP 500) (Request-ID: req-ff802942-e8ca-49c4-b3f7-782e3880837f) 00:27:03 Retrying delete... 00:27:03 Waiting for stack to delete... 00:27:03 Waiting for stack to delete... 00:27:03 Waiting for stack to delete... 00:27:03 Waiting for stack to delete... 00:27:03 Waiting for stack to delete... 00:27:03 Waiting for stack to delete... 00:27:03 Waiting for stack to delete... 00:27:03 Waiting for stack to delete... 00:27:03 Waiting for stack to delete... 00:27:03 Waiting for stack to delete... 00:27:03 Waiting for stack to delete... 00:27:03 Waiting for stack to delete... 00:27:03 Waiting for stack to delete... 00:27:03 Waiting for stack to delete... 00:27:03 Waiting for stack to delete... 00:27:03 Waiting for stack to delete... 00:27:03 Waiting for stack to delete... 00:27:03 WARN: Failed to delete $STACK_NAME. Reason: Resource DELETE failed: ResourceInError: resources.vm_0_group.resources[0].resources.instance: Went to status ERROR due to "Server releng-14749-96-0-builder-0 delete failed: (500) Build of instance 196a8d09-796a-4b78-90f3-91e97509a009 aborted: Failed to allocate the network(s), not rescheduling." 00:27:03 Retrying delete... 00:27:03 Successfully deleted stack b8528a94-61f1-451d-936e-be651e8fb9c1 00:27:03 ------------------------------------ 00:27:03 Stack Details 00:27:03 ------------------------------------ 00:27:03 {'added': None, 00:27:03 'capabilities': [], 00:27:03 'created_at': '2026-07-26T00:21:28Z', 00:27:03 'deleted': None, 00:27:03 'deleted_at': None, 00:27:03 'description': 'No description', 00:27:03 'environment': None, 00:27:03 'environment_files': None, 00:27:03 'files': None, 00:27:03 'files_container': None, 00:27:03 'id': 'b8528a94-61f1-451d-936e-be651e8fb9c1', 00:27:03 'is_rollback_disabled': True, 00:27:03 'links': [{'href': 'https://orchestration.public.mtl1.vexxhost.net/v1/12c36e260d8e4bb2913965203b1b491f/stacks/releng-controller-csit-3node-clustering-tell-all-titanium-96/b8528a94-61f1-451d-936e-be651e8fb9c1', 00:27:03 'rel': 'self'}], 00:27:03 'location': Munch({'cloud': 'vex', 'region_name': 'ca-ymq-1', 'zone': None, 'project': Munch({'id': '12c36e260d8e4bb2913965203b1b491f', 'name': '61975f2c-7c17-4d69-82fa-c3ae420ad6fd', 'domain_id': None, 'domain_name': 'Default'})}), 00:27:03 'name': 'releng-controller-csit-3node-clustering-tell-all-titanium-96', 00:27:03 'notification_topics': [], 00:27:03 'outputs': [{'description': 'IP addresses of the 2nd vm types', 00:27:03 'output_key': 'vm_1_ips', 00:27:03 'output_value': []}, 00:27:03 {'description': 'IP addresses of the 1st vm types', 00:27:03 'output_key': 'vm_0_ips', 00:27:03 'output_value': [None, None, None]}], 00:27:03 'owner_id': ****, 00:27:03 'parameters': {'OS::project_id': '12c36e260d8e4bb2913965203b1b491f', 00:27:03 'OS::stack_id': 'b8528a94-61f1-451d-936e-be651e8fb9c1', 00:27:03 'OS::stack_name': 'releng-controller-csit-3node-clustering-tell-all-titanium-96', 00:27:03 'job_name': '14749-96', 00:27:03 'silo': 'releng', 00:27:03 'vm_0_count': '3', 00:27:03 'vm_0_flavor': 'v3-standard-4', 00:27:03 'vm_0_image': 'ZZCI - Ubuntu 22.04 - builder - x86_64 - ' 00:27:03 '20260401-010115.623', 00:27:03 'vm_1_count': '0', 00:27:03 'vm_1_flavor': 'v3-standard-2', 00:27:03 'vm_1_image': 'ZZCI - Ubuntu 22.04 - mininet-ovs-217 - x86_64 ' 00:27:03 '- 20250917-133034.654'}, 00:27:03 'parent_id': None, 00:27:03 'replaced': None, 00:27:03 'status': 'CREATE_FAILED', 00:27:03 'status_reason': 'Resource CREATE failed: StackValidationFailed: ' 00:27:03 'resources.vm_0_group.resources[2].resources.instance: ' 00:27:03 'Property error: instance.Properties.flavor: Unexpected API ' 00:27:03 'Error. Please report this at http: ' 00:27:03 '//bugs.launchpad.net/nova/ and attach the Nova API log if ' 00:27:03 'possible.\n' 00:27:03 " (HTTP 500) " 00:27:03 '(Request-ID: req-6b2d5def-9aa8-4952-8dcd-c8ef7fe8b707)', 00:27:03 'tags': [], 00:27:03 'template': None, 00:27:03 'template_description': 'No description', 00:27:03 'template_url': None, 00:27:03 'timeout_mins': 15, 00:27:03 'unchanged': None, 00:27:03 'updated': None, 00:27:03 'updated_at': None, 00:27:03 'user_project_id': '3339c018b96646cea697091550b47e39'} 00:27:03 ------------------------------------ 00:27:04 + popd 00:27:04 /w/workspace/controller-csit-3node-clustering-tell-all-titanium 00:27:04 [controller-csit-3node-clustering-tell-all-titanium] $ /bin/bash -l /tmp/jenkins6166712530681731316.sh 00:27:04 ---> Copy SSH public keys to CSIT lab 00:27:04 Setup pyenv: 00:27:04 system 00:27:04 3.8.20 00:27:04 3.9.20 00:27:04 3.10.15 00:27:04 * 3.11.10 (set by /w/workspace/controller-csit-3node-clustering-tell-all-titanium/.python-version) 00:27:04 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-1T4P from file:/tmp/.os_lf_venv 00:27:04 lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv) 00:27:04 lf-activate-venv(): INFO: Attempting to install with network-safe options... 00:27:06 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 00:27:06 python-manilaclient 6.2.0 requires osc-lib>=3.2.0, but you have osc-lib 2.2.1 which is incompatible. 00:27:06 lf-activate-venv(): INFO: Base packages installed successfully 00:27:06 lf-activate-venv(): INFO: Installing additional packages: lftools[openstack] kubernetes python-heatclient python-openstackclient urllib3~=1.26.15 00:27:23 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 00:27:23 python-manilaclient 6.2.0 requires osc-lib>=3.2.0, but you have osc-lib 2.2.1 which is incompatible. 00:27:24 lf-activate-venv(): INFO: Adding /tmp/venv-1T4P/bin to PATH 00:27:25 Stack not found: releng-controller-csit-3node-clustering-tell-all-titanium-96 00:27:26 SSH ready on all stack servers. 00:27:26 [controller-csit-3node-clustering-tell-all-titanium] $ /bin/bash -l /tmp/jenkins13636085152188266034.sh 00:27:26 Setup pyenv: 00:27:26 system 00:27:26 3.8.20 00:27:26 3.9.20 00:27:26 3.10.15 00:27:26 * 3.11.10 (set by /w/workspace/controller-csit-3node-clustering-tell-all-titanium/.python-version) 00:27:30 lf-activate-venv(): INFO: Creating python3 venv at /tmp/venv-Fk7m 00:27:30 lf-activate-venv(): INFO: Save venv in file: /w/workspace/controller-csit-3node-clustering-tell-all-titanium/.robot_venv 00:27:30 lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv) 00:27:30 lf-activate-venv(): INFO: Attempting to install with network-safe options... 00:27:34 lf-activate-venv(): INFO: Base packages installed successfully 00:27:34 lf-activate-venv(): INFO: Installing additional packages: setuptools wheel 00:27:36 lf-activate-venv(): INFO: Adding /tmp/venv-Fk7m/bin to PATH 00:27:36 + echo 'Installing Python Requirements' 00:27:36 Installing Python Requirements 00:27:36 + cat 00:27:36 + python -m pip install -r requirements.txt 00:27:36 Looking in indexes: https://nexus3.opendaylight.org/repository/PyPi/simple 00:27:37 Collecting docker-py (from -r requirements.txt (line 1)) 00:27:37 Downloading docker_py-1.10.6-py2.py3-none-any.whl (50 kB) 00:27:37 Collecting ipaddr (from -r requirements.txt (line 2)) 00:27:37 Downloading ipaddr-2.2.0.tar.gz (26 kB) 00:27:37 Installing build dependencies: started 00:27:37 Installing build dependencies: finished with status 'done' 00:27:37 Getting requirements to build wheel: started 00:27:38 Getting requirements to build wheel: finished with status 'done' 00:27:38 Preparing metadata (pyproject.toml): started 00:27:38 Preparing metadata (pyproject.toml): finished with status 'done' 00:27:38 Collecting netaddr (from -r requirements.txt (line 3)) 00:27:38 Using cached netaddr-1.3.0-py3-none-any.whl (2.3 MB) 00:27:38 Collecting netifaces (from -r requirements.txt (line 4)) 00:27:38 Downloading netifaces-0.11.0.tar.gz (30 kB) 00:27:38 Installing build dependencies: started 00:27:39 Installing build dependencies: finished with status 'done' 00:27:39 Getting requirements to build wheel: started 00:27:40 Getting requirements to build wheel: finished with status 'done' 00:27:40 Preparing metadata (pyproject.toml): started 00:27:40 Preparing metadata (pyproject.toml): finished with status 'done' 00:27:40 Collecting pyhocon (from -r requirements.txt (line 5)) 00:27:40 Downloading pyhocon-0.3.63.tar.gz (307 kB) 00:27:40 Installing build dependencies: started 00:27:41 Installing build dependencies: finished with status 'done' 00:27:41 Getting requirements to build wheel: started 00:27:41 Getting requirements to build wheel: finished with status 'done' 00:27:41 Preparing metadata (pyproject.toml): started 00:27:41 Preparing metadata (pyproject.toml): finished with status 'done' 00:27:42 Collecting requests (from -r requirements.txt (line 6)) 00:27:42 Using cached requests-2.34.2-py3-none-any.whl (73 kB) 00:27:42 Collecting robotframework (from -r requirements.txt (line 7)) 00:27:42 Downloading robotframework-7.4.2-py3-none-any.whl (807 kB) 00:27:42 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 807.1/807.1 kB 30.4 MB/s 0:00:00 00:27:42 Collecting robotframework-httplibrary (from -r requirements.txt (line 8)) 00:27:42 Downloading robotframework-httplibrary-0.4.2.tar.gz (9.1 kB) 00:27:42 Installing build dependencies: started 00:27:43 Installing build dependencies: finished with status 'done' 00:27:43 Getting requirements to build wheel: started 00:27:43 Getting requirements to build wheel: finished with status 'done' 00:27:43 Preparing metadata (pyproject.toml): started 00:27:43 Preparing metadata (pyproject.toml): finished with status 'done' 00:27:43 Collecting robotframework-requests==0.9.7 (from -r requirements.txt (line 9)) 00:27:43 Downloading robotframework_requests-0.9.7-py3-none-any.whl (21 kB) 00:27:43 Collecting robotframework-selenium2library (from -r requirements.txt (line 10)) 00:27:43 Downloading robotframework_selenium2library-3.0.0-py2.py3-none-any.whl (6.2 kB) 00:27:43 Collecting robotframework-sshlibrary==3.8.0 (from -r requirements.txt (line 11)) 00:27:43 Downloading robotframework-sshlibrary-3.8.0.tar.gz (51 kB) 00:27:43 Installing build dependencies: started 00:27:44 Installing build dependencies: finished with status 'done' 00:27:44 Getting requirements to build wheel: started 00:27:45 Getting requirements to build wheel: finished with status 'done' 00:27:45 Preparing metadata (pyproject.toml): started 00:27:45 Preparing metadata (pyproject.toml): finished with status 'done' 00:27:45 Collecting scapy (from -r requirements.txt (line 12)) 00:27:45 Downloading scapy-2.7.0-py3-none-any.whl (2.6 MB) 00:27:45 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.6/2.6 MB 20.8 MB/s 0:00:00 00:27:45 Collecting jsonpath-rw (from -r requirements.txt (line 15)) 00:27:45 Downloading jsonpath-rw-1.4.0.tar.gz (13 kB) 00:27:45 Installing build dependencies: started 00:27:46 Installing build dependencies: finished with status 'done' 00:27:46 Getting requirements to build wheel: started 00:27:47 Getting requirements to build wheel: finished with status 'done' 00:27:47 Preparing metadata (pyproject.toml): started 00:27:47 Preparing metadata (pyproject.toml): finished with status 'done' 00:27:47 Collecting elasticsearch (from -r requirements.txt (line 18)) 00:27:47 Downloading elasticsearch-9.4.1-py3-none-any.whl (993 kB) 00:27:47 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 993.6/993.6 kB 5.2 MB/s 0:00:00 00:27:47 Collecting elasticsearch-dsl (from -r requirements.txt (line 19)) 00:27:47 Downloading elasticsearch_dsl-8.18.0-py3-none-any.whl (10 kB) 00:27:47 Collecting pyangbind (from -r requirements.txt (line 22)) 00:27:47 Downloading pyangbind-0.8.7-py3-none-any.whl (52 kB) 00:27:48 Collecting isodate (from -r requirements.txt (line 25)) 00:27:48 Downloading isodate-0.7.2-py3-none-any.whl (22 kB) 00:27:48 Collecting jmespath (from -r requirements.txt (line 28)) 00:27:48 Using cached jmespath-1.1.0-py3-none-any.whl (20 kB) 00:27:48 Collecting jsonpatch (from -r requirements.txt (line 31)) 00:27:48 Using cached jsonpatch-1.33-py2.py3-none-any.whl (12 kB) 00:27:48 Collecting paramiko>=1.15.3 (from robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11)) 00:27:48 Downloading paramiko-5.0.0-py3-none-any.whl (208 kB) 00:27:48 Collecting scp>=0.13.0 (from robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11)) 00:27:48 Downloading scp-0.16.0-py2.py3-none-any.whl (8.9 kB) 00:27:48 Collecting docker-pycreds>=0.2.1 (from docker-py->-r requirements.txt (line 1)) 00:27:48 Downloading docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB) 00:27:48 Collecting six>=1.4.0 (from docker-py->-r requirements.txt (line 1)) 00:27:48 Using cached six-1.17.0-py2.py3-none-any.whl (11 kB) 00:27:48 Collecting websocket-client>=0.32.0 (from docker-py->-r requirements.txt (line 1)) 00:27:48 Using cached websocket_client-1.9.0-py3-none-any.whl (82 kB) 00:27:48 Collecting pyparsing>=3.0.0 (from pyhocon->-r requirements.txt (line 5)) 00:27:48 Using cached pyparsing-3.3.2-py3-none-any.whl (122 kB) 00:27:48 Collecting charset_normalizer<4,>=2 (from requests->-r requirements.txt (line 6)) 00:27:48 Using cached charset_normalizer-3.4.9-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (221 kB) 00:27:48 Collecting idna<4,>=2.5 (from requests->-r requirements.txt (line 6)) 00:27:48 Using cached idna-3.18-py3-none-any.whl (65 kB) 00:27:48 Collecting urllib3<3,>=1.26 (from requests->-r requirements.txt (line 6)) 00:27:48 Using cached urllib3-2.7.0-py3-none-any.whl (131 kB) 00:27:49 Collecting certifi>=2023.5.7 (from requests->-r requirements.txt (line 6)) 00:27:49 Using cached certifi-2026.7.22-py3-none-any.whl (136 kB) 00:27:49 Collecting webtest>=2.0 (from robotframework-httplibrary->-r requirements.txt (line 8)) 00:27:49 Downloading webtest-3.0.7-py3-none-any.whl (32 kB) 00:27:49 Collecting jsonpointer (from robotframework-httplibrary->-r requirements.txt (line 8)) 00:27:49 Using cached jsonpointer-3.1.1-py3-none-any.whl (7.7 kB) 00:27:49 Collecting robotframework-seleniumlibrary>=3.0.0 (from robotframework-selenium2library->-r requirements.txt (line 10)) 00:27:49 Downloading robotframework_seleniumlibrary-6.9.0-py3-none-any.whl (105 kB) 00:27:49 Collecting ply (from jsonpath-rw->-r requirements.txt (line 15)) 00:27:49 Downloading ply-3.11-py2.py3-none-any.whl (49 kB) 00:27:49 Collecting decorator (from jsonpath-rw->-r requirements.txt (line 15)) 00:27:49 Using cached decorator-5.3.1-py3-none-any.whl (10 kB) 00:27:49 Collecting anyio (from elasticsearch->-r requirements.txt (line 18)) 00:27:49 Downloading anyio-4.14.2-py3-none-any.whl (125 kB) 00:27:49 Collecting elastic-transport<10,>=9.4.1 (from elasticsearch->-r requirements.txt (line 18)) 00:27:49 Downloading elastic_transport-9.4.2-py3-none-any.whl (66 kB) 00:27:49 Collecting python-dateutil (from elasticsearch->-r requirements.txt (line 18)) 00:27:49 Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) 00:27:49 Collecting sniffio (from elasticsearch->-r requirements.txt (line 18)) 00:27:49 Downloading sniffio-1.3.1-py3-none-any.whl (10 kB) 00:27:49 Collecting typing-extensions (from elasticsearch->-r requirements.txt (line 18)) 00:27:49 Using cached typing_extensions-4.16.0-py3-none-any.whl (45 kB) 00:27:49 Collecting elasticsearch (from -r requirements.txt (line 18)) 00:27:49 Downloading elasticsearch-8.19.3-py3-none-any.whl (952 kB) 00:27:49 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 952.8/952.8 kB 13.9 MB/s 0:00:00 00:27:49 INFO: pip is looking at multiple versions of elasticsearch-dsl to determine which version is compatible with other requirements. This could take a while. 00:27:49 Collecting elasticsearch-dsl (from -r requirements.txt (line 19)) 00:27:49 Downloading elasticsearch_dsl-8.17.1-py3-none-any.whl (158 kB) 00:27:49 Downloading elasticsearch_dsl-8.17.0-py3-none-any.whl (158 kB) 00:27:49 Downloading elasticsearch_dsl-8.16.0-py3-none-any.whl (158 kB) 00:27:49 Downloading elasticsearch_dsl-8.15.4-py3-none-any.whl (104 kB) 00:27:49 Collecting elastic-transport<9,>=8.15.1 (from elasticsearch->-r requirements.txt (line 18)) 00:27:49 Downloading elastic_transport-8.19.0-py3-none-any.whl (65 kB) 00:27:49 Collecting pyang (from pyangbind->-r requirements.txt (line 22)) 00:27:49 Downloading pyang-2.7.1-py2.py3-none-any.whl (598 kB) 00:27:49 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 598.5/598.5 kB 26.6 MB/s 0:00:00 00:27:50 Collecting lxml (from pyangbind->-r requirements.txt (line 22)) 00:27:50 Using cached lxml-6.1.1-cp311-cp311-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl (5.2 MB) 00:27:51 Collecting regex (from pyangbind->-r requirements.txt (line 22)) 00:27:51 Downloading regex-2026.7.19-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (801 kB) 00:27:51 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 801.4/801.4 kB 29.7 MB/s 0:00:00 00:27:51 Collecting enum34 (from pyangbind->-r requirements.txt (line 22)) 00:27:51 Downloading enum34-1.1.10-py3-none-any.whl (11 kB) 00:27:51 Collecting bcrypt>=3.2 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11)) 00:27:51 Downloading bcrypt-5.0.0-cp39-abi3-manylinux_2_34_x86_64.whl (278 kB) 00:27:52 Collecting cryptography>=3.3 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11)) 00:27:52 Using cached cryptography-49.0.0-cp311-abi3-manylinux_2_34_x86_64.whl (4.7 MB) 00:27:52 Collecting invoke>=2.0 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11)) 00:27:52 Downloading invoke-3.0.3-py3-none-any.whl (160 kB) 00:27:52 Collecting pynacl>=1.5 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11)) 00:27:52 Using cached pynacl-1.6.2-cp38-abi3-manylinux_2_34_x86_64.whl (1.4 MB) 00:27:52 Collecting cffi>=2.0.0 (from cryptography>=3.3->paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11)) 00:27:52 Using cached cffi-2.1.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (217 kB) 00:27:52 Collecting pycparser (from cffi>=2.0.0->cryptography>=3.3->paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11)) 00:27:52 Using cached pycparser-3.0-py3-none-any.whl (48 kB) 00:27:52 Collecting selenium>=4.25.0 (from robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) 00:27:52 Downloading selenium-4.46.0-py3-none-any.whl (9.4 MB) 00:27:52 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.4/9.4 MB 64.8 MB/s 0:00:00 00:27:52 Collecting robotframework-pythonlibcore>=4.4.1 (from robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) 00:27:52 Downloading robotframework_pythonlibcore-4.6.0-py3-none-any.whl (20 kB) 00:27:52 Collecting click>=8.0 (from robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) 00:27:52 Using cached click-8.4.2-py3-none-any.whl (119 kB) 00:27:52 Collecting trio<1.0,>=0.31.0 (from selenium>=4.25.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) 00:27:53 Downloading trio-0.33.0-py3-none-any.whl (510 kB) 00:27:53 Collecting trio-websocket<1.0,>=0.12.2 (from selenium>=4.25.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) 00:27:53 Downloading trio_websocket-0.12.2-py3-none-any.whl (21 kB) 00:27:53 Collecting attrs>=23.2.0 (from trio<1.0,>=0.31.0->selenium>=4.25.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) 00:27:53 Using cached attrs-26.1.0-py3-none-any.whl (67 kB) 00:27:53 Collecting sortedcontainers (from trio<1.0,>=0.31.0->selenium>=4.25.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) 00:27:53 Downloading sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB) 00:27:53 Collecting outcome (from trio<1.0,>=0.31.0->selenium>=4.25.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) 00:27:53 Downloading outcome-1.3.0.post0-py2.py3-none-any.whl (10 kB) 00:27:53 Collecting wsproto>=0.14 (from trio-websocket<1.0,>=0.12.2->selenium>=4.25.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) 00:27:53 Downloading wsproto-1.3.2-py3-none-any.whl (24 kB) 00:27:53 Collecting pysocks!=1.5.7,<2.0,>=1.5.6 (from urllib3[socks]<3.0,>=2.6.3->selenium>=4.25.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) 00:27:53 Downloading PySocks-1.7.1-py3-none-any.whl (16 kB) 00:27:53 Collecting WebOb>=1.2 (from webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8)) 00:27:53 Downloading webob-1.8.10-py2.py3-none-any.whl (115 kB) 00:27:53 Collecting waitress>=3.0.2 (from webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8)) 00:27:53 Downloading waitress-3.0.2-py3-none-any.whl (56 kB) 00:27:53 Collecting beautifulsoup4 (from webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8)) 00:27:53 Using cached beautifulsoup4-4.15.0-py3-none-any.whl (109 kB) 00:27:53 Collecting h11<1,>=0.16.0 (from wsproto>=0.14->trio-websocket<1.0,>=0.12.2->selenium>=4.25.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) 00:27:53 Downloading h11-0.16.0-py3-none-any.whl (37 kB) 00:27:53 Collecting soupsieve>=1.6.1 (from beautifulsoup4->webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8)) 00:27:53 Using cached soupsieve-2.9.1-py3-none-any.whl (37 kB) 00:27:53 Building wheels for collected packages: robotframework-sshlibrary, ipaddr, netifaces, pyhocon, robotframework-httplibrary, jsonpath-rw 00:27:53 Building wheel for robotframework-sshlibrary (pyproject.toml): started 00:27:53 Building wheel for robotframework-sshlibrary (pyproject.toml): finished with status 'done' 00:27:53 Created wheel for robotframework-sshlibrary: filename=robotframework_sshlibrary-3.8.0-py3-none-any.whl size=55203 sha256=12575e0ed7f4dc7004ffd3109fe1579dad522cd81441b0a84b06b8d1a234e172 00:27:53 Stored in directory: /home/jenkins/.cache/pip/wheels/f7/c9/b3/a977b7bcc410d45ae27d240df3d00a12585509180e373ecccc 00:27:53 Building wheel for ipaddr (pyproject.toml): started 00:27:54 Building wheel for ipaddr (pyproject.toml): finished with status 'done' 00:27:54 Created wheel for ipaddr: filename=ipaddr-2.2.0-py3-none-any.whl size=18353 sha256=be74d26a470d36e5e506f2b7e0e642eacd6e8769c9b47a53e9c55d373b6395bc 00:27:54 Stored in directory: /home/jenkins/.cache/pip/wheels/dc/6c/04/da2d847fa8d45c59af3e1d83e2acc29cb8adcbaf04c0898dbf 00:27:54 Building wheel for netifaces (pyproject.toml): started 00:27:57 Building wheel for netifaces (pyproject.toml): finished with status 'done' 00:27:57 Created wheel for netifaces: filename=netifaces-0.11.0-cp311-cp311-linux_x86_64.whl size=34974 sha256=ef1acbc72c0a2025999194f4be7646fd3fe34d4a55625bb60d6e668b7f01f03c 00:27:57 Stored in directory: /home/jenkins/.cache/pip/wheels/f8/18/88/e61d54b995bea304bdb1d040a92b72228a1bf72ca2a3eba7c9 00:27:57 Building wheel for pyhocon (pyproject.toml): started 00:27:57 Building wheel for pyhocon (pyproject.toml): finished with status 'done' 00:27:57 Created wheel for pyhocon: filename=pyhocon-0.3.63-py3-none-any.whl size=24720 sha256=9147ffdc672b6e981db4dce06cd7795e10dfa6b7946a1ac516779a71cbb37cea 00:27:57 Stored in directory: /home/jenkins/.cache/pip/wheels/ce/9e/92/cb17e17d79fafd46a767074b3ed1dca8f78036163f9bea914d 00:27:57 Building wheel for robotframework-httplibrary (pyproject.toml): started 00:27:57 Building wheel for robotframework-httplibrary (pyproject.toml): finished with status 'done' 00:27:57 Created wheel for robotframework-httplibrary: filename=robotframework_httplibrary-0.4.2-py3-none-any.whl size=10014 sha256=39c5192697b0471ad614961e095cbe3464d328e5e64990c0cc7760da55c387e5 00:27:57 Stored in directory: /home/jenkins/.cache/pip/wheels/aa/bc/0d/9a20dd51effef392aae2733cb4c7b66c6fa29fca33d88b57ed 00:27:57 Building wheel for jsonpath-rw (pyproject.toml): started 00:27:57 Building wheel for jsonpath-rw (pyproject.toml): finished with status 'done' 00:27:57 Created wheel for jsonpath-rw: filename=jsonpath_rw-1.4.0-py3-none-any.whl size=15176 sha256=9dee7b81edc4d4c55882c37fa27cf953b70f19d87efd5d8641f1b30427a7b117 00:27:57 Stored in directory: /home/jenkins/.cache/pip/wheels/f1/54/63/9a8da38cefae13755097b36cc852decc25d8ef69c37d58d4eb 00:27:57 Successfully built robotframework-sshlibrary ipaddr netifaces pyhocon robotframework-httplibrary jsonpath-rw 00:27:57 Installing collected packages: sortedcontainers, ply, netifaces, ipaddr, enum34, websocket-client, WebOb, waitress, urllib3, typing-extensions, soupsieve, sniffio, six, scapy, robotframework-pythonlibcore, robotframework, regex, pysocks, pyparsing, pycparser, netaddr, lxml, jsonpointer, jmespath, isodate, invoke, idna, h11, decorator, click, charset_normalizer, certifi, bcrypt, attrs, wsproto, requests, python-dateutil, pyhocon, pyang, outcome, jsonpath-rw, jsonpatch, elastic-transport, docker-pycreds, cffi, beautifulsoup4, webtest, trio, robotframework-requests, pynacl, pyangbind, elasticsearch, docker-py, cryptography, trio-websocket, robotframework-httplibrary, paramiko, elasticsearch-dsl, selenium, scp, robotframework-sshlibrary, robotframework-seleniumlibrary, robotframework-selenium2library 00:28:05 00:28:05 Successfully installed WebOb-1.8.10 attrs-26.1.0 bcrypt-5.0.0 beautifulsoup4-4.15.0 certifi-2026.7.22 cffi-2.1.0 charset_normalizer-3.4.9 click-8.4.2 cryptography-49.0.0 decorator-5.3.1 docker-py-1.10.6 docker-pycreds-0.4.0 elastic-transport-8.19.0 elasticsearch-8.19.3 elasticsearch-dsl-8.15.4 enum34-1.1.10 h11-0.16.0 idna-3.18 invoke-3.0.3 ipaddr-2.2.0 isodate-0.7.2 jmespath-1.1.0 jsonpatch-1.33 jsonpath-rw-1.4.0 jsonpointer-3.1.1 lxml-6.1.1 netaddr-1.3.0 netifaces-0.11.0 outcome-1.3.0.post0 paramiko-5.0.0 ply-3.11 pyang-2.7.1 pyangbind-0.8.7 pycparser-3.0 pyhocon-0.3.63 pynacl-1.6.2 pyparsing-3.3.2 pysocks-1.7.1 python-dateutil-2.9.0.post0 regex-2026.7.19 requests-2.34.2 robotframework-7.4.2 robotframework-httplibrary-0.4.2 robotframework-pythonlibcore-4.6.0 robotframework-requests-0.9.7 robotframework-selenium2library-3.0.0 robotframework-seleniumlibrary-6.9.0 robotframework-sshlibrary-3.8.0 scapy-2.7.0 scp-0.16.0 selenium-4.46.0 six-1.17.0 sniffio-1.3.1 sortedcontainers-2.4.0 soupsieve-2.9.1 trio-0.33.0 trio-websocket-0.12.2 typing-extensions-4.16.0 urllib3-2.7.0 waitress-3.0.2 websocket-client-1.9.0 webtest-3.0.7 wsproto-1.3.2 00:28:05 + pip freeze 00:28:06 attrs==26.1.0 00:28:06 bcrypt==5.0.0 00:28:06 beautifulsoup4==4.15.0 00:28:06 certifi==2026.7.22 00:28:06 cffi==2.1.0 00:28:06 charset-normalizer==3.4.9 00:28:06 click==8.4.2 00:28:06 cryptography==49.0.0 00:28:06 decorator==5.3.1 00:28:06 distlib==0.4.3 00:28:06 docker-py==1.10.6 00:28:06 docker-pycreds==0.4.0 00:28:06 elastic-transport==8.19.0 00:28:06 elasticsearch==8.19.3 00:28:06 elasticsearch-dsl==8.15.4 00:28:06 enum34==1.1.10 00:28:06 filelock==3.32.0 00:28:06 h11==0.16.0 00:28:06 idna==3.18 00:28:06 invoke==3.0.3 00:28:06 ipaddr==2.2.0 00:28:06 isodate==0.7.2 00:28:06 jmespath==1.1.0 00:28:06 jsonpatch==1.33 00:28:06 jsonpath-rw==1.4.0 00:28:06 jsonpointer==3.1.1 00:28:06 lxml==6.1.1 00:28:06 netaddr==1.3.0 00:28:06 netifaces==0.11.0 00:28:06 outcome==1.3.0.post0 00:28:06 packaging==26.2 00:28:06 paramiko==5.0.0 00:28:06 platformdirs==4.11.0 00:28:06 ply==3.11 00:28:06 pyang==2.7.1 00:28:06 pyangbind==0.8.7 00:28:06 pycparser==3.0 00:28:06 pyhocon==0.3.63 00:28:06 PyNaCl==1.6.2 00:28:06 pyparsing==3.3.2 00:28:06 PySocks==1.7.1 00:28:06 python-dateutil==2.9.0.post0 00:28:06 python-discovery==1.5.0 00:28:06 regex==2026.7.19 00:28:06 requests==2.34.2 00:28:06 robotframework==7.4.2 00:28:06 robotframework-httplibrary==0.4.2 00:28:06 robotframework-pythonlibcore==4.6.0 00:28:06 robotframework-requests==0.9.7 00:28:06 robotframework-selenium2library==3.0.0 00:28:06 robotframework-seleniumlibrary==6.9.0 00:28:06 robotframework-sshlibrary==3.8.0 00:28:06 scapy==2.7.0 00:28:06 scp==0.16.0 00:28:06 selenium==4.46.0 00:28:06 six==1.17.0 00:28:06 sniffio==1.3.1 00:28:06 sortedcontainers==2.4.0 00:28:06 soupsieve==2.9.1 00:28:06 trio==0.33.0 00:28:06 trio-websocket==0.12.2 00:28:06 typing_extensions==4.16.0 00:28:06 urllib3==2.7.0 00:28:06 virtualenv==21.7.0 00:28:06 waitress==3.0.2 00:28:06 WebOb==1.8.10 00:28:06 websocket-client==1.9.0 00:28:06 WebTest==3.0.7 00:28:06 wsproto==1.3.2 00:28:06 [EnvInject] - Injecting environment variables from a build step. 00:28:06 [EnvInject] - Injecting as environment variables the properties file path 'env.properties' 00:28:06 [EnvInject] - Variables injected successfully. 00:28:06 [controller-csit-3node-clustering-tell-all-titanium] $ /bin/bash -l /tmp/jenkins3432567151744412357.sh 00:28:06 Setup pyenv: 00:28:06 system 00:28:06 3.8.20 00:28:06 3.9.20 00:28:06 3.10.15 00:28:06 * 3.11.10 (set by /w/workspace/controller-csit-3node-clustering-tell-all-titanium/.python-version) 00:28:06 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-1T4P from file:/tmp/.os_lf_venv 00:28:06 lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv) 00:28:06 lf-activate-venv(): INFO: Attempting to install with network-safe options... 00:28:08 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 00:28:08 python-manilaclient 6.2.0 requires osc-lib>=3.2.0, but you have osc-lib 2.2.1 which is incompatible. 00:28:08 lf-activate-venv(): INFO: Base packages installed successfully 00:28:08 lf-activate-venv(): INFO: Installing additional packages: python-heatclient python-openstackclient yq 00:28:16 lf-activate-venv(): INFO: Adding /tmp/venv-1T4P/bin to PATH 00:28:16 + ODL_SYSTEM=() 00:28:16 + TOOLS_SYSTEM=() 00:28:16 + OPENSTACK_SYSTEM=() 00:28:16 + OPENSTACK_CONTROLLERS=() 00:28:16 + mapfile -t ADDR 00:28:16 ++ jq -r '.outputs[] | select(.output_key | match("^vm_[0-9]+_ips$")) | .output_value | .[]' 00:28:16 ++ openstack stack show -f json -c outputs releng-controller-csit-3node-clustering-tell-all-titanium-96 00:28:18 Stack not found: releng-controller-csit-3node-clustering-tell-all-titanium-96 00:28:18 + for i in "${ADDR[@]}" 00:28:18 ++ ssh '' hostname -s 00:28:18 ssh: Could not resolve hostname : Device or resource busy 00:28:18 + REMHOST= 00:28:18 + case ${REMHOST} in 00:28:18 + TOOLS_SYSTEM=("${TOOLS_SYSTEM[@]}" "${i}") 00:28:18 + echo NUM_ODL_SYSTEM=0 00:28:18 + echo NUM_TOOLS_SYSTEM=1 00:28:18 + '[' '' == yes ']' 00:28:18 + NUM_OPENSTACK_SYSTEM=0 00:28:18 + echo NUM_OPENSTACK_SYSTEM=0 00:28:18 + '[' 0 -eq 2 ']' 00:28:18 + echo ODL_SYSTEM_IP= 00:28:18 ++ seq 0 -1 00:28:18 + echo TOOLS_SYSTEM_IP= 00:28:18 ++ seq 0 0 00:28:18 + for i in $(seq 0 $(( ${#TOOLS_SYSTEM[@]} - 1 ))) 00:28:18 + echo TOOLS_SYSTEM_1_IP= 00:28:18 + openstack_index=0 00:28:18 + NUM_OPENSTACK_CONTROL_NODES=1 00:28:18 + echo NUM_OPENSTACK_CONTROL_NODES=1 00:28:18 ++ seq 0 0 00:28:18 + for i in $(seq 0 $((NUM_OPENSTACK_CONTROL_NODES - 1))) 00:28:18 + echo OPENSTACK_CONTROL_NODE_1_IP= 00:28:18 + NUM_OPENSTACK_COMPUTE_NODES=-1 00:28:18 + echo NUM_OPENSTACK_COMPUTE_NODES=-1 00:28:18 + '[' -1 -ge 2 ']' 00:28:18 ++ seq 0 -2 00:28:18 + NUM_OPENSTACK_HAPROXY_NODES=0 00:28:18 + echo NUM_OPENSTACK_HAPROXY_NODES=0 00:28:18 ++ seq 0 -1 00:28:18 + echo 'Contents of slave_addresses.txt:' 00:28:18 Contents of slave_addresses.txt: 00:28:18 + cat slave_addresses.txt 00:28:18 NUM_ODL_SYSTEM=0 00:28:18 NUM_TOOLS_SYSTEM=1 00:28:18 NUM_OPENSTACK_SYSTEM=0 00:28:18 ODL_SYSTEM_IP= 00:28:18 TOOLS_SYSTEM_IP= 00:28:18 TOOLS_SYSTEM_1_IP= 00:28:18 NUM_OPENSTACK_CONTROL_NODES=1 00:28:18 OPENSTACK_CONTROL_NODE_1_IP= 00:28:18 NUM_OPENSTACK_COMPUTE_NODES=-1 00:28:18 NUM_OPENSTACK_HAPROXY_NODES=0 00:28:18 [EnvInject] - Injecting environment variables from a build step. 00:28:18 [EnvInject] - Injecting as environment variables the properties file path 'slave_addresses.txt' 00:28:18 [EnvInject] - Variables injected successfully. 00:28:18 [controller-csit-3node-clustering-tell-all-titanium] $ /bin/sh /tmp/jenkins16058095981957682050.sh 00:28:18 Preparing for JRE Version 21 00:28:18 Karaf artifact is karaf 00:28:18 Karaf project is integration 00:28:18 Java home is /usr/lib/jvm/java-21-openjdk-amd64 00:28:18 [EnvInject] - Injecting environment variables from a build step. 00:28:18 [EnvInject] - Injecting as environment variables the properties file path 'set_variables.env' 00:28:18 [EnvInject] - Variables injected successfully. 00:28:18 [controller-csit-3node-clustering-tell-all-titanium] $ /bin/bash /tmp/jenkins463231930376152405.sh 00:28:18 2026-07-26 00:28:18 URL:https://raw.githubusercontent.com/opendaylight/integration-distribution/stable/titanium/pom.xml [2619/2619] -> "pom.xml" [1] 00:28:18 Bundle version is 0.22.3-SNAPSHOT 00:28:18 --2026-07-26 00:28:18-- https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.22.3-SNAPSHOT/maven-metadata.xml 00:28:18 Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6 00:28:18 Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected. 00:28:18 HTTP request sent, awaiting response... 200 OK 00:28:18 Length: 1410 (1.4K) [application/xml] 00:28:18 Saving to: ‘maven-metadata.xml’ 00:28:18 00:28:18 0K . 100% 1.54G=0s 00:28:18 00:28:18 2026-07-26 00:28:18 (1.54 GB/s) - ‘maven-metadata.xml’ saved [1410/1410] 00:28:18 00:28:18 00:28:18 00:28:18 org.opendaylight.integration 00:28:18 karaf 00:28:18 0.22.3-SNAPSHOT 00:28:18 00:28:18 00:28:18 20260725.175624 00:28:18 478 00:28:18 00:28:18 20260725175624 00:28:18 00:28:18 00:28:18 pom 00:28:18 0.22.3-20260725.175624-478 00:28:18 20260725175624 00:28:18 00:28:18 00:28:18 tar.gz 00:28:18 0.22.3-20260725.175624-478 00:28:18 20260725175624 00:28:18 00:28:18 00:28:18 zip 00:28:18 0.22.3-20260725.175624-478 00:28:18 20260725175624 00:28:18 00:28:18 00:28:18 cyclonedx 00:28:18 xml 00:28:18 0.22.3-20260725.175624-478 00:28:18 20260725175624 00:28:18 00:28:18 00:28:18 cyclonedx 00:28:18 json 00:28:18 0.22.3-20260725.175624-478 00:28:18 20260725175624 00:28:18 00:28:18 00:28:18 00:28:18 00:28:18 Nexus timestamp is 0.22.3-20260725.175624-478 00:28:18 Distribution bundle URL is https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.22.3-SNAPSHOT/karaf-0.22.3-20260725.175624-478.zip 00:28:18 Distribution bundle is karaf-0.22.3-20260725.175624-478.zip 00:28:18 Distribution bundle version is 0.22.3-SNAPSHOT 00:28:18 Distribution folder is karaf-0.22.3-SNAPSHOT 00:28:18 Nexus prefix is https://nexus.opendaylight.org 00:28:18 [EnvInject] - Injecting environment variables from a build step. 00:28:18 [EnvInject] - Injecting as environment variables the properties file path 'detect_variables.env' 00:28:18 [EnvInject] - Variables injected successfully. 00:28:18 [controller-csit-3node-clustering-tell-all-titanium] $ /bin/bash -l /tmp/jenkins4409714297295811582.sh 00:28:18 Setup pyenv: 00:28:18 system 00:28:18 3.8.20 00:28:18 3.9.20 00:28:18 3.10.15 00:28:18 * 3.11.10 (set by /w/workspace/controller-csit-3node-clustering-tell-all-titanium/.python-version) 00:28:19 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-1T4P from file:/tmp/.os_lf_venv 00:28:19 lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv) 00:28:19 lf-activate-venv(): INFO: Attempting to install with network-safe options... 00:28:21 lf-activate-venv(): INFO: Base packages installed successfully 00:28:21 lf-activate-venv(): INFO: Installing additional packages: python-heatclient python-openstackclient 00:28:27 lf-activate-venv(): INFO: Adding /tmp/venv-1T4P/bin to PATH 00:28:27 Copying common-functions.sh to /tmp 00:28:29 Stack not found: releng-controller-csit-3node-clustering-tell-all-titanium-96 00:28:29 Copying common-functions.sh to :/tmp 00:28:29 cp: cannot create regular file ':/tmp': No such file or directory 00:28:29 [controller-csit-3node-clustering-tell-all-titanium] $ /bin/bash /tmp/jenkins11137459431978444325.sh 00:28:29 common-functions.sh is being sourced 00:28:29 common-functions environment: 00:28:29 MAVENCONF: /tmp/karaf-0.22.3-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg 00:28:29 ACTUALFEATURES: 00:28:29 FEATURESCONF: /tmp/karaf-0.22.3-SNAPSHOT/etc/org.apache.karaf.features.cfg 00:28:29 CUSTOMPROP: /tmp/karaf-0.22.3-SNAPSHOT/etc/custom.properties 00:28:29 LOGCONF: /tmp/karaf-0.22.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg 00:28:29 MEMCONF: /tmp/karaf-0.22.3-SNAPSHOT/bin/setenv 00:28:29 CONTROLLERMEM: 2048m 00:28:29 AKKACONF: /tmp/karaf-0.22.3-SNAPSHOT/configuration/initial/pekko.conf 00:28:29 MODULESCONF: /tmp/karaf-0.22.3-SNAPSHOT/configuration/initial/modules.conf 00:28:29 MODULESHARDSCONF: /tmp/karaf-0.22.3-SNAPSHOT/configuration/initial/module-shards.conf 00:28:29 SUITES: 00:28:29 00:28:29 ################################################# 00:28:29 ## Configure Cluster and Start ## 00:28:29 ################################################# 00:28:29 ACTUALFEATURES: odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app 00:28:29 SPACE_SEPARATED_FEATURES: odl-integration-compatible-with-all odl-jolokia odl-restconf odl-clustering-test-app 00:28:29 Locating script plan to use... 00:28:29 script plan exists!!! 00:28:29 Changing the script plan path... 00:28:29 # Place the scripts in run order: 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/scripts/car_people_shard.sh 00:28:29 Executing /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/scripts/car_people_shard.sh... 00:28:29 Add car-people shards file 00:28:29 Finished running script plans 00:28:29 Locating config plan to use... 00:28:29 Finished running config plans 00:28:29 [controller-csit-3node-clustering-tell-all-titanium] $ /bin/bash /tmp/jenkins14172162489890016425.sh 00:28:29 common-functions.sh is being sourced 00:28:29 common-functions environment: 00:28:29 MAVENCONF: /tmp/karaf-0.22.3-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg 00:28:29 ACTUALFEATURES: 00:28:29 FEATURESCONF: /tmp/karaf-0.22.3-SNAPSHOT/etc/org.apache.karaf.features.cfg 00:28:29 CUSTOMPROP: /tmp/karaf-0.22.3-SNAPSHOT/etc/custom.properties 00:28:29 LOGCONF: /tmp/karaf-0.22.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg 00:28:29 MEMCONF: /tmp/karaf-0.22.3-SNAPSHOT/bin/setenv 00:28:29 CONTROLLERMEM: 2048m 00:28:29 AKKACONF: /tmp/karaf-0.22.3-SNAPSHOT/configuration/initial/pekko.conf 00:28:29 MODULESCONF: /tmp/karaf-0.22.3-SNAPSHOT/configuration/initial/modules.conf 00:28:29 MODULESHARDSCONF: /tmp/karaf-0.22.3-SNAPSHOT/configuration/initial/module-shards.conf 00:28:29 SUITES: 00:28:29 00:28:29 + echo '#################################################' 00:28:29 ################################################# 00:28:29 + echo '## Verify Cluster is UP ##' 00:28:29 ## Verify Cluster is UP ## 00:28:29 + echo '#################################################' 00:28:29 ################################################# 00:28:29 + create_post_startup_script 00:28:29 + cat 00:28:29 + copy_and_run_post_startup_script 00:28:29 + seed_index=1 00:28:29 ++ seq 1 0 00:28:29 + dump_controller_threads 00:28:29 ++ seq 1 0 00:28:29 + '[' 0 -gt 0 ']' 00:28:29 + echo 'Generating controller variables...' 00:28:29 Generating controller variables... 00:28:29 ++ seq 1 0 00:28:29 + echo 'Generating mininet variables...' 00:28:29 Generating mininet variables... 00:28:29 ++ seq 1 1 00:28:29 + for i in $(seq 1 "${NUM_TOOLS_SYSTEM}") 00:28:29 + MININETIP=TOOLS_SYSTEM_1_IP 00:28:29 + tools_variables=' -v TOOLS_SYSTEM_1_IP:' 00:28:29 + get_test_suites SUITES 00:28:29 + local __suite_list=SUITES 00:28:29 + echo 'Locating test plan to use...' 00:28:29 Locating test plan to use... 00:28:29 + testplan_filepath=/w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/testplans/controller-clustering-tell-titanium.txt 00:28:29 + '[' '!' -f /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/testplans/controller-clustering-tell-titanium.txt ']' 00:28:29 + testplan_filepath=/w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/testplans/controller-clustering-tell.txt 00:28:29 + '[' disabled '!=' disabled ']' 00:28:29 + echo 'Changing the testplan path...' 00:28:29 Changing the testplan path... 00:28:29 + sed s:integration:/w/workspace/controller-csit-3node-clustering-tell-all-titanium: /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/testplans/controller-clustering-tell.txt 00:28:29 + cat testplan.txt 00:28:29 # Place the suites in run order: 00:28:29 00:28:29 # disabling all "prefix based" suites. This functionality was newly 00:28:29 # introduced with these tests in 2017. The assumption is that 00:28:29 # development stopped in that area, but may continue at a later 00:28:29 # time. Still, the tests are failing and no resources to solve the 00:28:29 # failures at this time. If/When this changes, we can uncomment 00:28:29 # those suites. Note that every suite is commented out with it's 00:28:29 # accompanying "restart_with_tell_based_true" setup suite. 00:28:29 00:28:29 00:28:29 # /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot 00:28:29 # /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/ddb-sanity-prefix-based.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/ddb-sanity-module-based.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/clean_leader_shutdown.robot 00:28:29 # /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot 00:28:29 # /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/clean_leader_shutdown_prefbasedshard.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement.robot 00:28:29 # /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot 00:28:29 # /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement_prefbasedshard.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/leader_isolation.robot 00:28:29 # /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot 00:28:29 # /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/leader_isolation_prefbasedshard.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/client_isolation.robot 00:28:29 # /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot 00:28:29 # /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/client_isolation_prefbasedshard.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/listener_stability.robot 00:28:29 # /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot 00:28:29 # /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/listener_stability_prefbasedshard.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/cluster_singleton/master_stability.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot 00:28:29 /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot 00:28:29 + '[' -z '' ']' 00:28:29 ++ tr '\012' ' ' 00:28:29 ++ grep -E -v '(^[[:space:]]*#|^[[:space:]]*$)' testplan.txt 00:28:29 + suite_list='/w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/ddb-sanity-module-based.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/clean_leader_shutdown.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/leader_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/client_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/listener_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot ' 00:28:29 + eval 'SUITES='\''/w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/ddb-sanity-module-based.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/clean_leader_shutdown.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/leader_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/client_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/listener_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot '\''' 00:28:29 ++ SUITES='/w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/ddb-sanity-module-based.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/clean_leader_shutdown.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/leader_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/client_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/listener_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot ' 00:28:29 + echo 'Starting Robot test suites /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/ddb-sanity-module-based.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/clean_leader_shutdown.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/leader_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/client_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/listener_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot ...' 00:28:29 Starting Robot test suites /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/ddb-sanity-module-based.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/clean_leader_shutdown.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/leader_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/client_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/listener_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot ... 00:28:29 + robot -N controller-clustering-tell.txt --removekeywords wuks -e exclude -e skip_if_titanium -v BUNDLEFOLDER:karaf-0.22.3-SNAPSHOT -v BUNDLE_URL:https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.22.3-SNAPSHOT/karaf-0.22.3-20260725.175624-478.zip -v CONTROLLER: -v CONTROLLER1: -v CONTROLLER2: -v CONTROLLER_USER:jenkins -v JAVA_HOME:/usr/lib/jvm/java-21-openjdk-amd64 -v JDKVERSION:openjdk21 -v JENKINS_WORKSPACE:/w/workspace/controller-csit-3node-clustering-tell-all-titanium -v MININET: -v MININET1: -v MININET2: -v MININET_USER:jenkins -v NEXUSURL_PREFIX:https://nexus.opendaylight.org -v NUM_ODL_SYSTEM:0 -v NUM_TOOLS_SYSTEM:1 -v ODL_STREAM:titanium -v ODL_SYSTEM_IP: -v ODL_SYSTEM_USER:jenkins -v TOOLS_SYSTEM_IP: -v TOOLS_SYSTEM_1_IP: -v TOOLS_SYSTEM_USER:jenkins -v USER_HOME:/home/jenkins -v IS_KARAF_APPL:True -v WORKSPACE:/tmp /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/ddb-sanity-module-based.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/clean_leader_shutdown.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/leader_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/client_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/listener_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot 00:28:29 ============================================================================== 00:28:29 controller-clustering-tell.txt 00:28:29 ============================================================================== 00:28:30 controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell... 00:28:30 ============================================================================== 00:28:30 Stop_All_Members :: Stop every odl node. | PASS | 00:28:30 ------------------------------------------------------------------------------ 00:28:30 Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS | 00:28:30 ------------------------------------------------------------------------------ 00:28:30 Start_All_And_Sync :: Start each member and wait for sync. | FAIL | 00:33:32 Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 5 minutes. The last error was: No leader found. 00:33:32 ------------------------------------------------------------------------------ 00:33:32 controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL | 00:33:32 3 tests, 2 passed, 1 failed 00:33:32 ============================================================================== 00:33:32 controller-clustering-tell.txt.Ddb-Sanity-Module-Based :: DOMDataBroker tes... 00:33:32 ============================================================================== 00:33:32 Get_Shard_Role :: Get shard role. | PASS | 00:33:32 ------------------------------------------------------------------------------ 00:33:32 Subscribe_Listener_To_Leader :: Subscribe listener to leader. | FAIL | 00:33:32 ... click for list of related bugs or create a new one if needed (with the 00:33:32 "controller_clustering_tell_txt_ddb_sanity_module_based_subscribe_listener_to_leader" 00:33:32 reference somewhere inside) 00:33:32 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_ddb_sanity_module_based_subscribe_listener_to_leader&order=bug_status" 00:33:32 00:33:32 No leader found. 00:33:32 ------------------------------------------------------------------------------ 00:33:32 Subscribe_Listener_To_Follower :: Subscribe listener to follower. | FAIL | 00:33:32 ... click for list of related bugs or create a new one if needed (with the 00:33:32 "controller_clustering_tell_txt_ddb_sanity_module_based_subscribe_listener_to_follower" 00:33:32 reference somewhere inside) 00:33:32 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_ddb_sanity_module_based_subscribe_listener_to_follower&order=bug_status" 00:33:32 00:33:32 No leader found. 00:33:32 ------------------------------------------------------------------------------ 00:33:32 Make_Leader_Local :: Make the loeader local and verify. | FAIL | 00:33:32 ... click for list of related bugs or create a new one if needed (with the 00:33:32 "controller_clustering_tell_txt_ddb_sanity_module_based_make_leader_local" 00:33:32 reference somewhere inside) 00:33:32 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_ddb_sanity_module_based_make_leader_local&order=bug_status" 00:33:32 00:33:32 No leader found. 00:33:32 ------------------------------------------------------------------------------ 00:33:32 Remove_Leader_Shard_Replica_And_Add_It_Back :: Remove and add shar... | FAIL | 00:33:32 ... click for list of related bugs or create a new one if needed (with the 00:33:32 "controller_clustering_tell_txt_ddb_sanity_module_based_remove_leader_shard_replica_and_add_it_back" 00:33:32 reference somewhere inside) 00:33:32 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_ddb_sanity_module_based_remove_leader_shard_replica_and_add_it_back&order=bug_status" 00:33:32 00:33:32 No leader found. 00:33:32 ------------------------------------------------------------------------------ 00:33:32 Remove_Follower_Shard_Replica_And_Add_It_Back :: Remove and add sh... | FAIL | 00:33:32 ... click for list of related bugs or create a new one if needed (with the 00:33:32 "controller_clustering_tell_txt_ddb_sanity_module_based_remove_follower_shard_replica_and_add_it_back" 00:33:32 reference somewhere inside) 00:33:32 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_ddb_sanity_module_based_remove_follower_shard_replica_and_add_it_back&order=bug_status" 00:33:32 00:33:32 No leader found. 00:33:32 ------------------------------------------------------------------------------ 00:33:32 Write_Transactions_One_Node_Leader :: Write transactions. | FAIL | 00:33:32 ... click for list of related bugs or create a new one if needed (with the 00:33:32 "controller_clustering_tell_txt_ddb_sanity_module_based_write_transactions_one_node_leader" 00:33:32 reference somewhere inside) 00:33:32 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_ddb_sanity_module_based_write_transactions_one_node_leader&order=bug_status" 00:33:32 00:33:32 No leader found. 00:33:32 ------------------------------------------------------------------------------ 00:33:32 Write_Transactions_One_Node_Follower :: Write transactions. | FAIL | 00:33:32 ... click for list of related bugs or create a new one if needed (with the 00:33:32 "controller_clustering_tell_txt_ddb_sanity_module_based_write_transactions_one_node_follower" 00:33:32 reference somewhere inside) 00:33:32 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_ddb_sanity_module_based_write_transactions_one_node_follower&order=bug_status" 00:33:32 00:33:32 No leader found. 00:33:32 ------------------------------------------------------------------------------ 00:33:32 Write_Transactions_All_Nodes :: Write transactions. | PASS | 00:33:32 ------------------------------------------------------------------------------ 00:33:32 controller-clustering-tell.txt.Ddb-Sanity-Module-Based :: DOMDataB... | FAIL | 00:33:32 9 tests, 2 passed, 7 failed 00:33:32 ============================================================================== 00:33:32 controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell... 00:33:32 ============================================================================== 00:33:32 Stop_All_Members :: Stop every odl node. | PASS | 00:33:32 ------------------------------------------------------------------------------ 00:33:32 Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS | 00:33:32 ------------------------------------------------------------------------------ 00:33:32 Start_All_And_Sync :: Start each member and wait for sync. | FAIL | 00:38:34 Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 5 minutes. The last error was: No leader found. 00:38:34 ------------------------------------------------------------------------------ 00:38:34 controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL | 00:38:34 3 tests, 2 passed, 1 failed 00:38:34 ============================================================================== 00:38:34 controller-clustering-tell.txt.Clean Leader Shutdown :: DOMDataBroker testi... 00:38:34 ============================================================================== 00:38:34 Local_Leader_Shutdown :: Shutdown the leader on the same node as t... | FAIL | 00:40:35 ... click for list of related bugs or create a new one if needed (with the 00:40:35 "controller_clustering_tell_txt_clean_leader_shutdown_local_leader_shutdown" 00:40:35 reference somewhere inside) 00:40:35 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_clean_leader_shutdown_local_leader_shutdown&order=bug_status" 00:40:35 00:40:35 No leader found. 00:40:35 ------------------------------------------------------------------------------ 00:40:35 Restart :: Restart odl | FAIL | 00:45:36 ... click for list of related bugs or create a new one if needed (with the 00:45:36 "controller_clustering_tell_txt_clean_leader_shutdown_restart" 00:45:36 reference somewhere inside) 00:45:36 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_clean_leader_shutdown_restart&order=bug_status" 00:45:36 00:45:36 Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 5 minutes. The last error was: No leader found. 00:45:36 ------------------------------------------------------------------------------ 00:45:36 Remote_Leader_Shutdown :: Shutdown the leader on different node as... | FAIL | 00:47:37 ... click for list of related bugs or create a new one if needed (with the 00:47:37 "controller_clustering_tell_txt_clean_leader_shutdown_remote_leader_shutdown" 00:47:37 reference somewhere inside) 00:47:37 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_clean_leader_shutdown_remote_leader_shutdown&order=bug_status" 00:47:37 00:47:37 No leader found. 00:47:37 ------------------------------------------------------------------------------ 00:47:37 controller-clustering-tell.txt.Clean Leader Shutdown :: DOMDataBro... | FAIL | 00:47:37 3 tests, 0 passed, 3 failed 00:47:37 ============================================================================== 00:47:37 controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell... 00:47:37 ============================================================================== 00:47:37 Stop_All_Members :: Stop every odl node. | PASS | 00:47:37 ------------------------------------------------------------------------------ 00:47:37 Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS | 00:47:37 ------------------------------------------------------------------------------ 00:47:37 Start_All_And_Sync :: Start each member and wait for sync. | FAIL | 00:52:39 Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 5 minutes. The last error was: No leader found. 00:52:39 ------------------------------------------------------------------------------ 00:52:39 controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL | 00:52:39 3 tests, 2 passed, 1 failed 00:52:39 ============================================================================== 00:52:39 controller-clustering-tell.txt.Explicit Leader Movement :: DOMDataBroker te... 00:52:39 ============================================================================== 00:52:39 Local_To_Remote_Movement :: Leader moves from local to remote node... | FAIL | 00:52:39 ... click for list of related bugs or create a new one if needed (with the 00:52:39 "controller_clustering_tell_txt_explicit_leader_movement_local_to_remote_movement" 00:52:39 reference somewhere inside) 00:52:39 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_explicit_leader_movement_local_to_remote_movement&order=bug_status" 00:52:39 00:52:39 No leader found. 00:52:39 ------------------------------------------------------------------------------ 00:52:39 Restart1 :: Restart odl | FAIL | 00:57:40 ... click for list of related bugs or create a new one if needed (with the 00:57:40 "controller_clustering_tell_txt_explicit_leader_movement_restart1" 00:57:40 reference somewhere inside) 00:57:40 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_explicit_leader_movement_restart1&order=bug_status" 00:57:40 00:57:40 Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 5 minutes. The last error was: No leader found. 00:57:40 ------------------------------------------------------------------------------ 00:57:40 Remote_To_Remote_Movement :: Leader moves from one remote to other... | FAIL | 00:57:41 ... click for list of related bugs or create a new one if needed (with the 00:57:41 "controller_clustering_tell_txt_explicit_leader_movement_remote_to_remote_movement" 00:57:41 reference somewhere inside) 00:57:41 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_explicit_leader_movement_remote_to_remote_movement&order=bug_status" 00:57:41 00:57:41 No leader found. 00:57:41 ------------------------------------------------------------------------------ 00:57:41 Restart2 :: Restart odl | FAIL | 01:02:42 ... click for list of related bugs or create a new one if needed (with the 01:02:42 "controller_clustering_tell_txt_explicit_leader_movement_restart2" 01:02:42 reference somewhere inside) 01:02:42 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_explicit_leader_movement_restart2&order=bug_status" 01:02:42 01:02:42 Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 5 minutes. The last error was: No leader found. 01:02:42 ------------------------------------------------------------------------------ 01:02:42 Remote_To_Local_Movement :: Leader moves from remote to local node... | FAIL | 01:02:42 ... click for list of related bugs or create a new one if needed (with the 01:02:42 "controller_clustering_tell_txt_explicit_leader_movement_remote_to_local_movement" 01:02:42 reference somewhere inside) 01:02:42 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_explicit_leader_movement_remote_to_local_movement&order=bug_status" 01:02:42 01:02:42 No leader found. 01:02:42 ------------------------------------------------------------------------------ 01:02:42 controller-clustering-tell.txt.Explicit Leader Movement :: DOMData... | FAIL | 01:02:42 5 tests, 0 passed, 5 failed 01:02:42 ============================================================================== 01:02:42 controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell... 01:02:42 ============================================================================== 01:02:42 Stop_All_Members :: Stop every odl node. | PASS | 01:02:42 ------------------------------------------------------------------------------ 01:02:42 Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS | 01:02:43 ------------------------------------------------------------------------------ 01:02:43 Start_All_And_Sync :: Start each member and wait for sync. | FAIL | 01:07:44 Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 5 minutes. The last error was: No leader found. 01:07:44 ------------------------------------------------------------------------------ 01:07:44 controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL | 01:07:44 3 tests, 2 passed, 1 failed 01:07:44 ============================================================================== 01:07:44 controller-clustering-tell.txt.Leader Isolation :: DOMDataBroker testing: L... 01:07:44 ============================================================================== 01:07:44 Healing_Within_Request_Timeout :: The isolated node (leader) is re... | FAIL | 01:07:44 ... click for list of related bugs or create a new one if needed (with the 01:07:44 "controller_clustering_tell_txt_leader_isolation_healing_within_request_timeout" 01:07:44 reference somewhere inside) 01:07:44 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_leader_isolation_healing_within_request_timeout&order=bug_status" 01:07:44 01:07:44 No leader found. 01:07:44 ------------------------------------------------------------------------------ 01:07:44 Restart :: Restart odl | FAIL | 01:12:46 ... click for list of related bugs or create a new one if needed (with the 01:12:46 "controller_clustering_tell_txt_leader_isolation_restart" 01:12:46 reference somewhere inside) 01:12:46 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_leader_isolation_restart&order=bug_status" 01:12:46 01:12:46 Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 5 minutes. The last error was: No leader found. 01:12:46 ------------------------------------------------------------------------------ 01:12:46 Healing_After_Request_Timeout :: The isolated node (leader) is rej... | FAIL | 01:12:46 ... click for list of related bugs or create a new one if needed (with the 01:12:46 "controller_clustering_tell_txt_leader_isolation_healing_after_request_timeout" 01:12:46 reference somewhere inside) 01:12:46 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_leader_isolation_healing_after_request_timeout&order=bug_status" 01:12:46 01:12:46 No leader found. 01:12:46 ------------------------------------------------------------------------------ 01:12:46 controller-clustering-tell.txt.Leader Isolation :: DOMDataBroker t... | FAIL | 01:12:46 3 tests, 0 passed, 3 failed 01:12:46 ============================================================================== 01:12:46 controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell... 01:12:46 ============================================================================== 01:12:46 Stop_All_Members :: Stop every odl node. | PASS | 01:12:46 ------------------------------------------------------------------------------ 01:12:46 Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS | 01:12:46 ------------------------------------------------------------------------------ 01:12:46 Start_All_And_Sync :: Start each member and wait for sync. | FAIL | 01:17:48 Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 5 minutes. The last error was: No leader found. 01:17:48 ------------------------------------------------------------------------------ 01:17:48 controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL | 01:17:48 3 tests, 2 passed, 1 failed 01:17:48 ============================================================================== 01:17:48 controller-clustering-tell.txt.Client Isolation :: DOMDataBroker testing: C... 01:17:48 ============================================================================== 01:17:48 Producer_On_Shard_Leader_Node_ChainedTx :: Client isolation with p... | FAIL | 01:18:59 ... click for list of related bugs or create a new one if needed (with the 01:18:59 "controller_clustering_tell_txt_client_isolation_producer_on_shard_leader_node_chainedtx" 01:18:59 reference somewhere inside) 01:18:59 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_client_isolation_producer_on_shard_leader_node_chainedtx&order=bug_status" 01:18:59 01:18:59 No leader found. 01:18:59 01:18:59 Also keyword teardown failed: 01:18:59 Several failures occurred: 01:18:59 01:18:59 1) Variable '${client_node_dst}' not found. 01:18:59 01:18:59 2) Variable '${client_node_dst}' not found. 01:18:59 01:18:59 3) Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 1 minute 10 seconds. The last error was: Several failures occurred: 01:18:59 01:18:59 1) No leader found. 01:18:59 01:18:59 2) Too many Leaders. 01:18:59 01:18:59 3) IndexError: Given index 0 is out of the range 0--1. 01:18:59 01:18:59 4) No leader found. 01:18:59 01:18:59 [ Message content over the limit has been removed. ] 01:18:59 01:18:59 10) No leader found. 01:18:59 01:18:59 11) Too many Leaders. 01:18:59 01:18:59 12) IndexError: Given index 0 is out of the range 0--1. 01:18:59 01:18:59 13) No leader found. 01:18:59 01:18:59 14) Too many Leaders. 01:18:59 01:18:59 15) IndexError: Given index 0 is out of the range 0--1. 01:18:59 01:18:59 16) No leader found. 01:18:59 01:18:59 17) Too many Leaders. 01:18:59 01:18:59 18) IndexError: Given index 0 is out of the range 0--1. 01:18:59 01:18:59 4) KeyError: 'threads' 01:18:59 ------------------------------------------------------------------------------ 01:18:59 Restart1 :: Restart odl | FAIL | 01:24:01 ... click for list of related bugs or create a new one if needed (with the 01:24:01 "controller_clustering_tell_txt_client_isolation_restart1" 01:24:01 reference somewhere inside) 01:24:01 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_client_isolation_restart1&order=bug_status" 01:24:01 01:24:01 Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 5 minutes. The last error was: No leader found. 01:24:01 ------------------------------------------------------------------------------ 01:24:01 Producer_On_Shard_Leader_Node_SimpleTx :: Client isolation with pr... | FAIL | 01:25:12 ... click for list of related bugs or create a new one if needed (with the 01:25:12 "controller_clustering_tell_txt_client_isolation_producer_on_shard_leader_node_simpletx" 01:25:12 reference somewhere inside) 01:25:12 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_client_isolation_producer_on_shard_leader_node_simpletx&order=bug_status" 01:25:12 01:25:12 No leader found. 01:25:12 01:25:12 Also keyword teardown failed: 01:25:12 Several failures occurred: 01:25:12 01:25:12 1) Variable '${client_node_dst}' not found. 01:25:12 01:25:12 2) Variable '${client_node_dst}' not found. 01:25:12 01:25:12 3) Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 1 minute 10 seconds. The last error was: Several failures occurred: 01:25:12 01:25:12 1) No leader found. 01:25:12 01:25:12 2) Too many Leaders. 01:25:12 01:25:12 3) IndexError: Given index 0 is out of the range 0--1. 01:25:12 01:25:12 4) No leader found. 01:25:12 01:25:12 [ Message content over the limit has been removed. ] 01:25:12 01:25:12 10) No leader found. 01:25:12 01:25:12 11) Too many Leaders. 01:25:12 01:25:12 12) IndexError: Given index 0 is out of the range 0--1. 01:25:12 01:25:12 13) No leader found. 01:25:12 01:25:12 14) Too many Leaders. 01:25:12 01:25:12 15) IndexError: Given index 0 is out of the range 0--1. 01:25:12 01:25:12 16) No leader found. 01:25:12 01:25:12 17) Too many Leaders. 01:25:12 01:25:12 18) IndexError: Given index 0 is out of the range 0--1. 01:25:12 01:25:12 4) KeyError: 'threads' 01:25:12 ------------------------------------------------------------------------------ 01:25:12 Restart2 :: Restart odl | FAIL | 01:30:14 ... click for list of related bugs or create a new one if needed (with the 01:30:14 "controller_clustering_tell_txt_client_isolation_restart2" 01:30:14 reference somewhere inside) 01:30:14 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_client_isolation_restart2&order=bug_status" 01:30:14 01:30:14 Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 5 minutes. The last error was: No leader found. 01:30:14 ------------------------------------------------------------------------------ 01:30:14 Producer_On_Shard_Non_Leader_Node_ChainedTx :: Client isolation wi... | FAIL | 01:31:25 ... click for list of related bugs or create a new one if needed (with the 01:31:25 "controller_clustering_tell_txt_client_isolation_producer_on_shard_non_leader_node_chainedtx" 01:31:25 reference somewhere inside) 01:31:25 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_client_isolation_producer_on_shard_non_leader_node_chainedtx&order=bug_status" 01:31:25 01:31:25 No leader found. 01:31:25 01:31:25 Also keyword teardown failed: 01:31:25 Several failures occurred: 01:31:25 01:31:25 1) Variable '${client_node_dst}' not found. 01:31:25 01:31:25 2) Variable '${client_node_dst}' not found. 01:31:25 01:31:25 3) Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 1 minute 10 seconds. The last error was: Several failures occurred: 01:31:25 01:31:25 1) No leader found. 01:31:25 01:31:25 2) Too many Leaders. 01:31:25 01:31:25 3) IndexError: Given index 0 is out of the range 0--1. 01:31:25 01:31:25 4) No leader found. 01:31:25 01:31:25 [ Message content over the limit has been removed. ] 01:31:25 01:31:25 10) No leader found. 01:31:25 01:31:25 11) Too many Leaders. 01:31:25 01:31:25 12) IndexError: Given index 0 is out of the range 0--1. 01:31:25 01:31:25 13) No leader found. 01:31:25 01:31:25 14) Too many Leaders. 01:31:25 01:31:25 15) IndexError: Given index 0 is out of the range 0--1. 01:31:25 01:31:25 16) No leader found. 01:31:25 01:31:25 17) Too many Leaders. 01:31:25 01:31:25 18) IndexError: Given index 0 is out of the range 0--1. 01:31:25 01:31:25 4) KeyError: 'threads' 01:31:25 ------------------------------------------------------------------------------ 01:31:25 Restart3 :: Restart odl | FAIL | 01:36:27 ... click for list of related bugs or create a new one if needed (with the 01:36:27 "controller_clustering_tell_txt_client_isolation_restart3" 01:36:27 reference somewhere inside) 01:36:27 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_client_isolation_restart3&order=bug_status" 01:36:27 01:36:27 Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 5 minutes. The last error was: No leader found. 01:36:27 ------------------------------------------------------------------------------ 01:36:27 Producer_On_Shard_Non_Leader_Node_SimpleTx :: Client isolation wit... | FAIL | 01:37:38 ... click for list of related bugs or create a new one if needed (with the 01:37:38 "controller_clustering_tell_txt_client_isolation_producer_on_shard_non_leader_node_simpletx" 01:37:38 reference somewhere inside) 01:37:38 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_client_isolation_producer_on_shard_non_leader_node_simpletx&order=bug_status" 01:37:38 01:37:38 No leader found. 01:37:38 01:37:38 Also keyword teardown failed: 01:37:38 Several failures occurred: 01:37:38 01:37:38 1) Variable '${client_node_dst}' not found. 01:37:38 01:37:38 2) Variable '${client_node_dst}' not found. 01:37:38 01:37:38 3) Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 1 minute 10 seconds. The last error was: Several failures occurred: 01:37:38 01:37:38 1) No leader found. 01:37:38 01:37:38 2) Too many Leaders. 01:37:38 01:37:38 3) IndexError: Given index 0 is out of the range 0--1. 01:37:38 01:37:38 4) No leader found. 01:37:38 01:37:38 [ Message content over the limit has been removed. ] 01:37:38 01:37:38 10) No leader found. 01:37:38 01:37:38 11) Too many Leaders. 01:37:38 01:37:38 12) IndexError: Given index 0 is out of the range 0--1. 01:37:38 01:37:38 13) No leader found. 01:37:38 01:37:38 14) Too many Leaders. 01:37:38 01:37:38 15) IndexError: Given index 0 is out of the range 0--1. 01:37:38 01:37:38 16) No leader found. 01:37:38 01:37:38 17) Too many Leaders. 01:37:38 01:37:38 18) IndexError: Given index 0 is out of the range 0--1. 01:37:38 01:37:38 4) KeyError: 'threads' 01:37:38 ------------------------------------------------------------------------------ 01:37:38 controller-clustering-tell.txt.Client Isolation :: DOMDataBroker t... | FAIL | 01:37:38 7 tests, 0 passed, 7 failed 01:37:38 ============================================================================== 01:37:38 controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell... 01:37:38 ============================================================================== 01:37:38 Stop_All_Members :: Stop every odl node. | PASS | 01:37:38 ------------------------------------------------------------------------------ 01:37:38 Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS | 01:37:38 ------------------------------------------------------------------------------ 01:37:38 Start_All_And_Sync :: Start each member and wait for sync. | FAIL | 01:42:40 Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 5 minutes. The last error was: No leader found. 01:42:40 ------------------------------------------------------------------------------ 01:42:40 controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL | 01:42:40 3 tests, 2 passed, 1 failed 01:42:40 ============================================================================== 01:42:40 controller-clustering-tell.txt.Listener Stability :: DOMDataBroker testing:... 01:42:40 ============================================================================== 01:42:40 Move_Leader_From_Listener_Local_To_Remote :: Listener runs on lead... | FAIL | 01:42:40 ... click for list of related bugs or create a new one if needed (with the 01:42:40 "controller_clustering_tell_txt_listener_stability_move_leader_from_listener_local_to_remote" 01:42:40 reference somewhere inside) 01:42:40 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_listener_stability_move_leader_from_listener_local_to_remote&order=bug_status" 01:42:40 01:42:40 No leader found. 01:42:40 ------------------------------------------------------------------------------ 01:42:40 Restart_1 :: Restart odl. | FAIL | 01:47:42 ... click for list of related bugs or create a new one if needed (with the 01:47:42 "controller_clustering_tell_txt_listener_stability_restart_1" 01:47:42 reference somewhere inside) 01:47:42 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_listener_stability_restart_1&order=bug_status" 01:47:42 01:47:42 Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 5 minutes. The last error was: No leader found. 01:47:42 ------------------------------------------------------------------------------ 01:47:42 Move_Leader_From_Listener_Remote_To_Other_Remote :: Listener runs ... | FAIL | 01:47:42 ... click for list of related bugs or create a new one if needed (with the 01:47:42 "controller_clustering_tell_txt_listener_stability_move_leader_from_listener_remote_to_other_remote" 01:47:42 reference somewhere inside) 01:47:42 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_listener_stability_move_leader_from_listener_remote_to_other_remote&order=bug_status" 01:47:42 01:47:42 No leader found. 01:47:42 ------------------------------------------------------------------------------ 01:47:42 Restart_2 :: Restart odl. | FAIL | 01:52:43 ... click for list of related bugs or create a new one if needed (with the 01:52:43 "controller_clustering_tell_txt_listener_stability_restart_2" 01:52:43 reference somewhere inside) 01:52:43 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_listener_stability_restart_2&order=bug_status" 01:52:43 01:52:43 Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 5 minutes. The last error was: No leader found. 01:52:43 ------------------------------------------------------------------------------ 01:52:43 Move_Leader_From_Listener_Remote_To_Local :: Listener runs on foll... | FAIL | 01:52:43 ... click for list of related bugs or create a new one if needed (with the 01:52:43 "controller_clustering_tell_txt_listener_stability_move_leader_from_listener_remote_to_local" 01:52:43 reference somewhere inside) 01:52:43 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_listener_stability_move_leader_from_listener_remote_to_local&order=bug_status" 01:52:43 01:52:43 No leader found. 01:52:43 ------------------------------------------------------------------------------ 01:52:43 controller-clustering-tell.txt.Listener Stability :: DOMDataBroker... | FAIL | 01:52:43 5 tests, 0 passed, 5 failed 01:52:43 ============================================================================== 01:52:43 controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell... 01:52:43 ============================================================================== 01:52:43 Stop_All_Members :: Stop every odl node. | PASS | 01:52:44 ------------------------------------------------------------------------------ 01:52:44 Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS | 01:52:44 ------------------------------------------------------------------------------ 01:52:44 Start_All_And_Sync :: Start each member and wait for sync. | FAIL | 01:57:45 Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 5 minutes. The last error was: No leader found. 01:57:45 ------------------------------------------------------------------------------ 01:57:45 controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL | 01:57:45 3 tests, 2 passed, 1 failed 01:57:45 ============================================================================== 01:57:45 controller-clustering-tell.txt.Master Stability :: Cluster Singleton testin... 01:57:45 ============================================================================== 01:57:45 Register_Singleton_Constant_On_Each_Node_And_Verify :: Register a ... | FAIL | 01:57:45 ... click for list of related bugs or create a new one if needed (with the 01:57:45 "controller_clustering_tell_txt_master_stability_register_singleton_constant_on_each_node_and_verify" 01:57:45 reference somewhere inside) 01:57:45 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_master_stability_register_singleton_constant_on_each_node_and_verify&order=bug_status" 01:57:45 01:57:45 Could not parse owner and candidates for device get-singleton-constant-service'] 01:57:45 ------------------------------------------------------------------------------ 01:57:45 Unregister_Singleton_Constant_On_Non_Master_Node :: Unregister the... | FAIL | 01:57:45 ... click for list of related bugs or create a new one if needed (with the 01:57:45 "controller_clustering_tell_txt_master_stability_unregister_singleton_constant_on_non_master_node" 01:57:45 reference somewhere inside) 01:57:45 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_master_stability_unregister_singleton_constant_on_non_master_node&order=bug_status" 01:57:45 01:57:45 Variable '@{cs_candidates}' not found. Did you mean: 01:57:45 @{cs_exp_candidates} 01:57:45 ------------------------------------------------------------------------------ 01:57:45 Monitor_Stability_While_Unregistered :: Verify that the owner rema... | FAIL | 01:57:45 ... click for list of related bugs or create a new one if needed (with the 01:57:45 "controller_clustering_tell_txt_master_stability_monitor_stability_while_unregistered" 01:57:45 reference somewhere inside) 01:57:45 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_master_stability_monitor_stability_while_unregistered&order=bug_status" 01:57:45 01:57:45 Variable '${cs_owner}' not found. 01:57:45 ------------------------------------------------------------------------------ 01:57:45 Reregister_Singleton_Constant :: Re-register the unregistered cand... | FAIL | 01:57:45 ... click for list of related bugs or create a new one if needed (with the 01:57:45 "controller_clustering_tell_txt_master_stability_reregister_singleton_constant" 01:57:45 reference somewhere inside) 01:57:45 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_master_stability_reregister_singleton_constant&order=bug_status" 01:57:45 01:57:45 Variable '${unregistered_node}' not found. 01:57:45 ------------------------------------------------------------------------------ 01:57:45 Verify_Stability_After_Reregistration :: Verify that the owner rem... | FAIL | 01:57:45 ... click for list of related bugs or create a new one if needed (with the 01:57:45 "controller_clustering_tell_txt_master_stability_verify_stability_after_reregistration" 01:57:45 reference somewhere inside) 01:57:45 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_master_stability_verify_stability_after_reregistration&order=bug_status" 01:57:45 01:57:45 Variable '${cs_owner}' not found. 01:57:45 ------------------------------------------------------------------------------ 01:57:45 Unregister_Singleton_Constant_On_Each_Node :: Unregister the appli... | PASS | 01:57:46 ------------------------------------------------------------------------------ 01:57:46 controller-clustering-tell.txt.Master Stability :: Cluster Singlet... | FAIL | 01:57:46 6 tests, 1 passed, 5 failed 01:57:46 ============================================================================== 01:57:46 controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell... 01:57:46 ============================================================================== 01:57:46 Stop_All_Members :: Stop every odl node. | PASS | 01:57:46 ------------------------------------------------------------------------------ 01:57:46 Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS | 01:57:46 ------------------------------------------------------------------------------ 01:57:46 Start_All_And_Sync :: Start each member and wait for sync. | FAIL | 02:02:47 Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 5 minutes. The last error was: No leader found. 02:02:47 ------------------------------------------------------------------------------ 02:02:47 controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL | 02:02:47 3 tests, 2 passed, 1 failed 02:02:47 ============================================================================== 02:02:47 controller-clustering-tell.txt.Partition And Heal :: Cluster Singleton test... 02:02:47 ============================================================================== 02:02:48 Register_Singleton_Constant_On_Each_Node :: Register a candidate a... | PASS | 02:02:48 ------------------------------------------------------------------------------ 02:02:48 Verify_Singleton_Constant_On_Each_Node :: Store the owner and cand... | FAIL | 02:02:48 ... click for list of related bugs or create a new one if needed (with the 02:02:48 "controller_clustering_tell_txt_partition_and_heal_verify_singleton_constant_on_each_node" 02:02:48 reference somewhere inside) 02:02:48 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_partition_and_heal_verify_singleton_constant_on_each_node&order=bug_status" 02:02:48 02:02:48 Could not parse owner and candidates for device get-singleton-constant-service'] 02:02:48 ------------------------------------------------------------------------------ 02:02:48 Isolate_Owner_Node :: Isolate the cluster node which is the owner.... | FAIL | 02:02:48 ... click for list of related bugs or create a new one if needed (with the 02:02:48 "controller_clustering_tell_txt_partition_and_heal_isolate_owner_node" 02:02:48 reference somewhere inside) 02:02:48 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_partition_and_heal_isolate_owner_node&order=bug_status" 02:02:48 02:02:48 Variable '${cs_owner}' not found. 02:02:48 ------------------------------------------------------------------------------ 02:02:48 Monitor_Stability_While_Isolated :: Monitor the stability of the s... | PASS | 02:04:48 ------------------------------------------------------------------------------ 02:04:48 Rejoin_Isolated_node :: Rejoin isolated node. | FAIL | 02:04:48 ... click for list of related bugs or create a new one if needed (with the 02:04:48 "controller_clustering_tell_txt_partition_and_heal_rejoin_isolated_node" 02:04:48 reference somewhere inside) 02:04:48 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_partition_and_heal_rejoin_isolated_node&order=bug_status" 02:04:48 02:04:48 Variable '${cs_isolated_index}' not found. 02:04:48 ------------------------------------------------------------------------------ 02:04:48 Unregister_Singleton_Constant_On_Each_Node :: Unregister the appli... | PASS | 02:04:48 ------------------------------------------------------------------------------ 02:04:48 controller-clustering-tell.txt.Partition And Heal :: Cluster Singl... | FAIL | 02:04:48 6 tests, 3 passed, 3 failed 02:04:48 ============================================================================== 02:04:48 controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell... 02:04:48 ============================================================================== 02:04:48 Stop_All_Members :: Stop every odl node. | PASS | 02:04:48 ------------------------------------------------------------------------------ 02:04:48 Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS | 02:04:48 ------------------------------------------------------------------------------ 02:04:48 Start_All_And_Sync :: Start each member and wait for sync. | FAIL | 02:09:50 Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 5 minutes. The last error was: No leader found. 02:09:50 ------------------------------------------------------------------------------ 02:09:50 controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL | 02:09:50 3 tests, 2 passed, 1 failed 02:09:50 ============================================================================== 02:09:50 controller-clustering-tell.txt.Chasing The Leader :: Cluster Singleton test... 02:09:50 ============================================================================== 02:09:50 Register_Candidates :: Register a candidate application on each no... | PASS | 02:09:50 ------------------------------------------------------------------------------ 02:09:50 Do_Nothing :: Do nothing for the time of the test duration, becaus... | PASS | 02:10:50 ------------------------------------------------------------------------------ 02:10:50 Unregister_Candidates_And_Validate_Criteria :: Unregister the test... | FAIL | 02:10:50 ... click for list of related bugs or create a new one if needed (with the 02:10:50 "controller_clustering_tell_txt_chasing_the_leader_unregister_candidates_and_validate_criteria" 02:10:50 reference somewhere inside) 02:10:50 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_chasing_the_leader_unregister_candidates_and_validate_criteria&order=bug_status" 02:10:50 02:10:50 Acceptance rate 5 not reached, actual rate is 0.0. 02:10:50 ------------------------------------------------------------------------------ 02:10:50 controller-clustering-tell.txt.Chasing The Leader :: Cluster Singl... | FAIL | 02:10:50 3 tests, 2 passed, 1 failed 02:10:50 ============================================================================== 02:10:50 controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell... 02:10:50 ============================================================================== 02:10:50 Stop_All_Members :: Stop every odl node. | PASS | 02:10:50 ------------------------------------------------------------------------------ 02:10:50 Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS | 02:10:50 ------------------------------------------------------------------------------ 02:10:50 Start_All_And_Sync :: Start each member and wait for sync. | FAIL | 02:15:52 Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 5 minutes. The last error was: No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL | 02:15:52 3 tests, 2 passed, 1 failed 02:15:52 ============================================================================== 02:15:52 controller-clustering-tell.txt.Carpeople Crud :: Suite for performing basic... 02:15:52 ============================================================================== 02:15:52 Add_Cars_To_Leader :: Add 30 cars to car Leader by one big PUT. | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 See_Added_Cars_On_Leader :: GET response from Leader should match ... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 See_Added_Cars_On_Followers :: The same check on other members. | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Add_People_To_First_Follower :: Add 30 people to people first Foll... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 See_Added_People_On_Leader :: GET response from Leader should matc... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 See_Added_People_On_Followers :: The same check on other members. | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Buy_Cars_On_Leader :: Buy some cars on car-people Leader, loop of ... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Buy_Cars_On_Followers :: On each Follower buy corresponding ID seg... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 See_Added_CarPeople_On_Leader :: GET car-person mappings from Lead... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 See_Added_CarPeople_On_Followers :: The same check on other members. | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Delete_All_CarPeople_On_Leader :: DELETE car-people container. No ... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Delete_All_People_On_Leader :: DELETE people container. No verific... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Delete_All_Cars_On_Leader :: DELETE cars container. No verificatio... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 controller-clustering-tell.txt.Carpeople Crud :: Suite for perform... | FAIL | 02:15:52 Suite setup failed: 02:15:52 No leader found. 02:15:52 02:15:52 13 tests, 0 passed, 13 failed 02:15:52 ============================================================================== 02:15:52 controller-clustering-tell.txt.Car Failover Crud :: Suite mixing basic oper... 02:15:52 ============================================================================== 02:15:52 Add_Original_Cars_On_Old_Leader_And_Verify :: Add initial cars on ... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Stop_Original_Car_Leader :: Stop the car Leader to cause a new lea... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Wait_For_New_Leader :: Wait until new car Leader is elected. | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 See_Original_Cars_On_New_Leader :: GET cars from new Leader, shoul... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 See_Original_Cars_On_New_Followers :: The same check on other exis... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Delete_Original_Cars_On_New_Leader :: Delete cars on the new Leader. | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Add_Leader_Cars_On_New_Leader :: Add cars on the new Leader. | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 See_Leader_Cars_On_New_Leader :: GET cars from new Leader, should ... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 See_Leader_Cars_On_New_Followers :: The same check on other existi... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Delete_Leader_Cars_On_New_First_Follower :: Delete cars in new fir... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Add_Follower_Cars_On_New_First_Follower :: Add cars on the new fir... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 See_Folower_Cars_On_New_Leader :: Get cars from the new Leader, sh... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 See_Follower_Cars_On_New_Followers :: The same check on other exis... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Start_Old_Car_Leader :: Start the stopped member without deleting ... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 See_Folower_Cars_On_Old_Leader :: GET cars from the restarted memb... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Delete_Follower_Cars_On_New_Leader :: Delete cars on the last Leader. | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 controller-clustering-tell.txt.Car Failover Crud :: Suite mixing b... | FAIL | 02:15:52 Suite setup failed: 02:15:52 No leader found. 02:15:52 02:15:52 16 tests, 0 passed, 16 failed 02:15:52 ============================================================================== 02:15:52 controller-clustering-tell.txt.Car Persistence Recovery :: This test restar... 02:15:52 ============================================================================== 02:15:52 Add_Cars_On_Leader_And_Verify :: Single big PUT to datastore to ad... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Stop_All_Members :: Stop all controllers. | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Start_All_Members :: Start all controllers (should restore the per... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Memorize_Leader_And_Followers :: Locate current Leader of car Shard. | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 See_Cars_On_Leader :: GET cars from Leader, should match the PUT d... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 See_Cars_On_Followers :: The same check on other members. | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Delete_Cars_On_Leader :: Delete cars on the new Leader. | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 controller-clustering-tell.txt.Car Persistence Recovery :: This te... | FAIL | 02:15:52 Suite setup failed: 02:15:52 No leader found. 02:15:52 02:15:52 7 tests, 0 passed, 7 failed 02:15:52 ============================================================================== 02:15:52 controller-clustering-tell.txt.Buycar Failover :: This test focuses on test... 02:15:52 ============================================================================== 02:15:52 Add_Cars_To_Leader_And_Verify :: Add all needed cars to car Leader... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Add_People_To_First_Follower_And_Verify :: Add all needed people t... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Buy_Cars_On_Leader_And_Verify :: Buy some cars on the leader member. | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Buy_Cars_On_Follower_And_Verify :: Buy some cars on the first foll... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Reboot_People_Leader :: Previous people Leader is rebooted. We sho... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Buy_Cars_On_Leader_After_Reboot_And_Verify :: Buy some cars on the... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Buy_Cars_On_Follower_After_Reboot_And_Verify :: Buy some cars on t... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Delete_All_CarPeople :: DELETE car-people container. No verificati... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Delete_All_People :: DELETE people container. No verification beyo... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 Delete_All_Cars :: DELETE cars container. No verification beyond h... | FAIL | 02:15:52 Parent suite setup failed: 02:15:52 No leader found. 02:15:52 ------------------------------------------------------------------------------ 02:15:52 controller-clustering-tell.txt.Buycar Failover :: This test focuse... | FAIL | 02:15:52 Suite setup failed: 02:15:52 No leader found. 02:15:52 02:15:52 10 tests, 0 passed, 10 failed 02:15:52 ============================================================================== 02:15:53 controller-clustering-tell.txt.Entity Isolate :: Suite for performing membe... 02:15:53 ============================================================================== 02:15:53 Check All Shards Before Isolate :: Check all shards in controller. | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Isolate Entity Leader :: Isolate the entity-ownership Leader to ca... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Check All Shards After Isolate :: Check all shards in controller. | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Rejoin Entity Leader :: Rejoin the entity-ownership Leader. | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Check All Shards After Rejoin :: Check all shards in controller. | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 controller-clustering-tell.txt.Entity Isolate :: Suite for perform... | FAIL | 02:15:53 Suite setup failed: 02:15:53 No leader found. 02:15:53 02:15:53 5 tests, 0 passed, 5 failed 02:15:53 ============================================================================== 02:15:53 controller-clustering-tell.txt.Buycar Failover Isolation :: This test focus... 02:15:53 ============================================================================== 02:15:53 Add_Cars_To_Leader_And_Verify :: Add all needed cars to car Leader... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Add_People_To_First_Follower_And_Verify :: Add all needed people t... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Buy_Cars_On_Leader_And_Verify :: Buy some cars on the leader member. | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Buy_Cars_On_Follower_And_Verify :: Buy some cars on the first foll... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Isolate_and_Rejoin_People_Leader :: Previous people Leader is isol... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Buy_Cars_On_Leader_After_Rejoin_And_Verify :: Buy some cars on the... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Buy_Cars_On_Follower_After_Rejoin_And_Verify :: Buy some cars on t... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Delete_All_CarPeople :: DELETE car-people container. No verificati... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Delete_All_People :: DELETE people container. No verification beyo... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Delete_All_Cars :: DELETE cars container. No verification beyond h... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 controller-clustering-tell.txt.Buycar Failover Isolation :: This t... | FAIL | 02:15:53 Suite setup failed: 02:15:53 No leader found. 02:15:53 02:15:53 10 tests, 0 passed, 10 failed 02:15:53 ============================================================================== 02:15:53 controller-clustering-tell.txt.Car Failover Crud Isolation :: Suite mixing ... 02:15:53 ============================================================================== 02:15:53 Add_Original_Cars_On_Old_Leader_And_Verify :: Add initial cars on ... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Isolate_Original_Car_Leader :: Isolate the car Leader to cause a n... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Wait_For_New_Leader :: Wait until new car Leader is elected. | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 See_Original_Cars_On_New_Leader :: GET cars from new Leader, shoul... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 See_Original_Cars_On_New_Followers :: The same check on other exis... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Delete_Original_Cars_On_New_Leader :: Delete cars on the new Leader. | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Add_Leader_Cars_On_New_Leader :: Add cars on the new Leader. | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 See_Leader_Cars_On_New_Leader :: GET cars from new Leader, should ... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 See_Leader_Cars_On_New_Followers :: The same check on other existi... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Delete_Leader_Cars_On_New_First_Follower :: Delete cars in new fir... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Add_Follower_Cars_On_New_First_Follower :: Add cars on the new fir... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 See_Folower_Cars_On_New_Leader :: Get cars from the new Leader, sh... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 See_Follower_Cars_On_New_Followers :: The same check on other exis... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Rejoin_Old_Car_Leader :: Rejoin the isolated member without deleti... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 See_Folower_Cars_On_Old_Leader :: GET cars from the restarted memb... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Delete_Follower_Cars_On_New_Leader :: Delete cars on the last Leader. | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 controller-clustering-tell.txt.Car Failover Crud Isolation :: Suit... | FAIL | 02:15:53 Suite setup failed: 02:15:53 No leader found. 02:15:53 02:15:53 16 tests, 0 passed, 16 failed 02:15:53 ============================================================================== 02:15:53 controller-clustering-tell.txt.Car Outage Corners :: Cluster suite for test... 02:15:53 ============================================================================== 02:15:53 Stop_Majority_Of_The_Followers :: Stop half plus one car Follower ... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Attempt_To_Add_Cars_To_Leader :: Adding cars should fail, as major... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Start_Tipping_Follower :: Start one Follower member without persis... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Add_Cars_On_Tipping_Follower :: Add cars on the tipping Follower. | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 See_Cars_On_Existing_Members :: On each up member: GET cars, shoul... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Start_Other_Followers :: Start other followers without persisted d... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 See_Cars_On_New_Follower_Leader :: GET cars from a new follower to... | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 Delete_Cars_On_Leader :: Delete cars on Leader. | FAIL | 02:15:53 Parent suite setup failed: 02:15:53 No leader found. 02:15:53 ------------------------------------------------------------------------------ 02:15:53 controller-clustering-tell.txt.Car Outage Corners :: Cluster suite... | FAIL | 02:15:53 Suite setup failed: 02:15:53 No leader found. 02:15:53 02:15:53 8 tests, 0 passed, 8 failed 02:15:53 ============================================================================== 02:15:53 controller-clustering-tell.txt | FAIL | 02:15:53 162 tests, 28 passed, 134 failed 02:15:53 ============================================================================== 02:15:53 Output: /w/workspace/controller-csit-3node-clustering-tell-all-titanium/output.xml 02:15:55 Log: /w/workspace/controller-csit-3node-clustering-tell-all-titanium/log.html 02:15:55 Report: /w/workspace/controller-csit-3node-clustering-tell-all-titanium/report.html 02:15:55 + true 02:15:55 + echo 'Examining the files in data/log and checking filesize' 02:15:55 Examining the files in data/log and checking filesize 02:15:55 + ssh '' 'ls -altr /tmp/karaf-0.22.3-SNAPSHOT/data/log/' 02:15:55 ssh: Could not resolve hostname : Device or resource busy 02:15:55 Build step 'Execute shell' marked build as failure 02:15:55 $ ssh-agent -k 02:15:55 unset SSH_AUTH_SOCK; 02:15:55 unset SSH_AGENT_PID; 02:15:55 echo Agent pid 3168 killed; 02:15:55 [ssh-agent] Stopped. 02:15:55 Recording plot data 02:15:56 Robot results publisher started... 02:15:56 INFO: Checking test criticality is deprecated and will be dropped in a future release! 02:15:56 -Parsing output xml: 02:15:56 Done! 02:15:56 -Copying log files to build dir: 02:15:57 Done! 02:15:57 -Assigning results to build: 02:15:57 Done! 02:15:57 -Checking thresholds: 02:15:57 Done! 02:15:57 Done publishing Robot results. 02:15:57 [PostBuildScript] - [INFO] Executing post build scripts. 02:15:57 [controller-csit-3node-clustering-tell-all-titanium] $ /bin/bash /tmp/jenkins4627374750098369378.sh 02:15:57 Archiving csit artifacts 02:15:57 % Total % Received % Xferd Average Speed Time Time Time Current 02:15:57 Dload Upload Total Spent Left Speed 02:15:57 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 13916 0 13916 0 0 52943 0 --:--:-- --:--:-- --:--:-- 52912 100 1116k 0 1116k 0 0 2403k 0 --:--:-- --:--:-- --:--:-- 2400k 02:15:57 Archive: robot-plugin.zip 02:15:57 inflating: ./archives/robot-plugin/log.html 02:15:57 inflating: ./archives/robot-plugin/output.xml 02:15:58 inflating: ./archives/robot-plugin/report.html 02:15:58 mv: cannot stat '*.log': No such file or directory 02:15:58 mv: cannot stat '*.log.gz': No such file or directory 02:15:58 mv: cannot stat '*.csv': No such file or directory 02:15:58 mv: cannot stat '*.png': No such file or directory 02:15:58 [PostBuildScript] - [INFO] Executing post build scripts. 02:15:58 [controller-csit-3node-clustering-tell-all-titanium] $ /bin/bash /tmp/jenkins5659920082668968328.sh 02:15:58 [PostBuildScript] - [INFO] Executing post build scripts. 02:15:58 [EnvInject] - Injecting environment variables from a build step. 02:15:58 [EnvInject] - Injecting as environment variables the properties content 02:15:58 OS_CLOUD=vex 02:15:58 OS_STACK_NAME=releng-controller-csit-3node-clustering-tell-all-titanium-96 02:15:58 02:15:58 [EnvInject] - Variables injected successfully. 02:15:58 provisioning config files... 02:15:58 copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml 02:15:58 [controller-csit-3node-clustering-tell-all-titanium] $ /bin/bash /tmp/jenkins1518211422451148750.sh 02:15:58 ---> openstack-stack-delete.sh 02:15:58 Setup pyenv: 02:15:58 system 02:15:58 3.8.20 02:15:58 3.9.20 02:15:58 3.10.15 02:15:58 * 3.11.10 (set by /w/workspace/controller-csit-3node-clustering-tell-all-titanium/.python-version) 02:15:58 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-1T4P from file:/tmp/.os_lf_venv 02:15:58 lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv) 02:15:58 lf-activate-venv(): INFO: Attempting to install with network-safe options... 02:16:00 lf-activate-venv(): INFO: Base packages installed successfully 02:16:00 lf-activate-venv(): INFO: Installing additional packages: lftools[openstack] kubernetes python-heatclient python-openstackclient urllib3~=1.26.15 02:16:23 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 02:16:23 python-manilaclient 6.2.0 requires osc-lib>=3.2.0, but you have osc-lib 2.2.1 which is incompatible. 02:16:24 lf-activate-venv(): INFO: Adding /tmp/venv-1T4P/bin to PATH 02:16:24 INFO: Stack cost retrieval disabled, setting cost to 0 02:16:36 INFO: Deleting stack releng-controller-csit-3node-clustering-tell-all-titanium-96 02:16:36 Successfully deleted stack releng-controller-csit-3node-clustering-tell-all-titanium-96 02:16:36 [PostBuildScript] - [INFO] Executing post build scripts. 02:16:36 [controller-csit-3node-clustering-tell-all-titanium] $ /bin/bash /tmp/jenkins1823349471840155654.sh 02:16:36 ---> sysstat.sh 02:16:36 /tmp/jenkins1823349471840155654.sh: line 19: facter: command not found 02:16:36 [controller-csit-3node-clustering-tell-all-titanium] $ /bin/bash /tmp/jenkins18017262534917433579.sh 02:16:36 ---> package-listing.sh 02:16:36 ++ facter osfamily 02:16:36 /tmp/jenkins18017262534917433579.sh: line 19: facter: command not found 02:16:36 ++ tr '[:upper:]' '[:lower:]' 02:16:36 + OS_FAMILY= 02:16:36 [controller-csit-3node-clustering-tell-all-titanium] $ /bin/bash /tmp/jenkins3228594575118818533.sh 02:16:36 ---> capture-instance-metadata.sh 02:16:36 Setup pyenv: 02:16:36 system 02:16:36 3.8.20 02:16:36 3.9.20 02:16:36 3.10.15 02:16:36 * 3.11.10 (set by /w/workspace/controller-csit-3node-clustering-tell-all-titanium/.python-version) 02:16:36 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-1T4P from file:/tmp/.os_lf_venv 02:16:36 lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv) 02:16:36 lf-activate-venv(): INFO: Attempting to install with network-safe options... 02:16:38 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 02:16:38 python-manilaclient 6.2.0 requires osc-lib>=3.2.0, but you have osc-lib 2.2.1 which is incompatible. 02:16:38 lf-activate-venv(): INFO: Base packages installed successfully 02:16:38 lf-activate-venv(): INFO: Installing additional packages: lftools 02:16:47 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 02:16:47 python-manilaclient 6.2.0 requires osc-lib>=3.2.0, but you have osc-lib 2.2.1 which is incompatible. 02:16:48 lf-activate-venv(): INFO: Adding /tmp/venv-1T4P/bin to PATH 02:16:48 INFO: Running in OpenStack, capturing instance metadata 02:16:48 [controller-csit-3node-clustering-tell-all-titanium] $ /bin/bash /tmp/jenkins2492357651775212338.sh 02:16:48 provisioning config files... 02:16:49 Could not find credentials [logs] for controller-csit-3node-clustering-tell-all-titanium #96 02:16:49 copy managed file [jenkins-log-archives-settings] to file:/w/workspace/controller-csit-3node-clustering-tell-all-titanium@tmp/config1489376659258270361tmp 02:16:49 Regular expression run condition: Expression=[^.*logs-s3.*], Label=[odl-logs-s3-cloudfront-index] 02:16:49 Run condition [Regular expression match] enabling perform for step [Provide Configuration files] 02:16:49 provisioning config files... 02:16:49 copy managed file [jenkins-s3-log-ship] to file:/home/jenkins/.aws/credentials 02:16:49 [EnvInject] - Injecting environment variables from a build step. 02:16:49 [EnvInject] - Injecting as environment variables the properties content 02:16:49 SERVER_ID=logs 02:16:49 02:16:49 [EnvInject] - Variables injected successfully. 02:16:49 [controller-csit-3node-clustering-tell-all-titanium] $ /bin/bash /tmp/jenkins1859766528858240910.sh 02:16:49 ---> create-netrc.sh 02:16:49 WARN: Log server credential not found. 02:16:49 [controller-csit-3node-clustering-tell-all-titanium] $ /bin/bash /tmp/jenkins17232409535145065520.sh 02:16:49 ---> python-tools-install.sh 02:16:49 Setup pyenv: 02:16:49 system 02:16:49 3.8.20 02:16:49 3.9.20 02:16:49 3.10.15 02:16:49 * 3.11.10 (set by /w/workspace/controller-csit-3node-clustering-tell-all-titanium/.python-version) 02:16:49 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-1T4P from file:/tmp/.os_lf_venv 02:16:49 lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv) 02:16:49 lf-activate-venv(): INFO: Attempting to install with network-safe options... 02:16:51 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 02:16:51 python-manilaclient 6.2.0 requires osc-lib>=3.2.0, but you have osc-lib 2.2.1 which is incompatible. 02:16:51 lf-activate-venv(): INFO: Base packages installed successfully 02:16:51 lf-activate-venv(): INFO: Installing additional packages: lftools 02:17:01 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 02:17:01 python-manilaclient 6.2.0 requires osc-lib>=3.2.0, but you have osc-lib 2.2.1 which is incompatible. 02:17:01 lf-activate-venv(): INFO: Adding /tmp/venv-1T4P/bin to PATH 02:17:01 [controller-csit-3node-clustering-tell-all-titanium] $ /bin/sh /tmp/jenkins10375457879722384519.sh 02:17:01 ---> uv-install.sh 02:17:01 uv 0.11.32 is already installed 02:17:01 uvx 0.11.32 (x86_64-unknown-linux-gnu) 02:17:01 [controller-csit-3node-clustering-tell-all-titanium] $ /bin/bash /tmp/jenkins15898341110733333.sh 02:17:01 ---> sudo-logs.sh 02:17:01 Archiving 'sudo' log.. 02:17:01 /tmp/jenkins15898341110733333.sh: line 41: facter: command not found 02:17:01 [controller-csit-3node-clustering-tell-all-titanium] $ /bin/bash /tmp/jenkins16100512725217975610.sh 02:17:01 ---> job-cost.sh 02:17:01 INFO: Activating Python virtual environment... 02:17:01 Setup pyenv: 02:17:01 system 02:17:01 3.8.20 02:17:01 3.9.20 02:17:01 3.10.15 02:17:01 * 3.11.10 (set by /w/workspace/controller-csit-3node-clustering-tell-all-titanium/.python-version) 02:17:01 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-1T4P from file:/tmp/.os_lf_venv 02:17:01 lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv) 02:17:01 lf-activate-venv(): INFO: Attempting to install with network-safe options... 02:17:03 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 02:17:03 python-manilaclient 6.2.0 requires osc-lib>=3.2.0, but you have osc-lib 2.2.1 which is incompatible. 02:17:03 lf-activate-venv(): INFO: Base packages installed successfully 02:17:03 lf-activate-venv(): INFO: Installing additional packages: zipp==1.1.0 python-openstackclient urllib3~=1.26.15 02:17:12 lf-activate-venv(): INFO: Adding /tmp/venv-1T4P/bin to PATH 02:17:12 DEBUG: Contents of stack-cost file: 02:17:12 total: 0 02:17:12 INFO: Retrieving Stack Cost... 02:17:12 INFO: Stack cost: 0 02:17:12 INFO: Instance uptime: 7179s 02:17:12 INFO: Fetching instance metadata (attempt 1 of 3)... 02:17:12 DEBUG: URL: http://169.254.169.254/latest/meta-data/instance-type 02:17:12 INFO: Successfully fetched instance metadata 02:17:12 INFO: Instance type: v3-standard-2 02:17:12 INFO: Retrieving pricing info for: v3-standard-2 02:17:12 INFO: Fetching Vexxhost pricing API (attempt 1 of 3)... 02:17:12 DEBUG: URL: https://pricing.vexxhost.net/v1/pricing/v3-standard-2/cost?seconds=7179 02:17:12 INFO: Successfully fetched Vexxhost pricing API 02:17:12 INFO: Retrieved cost: 0.11 02:17:12 INFO: Retrieved resource: v3-standard-2 02:17:12 INFO: Creating archive directory: /w/workspace/controller-csit-3node-clustering-tell-all-titanium/archives/cost 02:17:12 INFO: Archiving costs to: /w/workspace/controller-csit-3node-clustering-tell-all-titanium/archives/cost.csv 02:17:12 INFO: Successfully archived job cost data 02:17:12 DEBUG: Cost data: controller-csit-3node-clustering-tell-all-titanium,96,2026-07-26 02:17:12,v3-standard-2,7179,0.11,0.00,FAILURE 02:17:12 [controller-csit-3node-clustering-tell-all-titanium] $ /bin/bash -l /tmp/jenkins15031301727740305807.sh 02:17:12 ---> logs-deploy.sh 02:17:12 Setup pyenv: 02:17:12 system 02:17:12 3.8.20 02:17:12 3.9.20 02:17:12 3.10.15 02:17:12 * 3.11.10 (set by /w/workspace/controller-csit-3node-clustering-tell-all-titanium/.python-version) 02:17:12 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-1T4P from file:/tmp/.os_lf_venv 02:17:12 lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv) 02:17:12 lf-activate-venv(): INFO: Attempting to install with network-safe options... 02:17:14 lf-activate-venv(): INFO: Base packages installed successfully 02:17:14 lf-activate-venv(): INFO: Installing additional packages: lftools urllib3~=1.26.15 02:17:27 lf-activate-venv(): INFO: Adding /tmp/venv-1T4P/bin to PATH 02:17:27 WARNING: Nexus logging server not set 02:17:27 INFO: S3 path logs/releng/vex-yul-odl-jenkins-1/controller-csit-3node-clustering-tell-all-titanium/96/ 02:17:27 INFO: archiving logs to S3 02:17:28 ---> uname -a: 02:17:28 Linux prd-ubuntu2404-robot-2c-8g-38843 6.8.0-90-generic #91-Ubuntu SMP PREEMPT_DYNAMIC Tue Nov 18 14:14:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux 02:17:28 02:17:28 02:17:28 ---> lscpu: 02:17:28 Architecture: x86_64 02:17:28 CPU op-mode(s): 32-bit, 64-bit 02:17:28 Address sizes: 40 bits physical, 48 bits virtual 02:17:28 Byte Order: Little Endian 02:17:28 CPU(s): 2 02:17:28 On-line CPU(s) list: 0,1 02:17:28 Vendor ID: AuthenticAMD 02:17:28 Model name: AMD EPYC-Rome Processor 02:17:28 CPU family: 23 02:17:28 Model: 49 02:17:28 Thread(s) per core: 1 02:17:28 Core(s) per socket: 1 02:17:28 Socket(s): 2 02:17:28 Stepping: 0 02:17:28 BogoMIPS: 5599.99 02:17:28 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 clzero xsaveerptr wbnoinvd arat npt nrip_save umip rdpid arch_capabilities 02:17:28 Virtualization: AMD-V 02:17:28 Hypervisor vendor: KVM 02:17:28 Virtualization type: full 02:17:28 L1d cache: 64 KiB (2 instances) 02:17:28 L1i cache: 64 KiB (2 instances) 02:17:28 L2 cache: 1 MiB (2 instances) 02:17:28 L3 cache: 32 MiB (2 instances) 02:17:28 NUMA node(s): 1 02:17:28 NUMA node0 CPU(s): 0,1 02:17:28 Vulnerability Gather data sampling: Not affected 02:17:28 Vulnerability Itlb multihit: Not affected 02:17:28 Vulnerability L1tf: Not affected 02:17:28 Vulnerability Mds: Not affected 02:17:28 Vulnerability Meltdown: Not affected 02:17:28 Vulnerability Mmio stale data: Not affected 02:17:28 Vulnerability Reg file data sampling: Not affected 02:17:28 Vulnerability Retbleed: Mitigation; untrained return thunk; SMT disabled 02:17:28 Vulnerability Spec rstack overflow: Vulnerable: Safe RET, no microcode 02:17:28 Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl 02:17:28 Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization 02:17:28 Vulnerability Spectre v2: Mitigation; Retpolines; IBPB conditional; STIBP disabled; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected 02:17:28 Vulnerability Srbds: Not affected 02:17:28 Vulnerability Tsx async abort: Not affected 02:17:28 Vulnerability Vmscape: Not affected 02:17:28 02:17:28 02:17:28 ---> nproc: 02:17:28 2 02:17:28 02:17:28 02:17:28 ---> df -h: 02:17:28 Filesystem Size Used Avail Use% Mounted on 02:17:28 tmpfs 795M 1.1M 794M 1% /run 02:17:28 /dev/vda1 38G 8.2G 30G 22% / 02:17:28 tmpfs 3.9G 0 3.9G 0% /dev/shm 02:17:28 tmpfs 5.0M 0 5.0M 0% /run/lock 02:17:28 /dev/vda16 881M 117M 703M 15% /boot 02:17:28 /dev/vda15 105M 6.2M 99M 6% /boot/efi 02:17:28 tmpfs 795M 16K 795M 1% /run/user/1001 02:17:28 02:17:28 02:17:28 ---> free -m: 02:17:28 total used free shared buff/cache available 02:17:28 Mem: 7941 790 4943 4 2509 7151 02:17:28 Swap: 1023 0 1023 02:17:28 02:17:28 02:17:28 ---> ip addr: 02:17:28 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 02:17:28 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 02:17:28 inet 127.0.0.1/8 scope host lo 02:17:28 valid_lft forever preferred_lft forever 02:17:28 inet6 ::1/128 scope host noprefixroute 02:17:28 valid_lft forever preferred_lft forever 02:17:28 2: ens3: mtu 1458 qdisc mq state UP group default qlen 1000 02:17:28 link/ether fa:16:3e:21:12:77 brd ff:ff:ff:ff:ff:ff 02:17:28 altname enp0s3 02:17:28 inet 10.30.170.191/23 metric 100 brd 10.30.171.255 scope global dynamic ens3 02:17:28 valid_lft 79215sec preferred_lft 79215sec 02:17:28 inet6 fe80::f816:3eff:fe21:1277/64 scope link 02:17:28 valid_lft forever preferred_lft forever 02:17:28 02:17:28 02:17:28 ---> sar -b -r -n DEV: 02:17:28 Linux 6.8.0-90-generic (prd-ubuntu2404-robot-2c-8g-38843) 07/26/26 _x86_64_ (2 CPU) 02:17:28 02:17:28 00:17:46 LINUX RESTART (2 CPU) 02:17:28 02:17:28 00:20:00 tps rtps wtps dtps bread/s bwrtn/s bdscd/s 02:17:28 00:30:01 17.02 1.88 13.31 1.84 158.31 2594.79 724.27 02:17:28 00:40:01 1.34 0.01 1.31 0.02 1.11 26.69 1.44 02:17:28 00:50:01 1.22 0.00 1.21 0.01 0.00 17.19 0.96 02:17:28 01:00:01 1.50 0.00 1.48 0.02 0.00 22.03 0.65 02:17:28 01:10:01 1.26 0.00 1.26 0.00 0.00 18.09 0.00 02:17:28 01:20:01 1.33 0.00 1.33 0.00 0.01 20.49 0.00 02:17:28 01:30:01 1.58 0.00 1.57 0.02 0.00 23.83 0.65 02:17:28 01:40:01 14.86 0.00 1.35 13.50 0.15 21.99 106418.26 02:17:28 01:50:01 1.36 0.00 1.35 0.00 0.00 19.93 0.03 02:17:28 02:00:01 1.62 0.00 1.59 0.03 0.00 23.61 1.13 02:17:28 02:10:01 1.33 0.00 1.33 0.00 0.00 18.83 0.00 02:17:28 Average: 4.04 0.17 2.47 1.40 14.54 255.72 9739.26 02:17:28 02:17:28 00:20:00 kbmemfree kbavail kbmemused %memused kbbuffers kbcached kbcommit %commit kbactive kbinact kbdirty 02:17:28 00:30:01 5163708 7383212 383344 4.71 87100 2292072 806364 8.78 588772 2055756 2096 02:17:28 00:40:01 5179148 7400444 366000 4.50 87296 2293688 609404 6.64 586796 2057248 196 02:17:28 00:50:01 5179148 7401684 364776 4.49 87448 2294768 610732 6.65 588776 2058488 292 02:17:28 01:00:01 5168416 7392376 374056 4.60 87616 2296004 611764 6.66 589092 2059900 140 02:17:28 01:10:01 5168164 7393540 372880 4.59 87740 2297300 611828 6.66 589272 2061312 300 02:17:28 01:20:01 5166148 7393496 372908 4.59 87880 2299140 612040 6.67 589988 2063156 172 02:17:28 01:30:01 5163896 7392940 373464 4.59 87976 2300728 612220 6.67 590188 2064848 240 02:17:28 01:40:01 5163896 7395296 370980 4.56 88124 2302928 613192 6.68 590876 2067160 196 02:17:28 01:50:01 5166196 7399004 367116 4.51 88260 2304188 668396 7.28 596224 2068568 124 02:17:28 02:00:01 5159660 7393996 372280 4.58 88404 2305560 613388 6.68 591144 2070088 152 02:17:28 02:10:01 5159660 7395548 370708 4.56 88540 2306960 613388 6.68 591284 2071496 304 02:17:28 Average: 5167095 7394685 371683 4.57 87853 2299394 634792 6.91 590219 2063456 383 02:17:28 02:17:28 00:20:00 IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s %ifutil 02:17:28 00:30:01 lo 1.26 1.26 0.08 0.08 0.00 0.00 0.00 0.00 02:17:28 00:30:01 ens3 18.03 11.06 154.01 2.45 0.00 0.00 0.00 0.00 02:17:28 00:40:01 lo 1.00 1.00 0.05 0.05 0.00 0.00 0.00 0.00 02:17:28 00:40:01 ens3 1.31 0.23 0.30 0.09 0.00 0.00 0.00 0.00 02:17:28 00:50:01 lo 1.00 1.00 0.05 0.05 0.00 0.00 0.00 0.00 02:17:28 00:50:01 ens3 1.31 0.13 0.20 0.07 0.00 0.00 0.00 0.00 02:17:28 01:00:01 lo 1.00 1.00 0.05 0.05 0.00 0.00 0.00 0.00 02:17:28 01:00:01 ens3 1.09 0.15 0.20 0.09 0.00 0.00 0.00 0.00 02:17:28 01:10:01 lo 1.00 1.00 0.05 0.05 0.00 0.00 0.00 0.00 02:17:28 01:10:01 ens3 0.78 0.15 0.14 0.04 0.00 0.00 0.00 0.00 02:17:28 01:20:01 lo 1.00 1.00 0.05 0.05 0.00 0.00 0.00 0.00 02:17:28 01:20:01 ens3 1.12 0.24 0.29 0.16 0.00 0.00 0.00 0.00 02:17:28 01:30:01 lo 1.00 1.00 0.05 0.05 0.00 0.00 0.00 0.00 02:17:28 01:30:01 ens3 3.32 0.25 0.61 0.21 0.00 0.00 0.00 0.00 02:17:28 01:40:01 lo 1.00 1.00 0.05 0.05 0.00 0.00 0.00 0.00 02:17:28 01:40:01 ens3 0.92 0.16 0.16 0.08 0.00 0.00 0.00 0.00 02:17:28 01:50:01 lo 1.00 1.00 0.05 0.05 0.00 0.00 0.00 0.00 02:17:28 01:50:01 ens3 0.54 0.16 0.12 0.06 0.00 0.00 0.00 0.00 02:17:28 02:00:01 lo 1.00 1.00 0.05 0.05 0.00 0.00 0.00 0.00 02:17:28 02:00:01 ens3 0.68 0.28 0.14 0.09 0.00 0.00 0.00 0.00 02:17:28 02:10:01 lo 1.00 1.00 0.05 0.05 0.00 0.00 0.00 0.00 02:17:28 02:10:01 ens3 0.39 0.19 0.07 0.06 0.00 0.00 0.00 0.00 02:17:28 Average: lo 1.02 1.02 0.05 0.05 0.00 0.00 0.00 0.00 02:17:28 Average: ens3 2.68 1.18 14.23 0.31 0.00 0.00 0.00 0.00 02:17:28 02:17:28 02:17:28 ---> sar -P ALL: 02:17:28 Linux 6.8.0-90-generic (prd-ubuntu2404-robot-2c-8g-38843) 07/26/26 _x86_64_ (2 CPU) 02:17:28 02:17:28 00:17:46 LINUX RESTART (2 CPU) 02:17:28 02:17:28 00:20:00 CPU %user %nice %system %iowait %steal %idle 02:17:28 00:30:01 all 7.85 0.00 1.25 0.61 0.04 90.25 02:17:28 00:30:01 0 5.96 0.00 1.20 0.88 0.04 91.92 02:17:28 00:30:01 1 9.74 0.00 1.30 0.34 0.05 88.58 02:17:28 00:40:01 all 0.66 0.00 0.14 0.02 0.05 99.13 02:17:28 00:40:01 0 0.61 0.00 0.14 0.02 0.05 99.18 02:17:28 00:40:01 1 0.70 0.00 0.14 0.02 0.05 99.09 02:17:28 00:50:01 all 0.59 0.00 0.13 0.02 0.04 99.21 02:17:28 00:50:01 0 0.56 0.00 0.12 0.01 0.04 99.27 02:17:28 00:50:01 1 0.62 0.00 0.15 0.04 0.05 99.15 02:17:28 01:00:01 all 0.67 0.00 0.24 0.01 0.04 99.04 02:17:28 01:00:01 0 0.69 0.00 0.29 0.01 0.05 98.96 02:17:28 01:00:01 1 0.64 0.00 0.18 0.01 0.03 99.13 02:17:28 01:10:01 all 0.61 0.00 0.12 0.04 0.04 99.19 02:17:28 01:10:01 0 0.51 0.00 0.08 0.06 0.02 99.32 02:17:28 01:10:01 1 0.71 0.00 0.16 0.02 0.05 99.06 02:17:28 01:20:01 all 0.71 0.00 0.12 0.02 0.04 99.11 02:17:28 01:20:01 0 0.70 0.00 0.09 0.02 0.03 99.17 02:17:28 01:20:01 1 0.72 0.00 0.15 0.03 0.05 99.06 02:17:28 01:30:01 all 0.69 0.00 0.23 0.04 0.04 99.00 02:17:28 01:30:01 0 0.75 0.00 0.21 0.05 0.04 98.96 02:17:28 01:30:01 1 0.64 0.00 0.24 0.03 0.05 99.04 02:17:28 01:40:01 all 0.75 0.00 0.14 0.03 0.04 99.05 02:17:28 01:40:01 0 0.73 0.00 0.10 0.01 0.03 99.13 02:17:28 01:40:01 1 0.77 0.00 0.17 0.06 0.05 98.96 02:17:28 01:50:01 all 0.65 0.00 0.13 0.01 0.04 99.18 02:17:28 01:50:01 0 0.65 0.00 0.12 0.01 0.04 99.18 02:17:28 01:50:01 1 0.64 0.00 0.14 0.01 0.04 99.17 02:17:28 02:00:01 all 0.70 0.00 0.21 0.01 0.04 99.03 02:17:28 02:00:01 0 0.67 0.00 0.15 0.02 0.03 99.13 02:17:28 02:00:01 1 0.72 0.00 0.27 0.01 0.06 98.94 02:17:28 02:10:01 all 0.59 0.00 0.13 0.01 0.04 99.23 02:17:28 02:10:01 0 0.48 0.00 0.10 0.01 0.03 99.38 02:17:28 02:10:01 1 0.70 0.00 0.16 0.01 0.06 99.07 02:17:28 02:17:28 Average: CPU %user %nice %system %iowait %steal %idle 02:17:28 Average: all 1.32 0.00 0.26 0.08 0.04 98.31 02:17:28 Average: 0 1.12 0.00 0.24 0.10 0.04 98.51 02:17:28 Average: 1 1.51 0.00 0.28 0.05 0.05 98.11 02:17:28 02:17:28 02:17:28