23:01:37 Started by upstream project "integration-distribution-mri-test-titanium" build number 29
23:01:37 originally caused by:
23:01:37 Started by timer
23:01:37 Running as SYSTEM
23:01:37 [EnvInject] - Loading node environment variables.
23:01:37 Building remotely on prd-centos8-robot-2c-8g-8068 (centos8-robot-2c-8g) in workspace /w/workspace/controller-csit-3node-clustering-ask-all-titanium
23:01:39 [ssh-agent] Looking for ssh-agent implementation...
23:01:39 [ssh-agent] Exec ssh-agent (binary ssh-agent on a remote machine)
23:01:39 $ ssh-agent
23:01:39 SSH_AUTH_SOCK=/tmp/ssh-O8iTDGdjP5gf/agent.5288
23:01:39 SSH_AGENT_PID=5289
23:01:39 [ssh-agent] Started.
23:01:39 Running ssh-add (command line suppressed)
23:01:40 Identity added: /w/workspace/controller-csit-3node-clustering-ask-all-titanium@tmp/private_key_13038524713806100436.key (/w/workspace/controller-csit-3node-clustering-ask-all-titanium@tmp/private_key_13038524713806100436.key)
23:01:40 [ssh-agent] Using credentials jenkins (Release Engineering Jenkins Key)
23:01:40 The recommended git tool is: NONE
23:01:42 using credential opendaylight-jenkins-ssh
23:01:42 Wiping out workspace first.
23:01:42 Cloning the remote Git repository
23:01:42 Cloning repository git://devvexx.opendaylight.org/mirror/integration/test
23:01:42 > git init /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test # timeout=10
23:01:42 Fetching upstream changes from git://devvexx.opendaylight.org/mirror/integration/test
23:01:42 > git --version # timeout=10
23:01:42 > git --version # 'git version 2.43.0'
23:01:42 using GIT_SSH to set credentials Release Engineering Jenkins Key
23:01:42 [INFO] Currently running in a labeled security context
23:01:42 [INFO] Currently SELinux is 'enforcing' on the host
23:01:42 > /usr/bin/chcon --type=ssh_home_t /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test@tmp/jenkins-gitclient-ssh17601149514576326223.key
23:01:42 Verifying host key using known hosts file
23:01:42 You're using 'Known hosts file' strategy to verify ssh host keys, but your known_hosts file does not exist, please go to 'Manage Jenkins' -> 'Security' -> 'Git Host Key Verification Configuration' and configure host key verification.
23:01:42 > git fetch --tags --force --progress -- git://devvexx.opendaylight.org/mirror/integration/test +refs/heads/*:refs/remotes/origin/* # timeout=10
23:01:45 > git config remote.origin.url git://devvexx.opendaylight.org/mirror/integration/test # timeout=10
23:01:45 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
23:01:45 > git config remote.origin.url git://devvexx.opendaylight.org/mirror/integration/test # timeout=10
23:01:45 Fetching upstream changes from git://devvexx.opendaylight.org/mirror/integration/test
23:01:45 using GIT_SSH to set credentials Release Engineering Jenkins Key
23:01:45 [INFO] Currently running in a labeled security context
23:01:45 [INFO] Currently SELinux is 'enforcing' on the host
23:01:45 > /usr/bin/chcon --type=ssh_home_t /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test@tmp/jenkins-gitclient-ssh205405359758229232.key
23:01:45 Verifying host key using known hosts file
23:01:45 You're using 'Known hosts file' strategy to verify ssh host keys, but your known_hosts file does not exist, please go to 'Manage Jenkins' -> 'Security' -> 'Git Host Key Verification Configuration' and configure host key verification.
23:01:45 > git fetch --tags --force --progress -- git://devvexx.opendaylight.org/mirror/integration/test master # timeout=10
23:01:46 > git rev-parse FETCH_HEAD^{commit} # timeout=10
23:01:46 Checking out Revision 1d2a1a30bcc48e3c8d571eb0d9b6891fe29dfed4 (origin/master)
23:01:46 > git config core.sparsecheckout # timeout=10
23:01:46 > git checkout -f 1d2a1a30bcc48e3c8d571eb0d9b6891fe29dfed4 # timeout=10
23:01:46 Commit message: "Add bgpcep variables for titanium"
23:01:46 > git rev-parse FETCH_HEAD^{commit} # timeout=10
23:01:46 > git rev-list --no-walk d42b2c24fe4fcc7fe3ae0d352311e768f07a15b7 # timeout=10
23:01:46 No emails were triggered.
23:01:46 provisioning config files...
23:01:46 copy managed file [npmrc] to file:/home/jenkins/.npmrc
23:01:46 copy managed file [pipconf] to file:/home/jenkins/.config/pip/pip.conf
23:01:47 copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
23:01:47 [controller-csit-3node-clustering-ask-all-titanium] $ /bin/bash /tmp/jenkins1575639740843319992.sh
23:01:47 ---> python-tools-install.sh
23:01:47 Setup pyenv:
23:01:47 system
23:01:47 * 3.8.13 (set by /opt/pyenv/version)
23:01:47 * 3.9.13 (set by /opt/pyenv/version)
23:01:47 * 3.10.13 (set by /opt/pyenv/version)
23:01:47 * 3.11.7 (set by /opt/pyenv/version)
23:01:52 lf-activate-venv(): INFO: Creating python3 venv at /tmp/venv-Vbzb
23:01:52 lf-activate-venv(): INFO: Save venv in file: /tmp/.os_lf_venv
23:01:55 lf-activate-venv(): INFO: Installing: lftools
23:02:22 lf-activate-venv(): INFO: Adding /tmp/venv-Vbzb/bin to PATH
23:02:22 Generating Requirements File
23:02:42 Python 3.11.7
23:02:42 pip 25.0.1 from /tmp/venv-Vbzb/lib/python3.11/site-packages/pip (python 3.11)
23:02:42 appdirs==1.4.4
23:02:42 argcomplete==3.6.0
23:02:42 aspy.yaml==1.3.0
23:02:42 attrs==25.3.0
23:02:42 autopage==0.5.2
23:02:42 beautifulsoup4==4.13.3
23:02:42 boto3==1.37.13
23:02:42 botocore==1.37.13
23:02:42 bs4==0.0.2
23:02:42 cachetools==5.5.2
23:02:42 certifi==2025.1.31
23:02:42 cffi==1.17.1
23:02:42 cfgv==3.4.0
23:02:42 chardet==5.2.0
23:02:42 charset-normalizer==3.4.1
23:02:42 click==8.1.8
23:02:42 cliff==4.9.1
23:02:42 cmd2==2.5.11
23:02:42 cryptography==3.3.2
23:02:42 debtcollector==3.0.0
23:02:42 decorator==5.2.1
23:02:42 defusedxml==0.7.1
23:02:42 Deprecated==1.2.18
23:02:42 distlib==0.3.9
23:02:42 dnspython==2.7.0
23:02:42 docker==4.2.2
23:02:42 dogpile.cache==1.3.4
23:02:42 durationpy==0.9
23:02:42 email_validator==2.2.0
23:02:42 filelock==3.18.0
23:02:42 future==1.0.0
23:02:42 gitdb==4.0.12
23:02:42 GitPython==3.1.44
23:02:42 google-auth==2.38.0
23:02:42 httplib2==0.22.0
23:02:42 identify==2.6.9
23:02:42 idna==3.10
23:02:42 importlib-resources==1.5.0
23:02:42 iso8601==2.1.0
23:02:42 Jinja2==3.1.6
23:02:42 jmespath==1.0.1
23:02:42 jsonpatch==1.33
23:02:42 jsonpointer==3.0.0
23:02:42 jsonschema==4.23.0
23:02:42 jsonschema-specifications==2024.10.1
23:02:42 keystoneauth1==5.10.0
23:02:42 kubernetes==32.0.1
23:02:42 lftools==0.37.11
23:02:42 lxml==5.3.1
23:02:42 MarkupSafe==3.0.2
23:02:42 msgpack==1.1.0
23:02:42 multi_key_dict==2.0.3
23:02:42 munch==4.0.0
23:02:42 netaddr==1.3.0
23:02:42 niet==1.4.2
23:02:42 nodeenv==1.9.1
23:02:42 oauth2client==4.1.3
23:02:42 oauthlib==3.2.2
23:02:42 openstacksdk==4.4.0
23:02:42 os-client-config==2.1.0
23:02:42 os-service-types==1.7.0
23:02:42 osc-lib==3.2.0
23:02:42 oslo.config==9.7.1
23:02:42 oslo.context==5.7.1
23:02:42 oslo.i18n==6.5.1
23:02:42 oslo.log==7.1.0
23:02:42 oslo.serialization==5.7.0
23:02:42 oslo.utils==8.2.0
23:02:42 packaging==24.2
23:02:42 pbr==6.1.1
23:02:42 platformdirs==4.3.6
23:02:42 prettytable==3.15.1
23:02:42 psutil==7.0.0
23:02:42 pyasn1==0.6.1
23:02:42 pyasn1_modules==0.4.1
23:02:42 pycparser==2.22
23:02:42 pygerrit2==2.0.15
23:02:42 PyGithub==2.6.1
23:02:42 PyJWT==2.10.1
23:02:42 PyNaCl==1.5.0
23:02:42 pyparsing==2.4.7
23:02:42 pyperclip==1.9.0
23:02:42 pyrsistent==0.20.0
23:02:42 python-cinderclient==9.7.0
23:02:42 python-dateutil==2.9.0.post0
23:02:42 python-heatclient==4.1.0
23:02:42 python-jenkins==1.8.2
23:02:42 python-keystoneclient==5.6.0
23:02:42 python-magnumclient==4.8.1
23:02:42 python-openstackclient==7.4.0
23:02:42 python-swiftclient==4.7.0
23:02:42 PyYAML==6.0.2
23:02:42 referencing==0.36.2
23:02:42 requests==2.32.3
23:02:42 requests-oauthlib==2.0.0
23:02:42 requestsexceptions==1.4.0
23:02:42 rfc3986==2.0.0
23:02:42 rpds-py==0.23.1
23:02:42 rsa==4.9
23:02:42 ruamel.yaml==0.18.10
23:02:42 ruamel.yaml.clib==0.2.12
23:02:42 s3transfer==0.11.4
23:02:42 simplejson==3.20.1
23:02:42 six==1.17.0
23:02:42 smmap==5.0.2
23:02:42 soupsieve==2.6
23:02:42 stevedore==5.4.1
23:02:42 tabulate==0.9.0
23:02:42 toml==0.10.2
23:02:42 tomlkit==0.13.2
23:02:42 tqdm==4.67.1
23:02:42 typing_extensions==4.12.2
23:02:42 tzdata==2025.1
23:02:42 urllib3==1.26.20
23:02:42 virtualenv==20.29.3
23:02:42 wcwidth==0.2.13
23:02:42 websocket-client==1.8.0
23:02:42 wrapt==1.17.2
23:02:42 xdg==6.0.0
23:02:42 xmltodict==0.14.2
23:02:42 yq==3.4.3
23:02:42 [EnvInject] - Injecting environment variables from a build step.
23:02:42 [EnvInject] - Injecting as environment variables the properties content
23:02:42 OS_STACK_TEMPLATE=csit-2-instance-type.yaml
23:02:42 OS_CLOUD=vex
23:02:42 OS_STACK_NAME=releng-controller-csit-3node-clustering-ask-all-titanium-29
23:02:42 OS_STACK_TEMPLATE_DIR=openstack-hot
23:02:42
23:02:42 [EnvInject] - Variables injected successfully.
23:02:42 provisioning config files...
23:02:44 copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
23:02:44 [controller-csit-3node-clustering-ask-all-titanium] $ /bin/bash /tmp/jenkins1742238736263568790.sh
23:02:44 ---> Create parameters file for OpenStack HOT
23:02:44 OpenStack Heat parameters generated
23:02:44 -----------------------------------
23:02:44 parameters:
23:02:44 vm_0_count: '3'
23:02:44 vm_0_flavor: 'v3-standard-4'
23:02:44 vm_0_image: 'ZZCI - Ubuntu 20.04 - builder - x86_64 - 20250101-160150.998'
23:02:44 vm_1_count: '0'
23:02:44 vm_1_flavor: 'v3-standard-2'
23:02:44 vm_1_image: 'ZZCI - Ubuntu 22.04 - mininet-ovs-217 - x86_64 - 20250201-060151.911'
23:02:44
23:02:44 job_name: '19183-29'
23:02:44 silo: 'releng'
23:02:44 [controller-csit-3node-clustering-ask-all-titanium] $ /bin/bash -l /tmp/jenkins9986911652961675089.sh
23:02:44 ---> Create HEAT stack
23:02:44 + source /home/jenkins/lf-env.sh
23:02:44 + lf-activate-venv --python python3 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
23:02:44 ++ mktemp -d /tmp/venv-XXXX
23:02:44 + lf_venv=/tmp/venv-YTRE
23:02:44 + local venv_file=/tmp/.os_lf_venv
23:02:44 + local python=python3
23:02:44 + local options
23:02:44 + local set_path=true
23:02:44 + local install_args=
23:02:44 ++ getopt -o np:v: -l no-path,system-site-packages,python:,venv-file: -n lf-activate-venv -- --python python3 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
23:02:44 + options=' --python '\''python3'\'' -- '\''lftools[openstack]'\'' '\''kubernetes'\'' '\''niet'\'' '\''python-heatclient'\'' '\''python-openstackclient'\'' '\''python-magnumclient'\'' '\''yq'\'''
23:02:44 + eval set -- ' --python '\''python3'\'' -- '\''lftools[openstack]'\'' '\''kubernetes'\'' '\''niet'\'' '\''python-heatclient'\'' '\''python-openstackclient'\'' '\''python-magnumclient'\'' '\''yq'\'''
23:02:44 ++ set -- --python python3 -- 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
23:02:44 + true
23:02:44 + case $1 in
23:02:44 + python=python3
23:02:44 + shift 2
23:02:44 + true
23:02:44 + case $1 in
23:02:44 + shift
23:02:44 + break
23:02:44 + case $python in
23:02:44 + local pkg_list=
23:02:44 + [[ -d /opt/pyenv ]]
23:02:44 + echo 'Setup pyenv:'
23:02:44 Setup pyenv:
23:02:44 + export PYENV_ROOT=/opt/pyenv
23:02:44 + PYENV_ROOT=/opt/pyenv
23:02:44 + export PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
23:02:44 + PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
23:02:44 + pyenv versions
23:02:44 system
23:02:44 3.8.13
23:02:44 3.9.13
23:02:44 3.10.13
23:02:44 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-titanium/.python-version)
23:02:44 + command -v pyenv
23:02:44 ++ pyenv init - --no-rehash
23:02:44 + eval 'PATH="$(bash --norc -ec '\''IFS=:; paths=($PATH);
23:02:44 for i in ${!paths[@]}; do
23:02:44 if [[ ${paths[i]} == "'\'''\''/opt/pyenv/shims'\'''\''" ]]; then unset '\''\'\'''\''paths[i]'\''\'\'''\'';
23:02:44 fi; done;
23:02:44 echo "${paths[*]}"'\'')"
23:02:44 export PATH="/opt/pyenv/shims:${PATH}"
23:02:44 export PYENV_SHELL=bash
23:02:44 source '\''/opt/pyenv/libexec/../completions/pyenv.bash'\''
23:02:44 pyenv() {
23:02:44 local command
23:02:44 command="${1:-}"
23:02:44 if [ "$#" -gt 0 ]; then
23:02:44 shift
23:02:44 fi
23:02:44
23:02:44 case "$command" in
23:02:44 rehash|shell)
23:02:44 eval "$(pyenv "sh-$command" "$@")"
23:02:44 ;;
23:02:44 *)
23:02:44 command pyenv "$command" "$@"
23:02:44 ;;
23:02:44 esac
23:02:44 }'
23:02:44 +++ bash --norc -ec 'IFS=:; paths=($PATH);
23:02:44 for i in ${!paths[@]}; do
23:02:44 if [[ ${paths[i]} == "/opt/pyenv/shims" ]]; then unset '\''paths[i]'\'';
23:02:44 fi; done;
23:02:44 echo "${paths[*]}"'
23:02:44 ++ PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
23:02:44 ++ export PATH=/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
23:02:44 ++ PATH=/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
23:02:44 ++ export PYENV_SHELL=bash
23:02:44 ++ PYENV_SHELL=bash
23:02:44 ++ source /opt/pyenv/libexec/../completions/pyenv.bash
23:02:44 +++ complete -F _pyenv pyenv
23:02:44 ++ lf-pyver python3
23:02:44 ++ local py_version_xy=python3
23:02:44 ++ local py_version_xyz=
23:02:44 ++ awk '{ print $1 }'
23:02:44 ++ pyenv versions
23:02:44 ++ local command
23:02:44 ++ command=versions
23:02:44 ++ '[' 1 -gt 0 ']'
23:02:44 ++ shift
23:02:44 ++ case "$command" in
23:02:44 ++ command pyenv versions
23:02:44 ++ pyenv versions
23:02:44 ++ sed 's/^[ *]* //'
23:02:44 ++ grep -E '^[0-9.]*[0-9]$'
23:02:44 ++ [[ ! -s /tmp/.pyenv_versions ]]
23:02:44 +++ grep '^3' /tmp/.pyenv_versions
23:02:44 +++ tail -n 1
23:02:44 +++ sort -V
23:02:44 ++ py_version_xyz=3.11.7
23:02:44 ++ [[ -z 3.11.7 ]]
23:02:44 ++ echo 3.11.7
23:02:44 ++ return 0
23:02:44 + pyenv local 3.11.7
23:02:44 + local command
23:02:44 + command=local
23:02:44 + '[' 2 -gt 0 ']'
23:02:44 + shift
23:02:44 + case "$command" in
23:02:44 + command pyenv local 3.11.7
23:02:44 + pyenv local 3.11.7
23:02:44 + for arg in "$@"
23:02:44 + case $arg in
23:02:44 + pkg_list+='lftools[openstack] '
23:02:44 + for arg in "$@"
23:02:44 + case $arg in
23:02:44 + pkg_list+='kubernetes '
23:02:44 + for arg in "$@"
23:02:44 + case $arg in
23:02:44 + pkg_list+='niet '
23:02:44 + for arg in "$@"
23:02:44 + case $arg in
23:02:44 + pkg_list+='python-heatclient '
23:02:44 + for arg in "$@"
23:02:44 + case $arg in
23:02:44 + pkg_list+='python-openstackclient '
23:02:44 + for arg in "$@"
23:02:44 + case $arg in
23:02:44 + pkg_list+='python-magnumclient '
23:02:44 + for arg in "$@"
23:02:44 + case $arg in
23:02:44 + pkg_list+='yq '
23:02:44 + [[ -f /tmp/.os_lf_venv ]]
23:02:44 ++ cat /tmp/.os_lf_venv
23:02:44 + lf_venv=/tmp/venv-Vbzb
23:02:44 + echo 'lf-activate-venv(): INFO: Reuse venv:/tmp/venv-Vbzb from' file:/tmp/.os_lf_venv
23:02:44 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-Vbzb from file:/tmp/.os_lf_venv
23:02:44 + /tmp/venv-Vbzb/bin/python3 -m pip install --upgrade --quiet pip virtualenv
23:02:45 + [[ -z lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq ]]
23:02:45 + echo 'lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq '
23:02:45 lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
23:02:45 + /tmp/venv-Vbzb/bin/python3 -m pip install --upgrade --quiet --upgrade-strategy eager 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
23:02:59 + type python3
23:02:59 + true
23:02:59 + echo 'lf-activate-venv(): INFO: Adding /tmp/venv-Vbzb/bin to PATH'
23:02:59 lf-activate-venv(): INFO: Adding /tmp/venv-Vbzb/bin to PATH
23:02:59 + PATH=/tmp/venv-Vbzb/bin:/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
23:02:59 + return 0
23:02:59 + openstack --os-cloud vex limits show --absolute
23:03:00 +--------------------------+---------+
23:03:00 | Name | Value |
23:03:00 +--------------------------+---------+
23:03:00 | maxTotalInstances | -1 |
23:03:00 | maxTotalCores | 450 |
23:03:00 | maxTotalRAMSize | 1000000 |
23:03:00 | maxServerMeta | 128 |
23:03:00 | maxImageMeta | 128 |
23:03:00 | maxPersonality | 5 |
23:03:00 | maxPersonalitySize | 10240 |
23:03:00 | maxTotalKeypairs | 100 |
23:03:00 | maxServerGroups | 10 |
23:03:00 | maxServerGroupMembers | 10 |
23:03:00 | maxTotalFloatingIps | -1 |
23:03:00 | maxSecurityGroups | -1 |
23:03:00 | maxSecurityGroupRules | -1 |
23:03:00 | totalRAMUsed | 630784 |
23:03:00 | totalCoresUsed | 154 |
23:03:00 | totalInstancesUsed | 55 |
23:03:00 | totalFloatingIpsUsed | 0 |
23:03:00 | totalSecurityGroupsUsed | 0 |
23:03:00 | totalServerGroupsUsed | 0 |
23:03:00 | maxTotalVolumes | -1 |
23:03:00 | maxTotalSnapshots | 10 |
23:03:00 | maxTotalVolumeGigabytes | 4096 |
23:03:00 | maxTotalBackups | 10 |
23:03:00 | maxTotalBackupGigabytes | 1000 |
23:03:00 | totalVolumesUsed | 3 |
23:03:00 | totalGigabytesUsed | 60 |
23:03:00 | totalSnapshotsUsed | 0 |
23:03:00 | totalBackupsUsed | 0 |
23:03:00 | totalBackupGigabytesUsed | 0 |
23:03:00 +--------------------------+---------+
23:03:00 + pushd /opt/ciman/openstack-hot
23:03:00 /opt/ciman/openstack-hot /w/workspace/controller-csit-3node-clustering-ask-all-titanium
23:03:00 + lftools openstack --os-cloud vex stack create releng-controller-csit-3node-clustering-ask-all-titanium-29 csit-2-instance-type.yaml /w/workspace/controller-csit-3node-clustering-ask-all-titanium/stack-parameters.yaml
23:03:26 Creating stack releng-controller-csit-3node-clustering-ask-all-titanium-29
23:03:26 Waiting to initialize infrastructure...
23:03:26 Stack initialization successful.
23:03:26 ------------------------------------
23:03:26 Stack Details
23:03:26 ------------------------------------
23:03:26 {'added': None,
23:03:26 'capabilities': [],
23:03:26 'created_at': '2025-03-16T23:03:03Z',
23:03:26 'deleted': None,
23:03:26 'deleted_at': None,
23:03:26 'description': 'No description',
23:03:26 'environment': None,
23:03:26 'environment_files': None,
23:03:26 'files': None,
23:03:26 'files_container': None,
23:03:26 'id': 'befe14a8-b81f-4ab7-ac35-08da7aa99a7d',
23:03:26 'is_rollback_disabled': True,
23:03:26 'links': [{'href': 'https://orchestration.public.mtl1.vexxhost.net/v1/12c36e260d8e4bb2913965203b1b491f/stacks/releng-controller-csit-3node-clustering-ask-all-titanium-29/befe14a8-b81f-4ab7-ac35-08da7aa99a7d',
23:03:26 'rel': 'self'}],
23:03:26 'location': Munch({'cloud': 'vex', 'region_name': 'ca-ymq-1', 'zone': None, 'project': Munch({'id': '12c36e260d8e4bb2913965203b1b491f', 'name': '61975f2c-7c17-4d69-82fa-c3ae420ad6fd', 'domain_id': None, 'domain_name': 'Default'})}),
23:03:26 'name': 'releng-controller-csit-3node-clustering-ask-all-titanium-29',
23:03:26 'notification_topics': [],
23:03:26 'outputs': [{'description': 'IP addresses of the 2nd vm types',
23:03:26 'output_key': 'vm_1_ips',
23:03:26 'output_value': []},
23:03:26 {'description': 'IP addresses of the 1st vm types',
23:03:26 'output_key': 'vm_0_ips',
23:03:26 'output_value': ['10.30.170.209',
23:03:26 '10.30.170.81',
23:03:26 '10.30.170.131']}],
23:03:26 'owner_id': ****,
23:03:26 'parameters': {'OS::project_id': '12c36e260d8e4bb2913965203b1b491f',
23:03:26 'OS::stack_id': 'befe14a8-b81f-4ab7-ac35-08da7aa99a7d',
23:03:26 'OS::stack_name': 'releng-controller-csit-3node-clustering-ask-all-titanium-29',
23:03:26 'job_name': '19183-29',
23:03:26 'silo': 'releng',
23:03:26 'vm_0_count': '3',
23:03:26 'vm_0_flavor': 'v3-standard-4',
23:03:26 'vm_0_image': 'ZZCI - Ubuntu 20.04 - builder - x86_64 - '
23:03:26 '20250101-160150.998',
23:03:26 'vm_1_count': '0',
23:03:26 'vm_1_flavor': 'v3-standard-2',
23:03:26 'vm_1_image': 'ZZCI - Ubuntu 22.04 - mininet-ovs-217 - x86_64 '
23:03:26 '- 20250201-060151.911'},
23:03:26 'parent_id': None,
23:03:26 'replaced': None,
23:03:26 'status': 'CREATE_COMPLETE',
23:03:26 'status_reason': 'Stack CREATE completed successfully',
23:03:26 'tags': [],
23:03:26 'template': None,
23:03:26 'template_description': 'No description',
23:03:26 'template_url': None,
23:03:26 'timeout_mins': 15,
23:03:26 'unchanged': None,
23:03:26 'updated': None,
23:03:26 'updated_at': None,
23:03:26 'user_project_id': 'f866186e22294fb484b9be58696197b9'}
23:03:26 ------------------------------------
23:03:27 + popd
23:03:27 /w/workspace/controller-csit-3node-clustering-ask-all-titanium
23:03:27 [controller-csit-3node-clustering-ask-all-titanium] $ /bin/bash -l /tmp/jenkins11754999730676606307.sh
23:03:27 ---> Copy SSH public keys to CSIT lab
23:03:27 Setup pyenv:
23:03:27 system
23:03:27 3.8.13
23:03:27 3.9.13
23:03:27 3.10.13
23:03:27 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-titanium/.python-version)
23:03:27 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-Vbzb from file:/tmp/.os_lf_venv
23:03:28 lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes python-heatclient python-openstackclient
23:03:41 lf-activate-venv(): INFO: Adding /tmp/venv-Vbzb/bin to PATH
23:03:42 SSH not responding on 10.30.170.131. Retrying in 10 seconds...
23:03:42 SSH not responding on 10.30.170.209. Retrying in 10 seconds...
23:03:42 SSH not responding on 10.30.170.81. Retrying in 10 seconds...
23:03:52 Ping to 10.30.170.131 successful.
23:03:52 Ping to 10.30.170.81 successful.
23:03:52 Ping to 10.30.170.209 successful.
23:03:53 SSH not responding on 10.30.170.131. Retrying in 10 seconds...
23:03:53 SSH not responding on 10.30.170.81. Retrying in 10 seconds...
23:03:53 SSH not responding on 10.30.170.209. Retrying in 10 seconds...
23:04:03 Ping to 10.30.170.81 successful.
23:04:03 Ping to 10.30.170.131 successful.
23:04:03 Ping to 10.30.170.209 successful.
23:04:04 Warning: Permanently added '10.30.170.131' (ECDSA) to the list of known hosts.
23:04:04 Warning: Permanently added '10.30.170.81' (ECDSA) to the list of known hosts.
23:04:04 Warning: Permanently added '10.30.170.209' (ECDSA) to the list of known hosts.
23:04:04 releng-19183-29-0-builder-2
23:04:04 Successfully copied public keys to slave 10.30.170.131
23:04:04 releng-19183-29-0-builder-1
23:04:04 Successfully copied public keys to slave 10.30.170.81
23:04:04 releng-19183-29-0-builder-0
23:04:04 Successfully copied public keys to slave 10.30.170.209
23:04:04 Process 6512 ready.
23:04:04 Process 6513 ready.
23:04:04 Process 6514 ready.
23:04:04 SSH ready on all stack servers.
23:04:04 [controller-csit-3node-clustering-ask-all-titanium] $ /bin/bash -l /tmp/jenkins40447868460525044.sh
23:04:04 Setup pyenv:
23:04:05 system
23:04:05 3.8.13
23:04:05 3.9.13
23:04:05 3.10.13
23:04:05 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-titanium/.python-version)
23:04:09 lf-activate-venv(): INFO: Creating python3 venv at /tmp/venv-4m8h
23:04:09 lf-activate-venv(): INFO: Save venv in file: /w/workspace/controller-csit-3node-clustering-ask-all-titanium/.robot_venv
23:04:12 lf-activate-venv(): INFO: Installing: setuptools wheel
23:04:13 lf-activate-venv(): INFO: Adding /tmp/venv-4m8h/bin to PATH
23:04:13 + echo 'Installing Python Requirements'
23:04:13 Installing Python Requirements
23:04:13 + cat
23:04:13 + python -m pip install -r requirements.txt
23:04:14 Looking in indexes: https://nexus3.opendaylight.org/repository/PyPi/simple
23:04:14 Collecting docker-py (from -r requirements.txt (line 1))
23:04:14 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/docker-py/1.10.6/docker_py-1.10.6-py2.py3-none-any.whl (50 kB)
23:04:14 Collecting ipaddr (from -r requirements.txt (line 2))
23:04:14 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/ipaddr/2.2.0/ipaddr-2.2.0.tar.gz (26 kB)
23:04:14 Preparing metadata (setup.py): started
23:04:14 Preparing metadata (setup.py): finished with status 'done'
23:04:14 Collecting netaddr (from -r requirements.txt (line 3))
23:04:14 Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/netaddr/1.3.0/netaddr-1.3.0-py3-none-any.whl (2.3 MB)
23:04:14 Collecting netifaces (from -r requirements.txt (line 4))
23:04:14 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/netifaces/0.11.0/netifaces-0.11.0.tar.gz (30 kB)
23:04:14 Preparing metadata (setup.py): started
23:04:14 Preparing metadata (setup.py): finished with status 'done'
23:04:14 Collecting pyhocon (from -r requirements.txt (line 5))
23:04:14 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/pyhocon/0.3.61/pyhocon-0.3.61-py3-none-any.whl (25 kB)
23:04:14 Collecting requests (from -r requirements.txt (line 6))
23:04:14 Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/requests/2.32.3/requests-2.32.3-py3-none-any.whl (64 kB)
23:04:14 Collecting robotframework (from -r requirements.txt (line 7))
23:04:14 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework/7.2.2/robotframework-7.2.2-py3-none-any.whl (777 kB)
23:04:15 â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â” 777.7/777.7 kB 30.6 MB/s eta 0:00:00
23:04:15 Collecting robotframework-httplibrary (from -r requirements.txt (line 8))
23:04:15 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-httplibrary/0.4.2/robotframework-httplibrary-0.4.2.tar.gz (9.1 kB)
23:04:15 Preparing metadata (setup.py): started
23:04:15 Preparing metadata (setup.py): finished with status 'done'
23:04:15 Collecting robotframework-requests==0.9.7 (from -r requirements.txt (line 9))
23:04:15 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-requests/0.9.7/robotframework_requests-0.9.7-py3-none-any.whl (21 kB)
23:04:15 Collecting robotframework-selenium2library (from -r requirements.txt (line 10))
23:04:15 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-selenium2library/3.0.0/robotframework_selenium2library-3.0.0-py2.py3-none-any.whl (6.2 kB)
23:04:15 Collecting robotframework-sshlibrary==3.8.0 (from -r requirements.txt (line 11))
23:04:15 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-sshlibrary/3.8.0/robotframework-sshlibrary-3.8.0.tar.gz (51 kB)
23:04:15 Preparing metadata (setup.py): started
23:04:15 Preparing metadata (setup.py): finished with status 'done'
23:04:15 Collecting scapy (from -r requirements.txt (line 12))
23:04:15 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/scapy/2.6.1/scapy-2.6.1-py3-none-any.whl (2.4 MB)
23:04:15 â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â” 2.4/2.4 MB 52.4 MB/s eta 0:00:00
23:04:15 Collecting jsonpath-rw (from -r requirements.txt (line 15))
23:04:15 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/jsonpath-rw/1.4.0/jsonpath-rw-1.4.0.tar.gz (13 kB)
23:04:15 Preparing metadata (setup.py): started
23:04:15 Preparing metadata (setup.py): finished with status 'done'
23:04:15 Collecting elasticsearch (from -r requirements.txt (line 18))
23:04:15 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elasticsearch/8.17.2/elasticsearch-8.17.2-py3-none-any.whl (717 kB)
23:04:16 â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â” 718.0/718.0 kB 34.0 MB/s eta 0:00:00
23:04:16 Collecting elasticsearch-dsl (from -r requirements.txt (line 19))
23:04:16 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elasticsearch-dsl/8.17.1/elasticsearch_dsl-8.17.1-py3-none-any.whl (158 kB)
23:04:16 Collecting pyangbind (from -r requirements.txt (line 22))
23:04:16 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/pyangbind/0.8.6/pyangbind-0.8.6-py3-none-any.whl (52 kB)
23:04:16 Collecting isodate (from -r requirements.txt (line 25))
23:04:16 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/isodate/0.7.2/isodate-0.7.2-py3-none-any.whl (22 kB)
23:04:16 Collecting jmespath (from -r requirements.txt (line 28))
23:04:16 Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/jmespath/1.0.1/jmespath-1.0.1-py3-none-any.whl (20 kB)
23:04:16 Collecting jsonpatch (from -r requirements.txt (line 31))
23:04:16 Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/jsonpatch/1.33/jsonpatch-1.33-py2.py3-none-any.whl (12 kB)
23:04:16 Collecting paramiko>=1.15.3 (from robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
23:04:16 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/paramiko/3.5.1/paramiko-3.5.1-py3-none-any.whl (227 kB)
23:04:16 Collecting scp>=0.13.0 (from robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
23:04:16 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/scp/0.15.0/scp-0.15.0-py2.py3-none-any.whl (8.8 kB)
23:04:16 Collecting docker-pycreds>=0.2.1 (from docker-py->-r requirements.txt (line 1))
23:04:16 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/docker-pycreds/0.4.0/docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)
23:04:16 Collecting six>=1.4.0 (from docker-py->-r requirements.txt (line 1))
23:04:16 Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/six/1.17.0/six-1.17.0-py2.py3-none-any.whl (11 kB)
23:04:16 Collecting websocket-client>=0.32.0 (from docker-py->-r requirements.txt (line 1))
23:04:16 Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/websocket-client/1.8.0/websocket_client-1.8.0-py3-none-any.whl (58 kB)
23:04:16 Collecting pyparsing<4,>=2 (from pyhocon->-r requirements.txt (line 5))
23:04:16 Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/pyparsing/3.2.1/pyparsing-3.2.1-py3-none-any.whl (107 kB)
23:04:16 Collecting charset-normalizer<4,>=2 (from requests->-r requirements.txt (line 6))
23:04:16 Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/charset-normalizer/3.4.1/charset_normalizer-3.4.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (143 kB)
23:04:16 Collecting idna<4,>=2.5 (from requests->-r requirements.txt (line 6))
23:04:16 Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/idna/3.10/idna-3.10-py3-none-any.whl (70 kB)
23:04:16 Collecting urllib3<3,>=1.21.1 (from requests->-r requirements.txt (line 6))
23:04:16 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/urllib3/2.3.0/urllib3-2.3.0-py3-none-any.whl (128 kB)
23:04:16 Collecting certifi>=2017.4.17 (from requests->-r requirements.txt (line 6))
23:04:16 Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/certifi/2025.1.31/certifi-2025.1.31-py3-none-any.whl (166 kB)
23:04:16 Collecting webtest>=2.0 (from robotframework-httplibrary->-r requirements.txt (line 8))
23:04:16 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/webtest/3.0.4/WebTest-3.0.4-py3-none-any.whl (32 kB)
23:04:16 Collecting jsonpointer (from robotframework-httplibrary->-r requirements.txt (line 8))
23:04:16 Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/jsonpointer/3.0.0/jsonpointer-3.0.0-py2.py3-none-any.whl (7.6 kB)
23:04:17 Collecting robotframework-seleniumlibrary>=3.0.0 (from robotframework-selenium2library->-r requirements.txt (line 10))
23:04:17 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-seleniumlibrary/6.7.1/robotframework_seleniumlibrary-6.7.1-py2.py3-none-any.whl (104 kB)
23:04:17 Collecting ply (from jsonpath-rw->-r requirements.txt (line 15))
23:04:17 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/ply/3.11/ply-3.11-py2.py3-none-any.whl (49 kB)
23:04:17 Collecting decorator (from jsonpath-rw->-r requirements.txt (line 15))
23:04:17 Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/decorator/5.2.1/decorator-5.2.1-py3-none-any.whl (9.2 kB)
23:04:17 Collecting elastic-transport<9,>=8.15.1 (from elasticsearch->-r requirements.txt (line 18))
23:04:17 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elastic-transport/8.17.1/elastic_transport-8.17.1-py3-none-any.whl (64 kB)
23:04:17 Collecting python-dateutil (from elasticsearch-dsl->-r requirements.txt (line 19))
23:04:17 Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/python-dateutil/2.9.0.post0/python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
23:04:17 Collecting typing-extensions (from elasticsearch-dsl->-r requirements.txt (line 19))
23:04:17 Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/typing-extensions/4.12.2/typing_extensions-4.12.2-py3-none-any.whl (37 kB)
23:04:17 Collecting pyang (from pyangbind->-r requirements.txt (line 22))
23:04:17 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/pyang/2.6.1/pyang-2.6.1-py2.py3-none-any.whl (594 kB)
23:04:17 â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â” 594.7/594.7 kB 31.2 MB/s eta 0:00:00
23:04:17 Collecting lxml (from pyangbind->-r requirements.txt (line 22))
23:04:17 Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/lxml/5.3.1/lxml-5.3.1-cp311-cp311-manylinux_2_28_x86_64.whl (5.0 MB)
23:04:18 Collecting regex (from pyangbind->-r requirements.txt (line 22))
23:04:18 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/regex/2024.11.6/regex-2024.11.6-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (792 kB)
23:04:18 â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â” 792.7/792.7 kB 26.3 MB/s eta 0:00:00
23:04:18 Collecting enum34 (from pyangbind->-r requirements.txt (line 22))
23:04:18 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/enum34/1.1.10/enum34-1.1.10-py3-none-any.whl (11 kB)
23:04:18 Collecting bcrypt>=3.2 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
23:04:18 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/bcrypt/4.3.0/bcrypt-4.3.0-cp39-abi3-manylinux_2_28_x86_64.whl (284 kB)
23:04:19 Collecting cryptography>=3.3 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
23:04:19 Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/cryptography/44.0.2/cryptography-44.0.2-cp39-abi3-manylinux_2_28_x86_64.whl (4.2 MB)
23:04:19 Collecting pynacl>=1.5 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
23:04:19 Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/pynacl/1.5.0/PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (856 kB)
23:04:19 Collecting selenium>=4.3.0 (from robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
23:04:19 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/selenium/4.29.0/selenium-4.29.0-py3-none-any.whl (9.5 MB)
23:04:19 â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â”â” 9.5/9.5 MB 74.2 MB/s eta 0:00:00
23:04:19 Collecting robotframework-pythonlibcore>=4.4.1 (from robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
23:04:19 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-pythonlibcore/4.4.1/robotframework_pythonlibcore-4.4.1-py2.py3-none-any.whl (12 kB)
23:04:19 Collecting click>=8.0 (from robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
23:04:19 Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/click/8.1.8/click-8.1.8-py3-none-any.whl (98 kB)
23:04:19 Collecting WebOb>=1.2 (from webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8))
23:04:19 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/webob/1.8.9/WebOb-1.8.9-py2.py3-none-any.whl (115 kB)
23:04:19 Collecting waitress>=3.0.2 (from webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8))
23:04:19 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/waitress/3.0.2/waitress-3.0.2-py3-none-any.whl (56 kB)
23:04:19 Collecting beautifulsoup4 (from webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8))
23:04:19 Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/beautifulsoup4/4.13.3/beautifulsoup4-4.13.3-py3-none-any.whl (186 kB)
23:04:20 Collecting cffi>=1.12 (from cryptography>=3.3->paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
23:04:20 Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/cffi/1.17.1/cffi-1.17.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (467 kB)
23:04:20 Collecting trio~=0.17 (from selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
23:04:20 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/trio/0.29.0/trio-0.29.0-py3-none-any.whl (492 kB)
23:04:20 Collecting trio-websocket~=0.9 (from selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
23:04:20 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/trio-websocket/0.12.2/trio_websocket-0.12.2-py3-none-any.whl (21 kB)
23:04:20 Collecting soupsieve>1.2 (from beautifulsoup4->webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8))
23:04:20 Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/soupsieve/2.6/soupsieve-2.6-py3-none-any.whl (36 kB)
23:04:20 Collecting pycparser (from cffi>=1.12->cryptography>=3.3->paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
23:04:20 Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/pycparser/2.22/pycparser-2.22-py3-none-any.whl (117 kB)
23:04:20 Collecting attrs>=23.2.0 (from trio~=0.17->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
23:04:20 Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/attrs/25.3.0/attrs-25.3.0-py3-none-any.whl (63 kB)
23:04:20 Collecting sortedcontainers (from trio~=0.17->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
23:04:20 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/sortedcontainers/2.4.0/sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
23:04:20 Collecting outcome (from trio~=0.17->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
23:04:20 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/outcome/1.3.0.post0/outcome-1.3.0.post0-py2.py3-none-any.whl (10 kB)
23:04:20 Collecting sniffio>=1.3.0 (from trio~=0.17->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
23:04:20 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/sniffio/1.3.1/sniffio-1.3.1-py3-none-any.whl (10 kB)
23:04:20 Collecting wsproto>=0.14 (from trio-websocket~=0.9->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
23:04:20 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/wsproto/1.2.0/wsproto-1.2.0-py3-none-any.whl (24 kB)
23:04:20 Collecting pysocks!=1.5.7,<2.0,>=1.5.6 (from urllib3[socks]<3,>=1.26->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
23:04:20 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/pysocks/1.7.1/PySocks-1.7.1-py3-none-any.whl (16 kB)
23:04:20 Collecting h11<1,>=0.9.0 (from wsproto>=0.14->trio-websocket~=0.9->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
23:04:20 Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/h11/0.14.0/h11-0.14.0-py3-none-any.whl (58 kB)
23:04:20 Building wheels for collected packages: robotframework-sshlibrary, ipaddr, netifaces, robotframework-httplibrary, jsonpath-rw
23:04:20 Building wheel for robotframework-sshlibrary (setup.py): started
23:04:21 Building wheel for robotframework-sshlibrary (setup.py): finished with status 'done'
23:04:21 Created wheel for robotframework-sshlibrary: filename=robotframework_sshlibrary-3.8.0-py3-none-any.whl size=55175 sha256=77d65c727033f39c158eeba844e8aa973ab290206a57789908d622a60f96cea4
23:04:21 Stored in directory: /home/jenkins/.cache/pip/wheels/f7/c9/b3/a977b7bcc410d45ae27d240df3d00a12585509180e373ecccc
23:04:21 Building wheel for ipaddr (setup.py): started
23:04:21 Building wheel for ipaddr (setup.py): finished with status 'done'
23:04:21 Created wheel for ipaddr: filename=ipaddr-2.2.0-py3-none-any.whl size=18322 sha256=dde1f734a65cbddbb50996e1559b11909b5ff23193e625d585d288febd185bff
23:04:21 Stored in directory: /home/jenkins/.cache/pip/wheels/dc/6c/04/da2d847fa8d45c59af3e1d83e2acc29cb8adcbaf04c0898dbf
23:04:21 Building wheel for netifaces (setup.py): started
23:04:23 Building wheel for netifaces (setup.py): finished with status 'done'
23:04:23 Created wheel for netifaces: filename=netifaces-0.11.0-cp311-cp311-linux_x86_64.whl size=41060 sha256=e5beb00b94b12869639b13846ca75911b7b9464663cd0f2469a00002281ea697
23:04:23 Stored in directory: /home/jenkins/.cache/pip/wheels/f8/18/88/e61d54b995bea304bdb1d040a92b72228a1bf72ca2a3eba7c9
23:04:23 Building wheel for robotframework-httplibrary (setup.py): started
23:04:24 Building wheel for robotframework-httplibrary (setup.py): finished with status 'done'
23:04:24 Created wheel for robotframework-httplibrary: filename=robotframework_httplibrary-0.4.2-py3-none-any.whl size=10013 sha256=bd578d5b052ed2edea61fa5c6528914d79c607083394eccfaae12d279d1ac447
23:04:24 Stored in directory: /home/jenkins/.cache/pip/wheels/aa/bc/0d/9a20dd51effef392aae2733cb4c7b66c6fa29fca33d88b57ed
23:04:24 Building wheel for jsonpath-rw (setup.py): started
23:04:24 Building wheel for jsonpath-rw (setup.py): finished with status 'done'
23:04:24 Created wheel for jsonpath-rw: filename=jsonpath_rw-1.4.0-py3-none-any.whl size=15175 sha256=5e225b14ad4632522e8376b22d7d8604487502f8dd45b4db94996a33fdb9f57b
23:04:24 Stored in directory: /home/jenkins/.cache/pip/wheels/f1/54/63/9a8da38cefae13755097b36cc852decc25d8ef69c37d58d4eb
23:04:24 Successfully built robotframework-sshlibrary ipaddr netifaces robotframework-httplibrary jsonpath-rw
23:04:24 Installing collected packages: sortedcontainers, ply, netifaces, ipaddr, enum34, websocket-client, WebOb, waitress, urllib3, typing-extensions, soupsieve, sniffio, six, scapy, robotframework-pythonlibcore, robotframework, regex, pysocks, pyparsing, pycparser, netaddr, lxml, jsonpointer, jmespath, isodate, idna, h11, decorator, click, charset-normalizer, certifi, bcrypt, attrs, wsproto, requests, python-dateutil, pyhocon, pyang, outcome, jsonpath-rw, jsonpatch, elastic-transport, docker-pycreds, cffi, beautifulsoup4, webtest, trio, robotframework-requests, pynacl, pyangbind, elasticsearch, docker-py, cryptography, trio-websocket, robotframework-httplibrary, paramiko, elasticsearch-dsl, selenium, scp, robotframework-sshlibrary, robotframework-seleniumlibrary, robotframework-selenium2library
23:04:31 Successfully installed WebOb-1.8.9 attrs-25.3.0 bcrypt-4.3.0 beautifulsoup4-4.13.3 certifi-2025.1.31 cffi-1.17.1 charset-normalizer-3.4.1 click-8.1.8 cryptography-44.0.2 decorator-5.2.1 docker-py-1.10.6 docker-pycreds-0.4.0 elastic-transport-8.17.1 elasticsearch-8.17.2 elasticsearch-dsl-8.17.1 enum34-1.1.10 h11-0.14.0 idna-3.10 ipaddr-2.2.0 isodate-0.7.2 jmespath-1.0.1 jsonpatch-1.33 jsonpath-rw-1.4.0 jsonpointer-3.0.0 lxml-5.3.1 netaddr-1.3.0 netifaces-0.11.0 outcome-1.3.0.post0 paramiko-3.5.1 ply-3.11 pyang-2.6.1 pyangbind-0.8.6 pycparser-2.22 pyhocon-0.3.61 pynacl-1.5.0 pyparsing-3.2.1 pysocks-1.7.1 python-dateutil-2.9.0.post0 regex-2024.11.6 requests-2.32.3 robotframework-7.2.2 robotframework-httplibrary-0.4.2 robotframework-pythonlibcore-4.4.1 robotframework-requests-0.9.7 robotframework-selenium2library-3.0.0 robotframework-seleniumlibrary-6.7.1 robotframework-sshlibrary-3.8.0 scapy-2.6.1 scp-0.15.0 selenium-4.29.0 six-1.17.0 sniffio-1.3.1 sortedcontainers-2.4.0 soupsieve-2.6 trio-0.29.0 trio-websocket-0.12.2 typing-extensions-4.12.2 urllib3-2.3.0 waitress-3.0.2 websocket-client-1.8.0 webtest-3.0.4 wsproto-1.2.0
23:04:31 + pip freeze
23:04:31 attrs==25.3.0
23:04:31 bcrypt==4.3.0
23:04:31 beautifulsoup4==4.13.3
23:04:31 certifi==2025.1.31
23:04:31 cffi==1.17.1
23:04:31 charset-normalizer==3.4.1
23:04:31 click==8.1.8
23:04:31 cryptography==44.0.2
23:04:31 decorator==5.2.1
23:04:31 distlib==0.3.9
23:04:31 docker-py==1.10.6
23:04:31 docker-pycreds==0.4.0
23:04:31 elastic-transport==8.17.1
23:04:31 elasticsearch==8.17.2
23:04:31 elasticsearch-dsl==8.17.1
23:04:31 enum34==1.1.10
23:04:31 filelock==3.18.0
23:04:31 h11==0.14.0
23:04:31 idna==3.10
23:04:31 ipaddr==2.2.0
23:04:31 isodate==0.7.2
23:04:31 jmespath==1.0.1
23:04:31 jsonpatch==1.33
23:04:31 jsonpath-rw==1.4.0
23:04:31 jsonpointer==3.0.0
23:04:31 lxml==5.3.1
23:04:31 netaddr==1.3.0
23:04:31 netifaces==0.11.0
23:04:31 outcome==1.3.0.post0
23:04:31 paramiko==3.5.1
23:04:31 platformdirs==4.3.6
23:04:31 ply==3.11
23:04:31 pyang==2.6.1
23:04:31 pyangbind==0.8.6
23:04:31 pycparser==2.22
23:04:31 pyhocon==0.3.61
23:04:31 PyNaCl==1.5.0
23:04:31 pyparsing==3.2.1
23:04:31 PySocks==1.7.1
23:04:31 python-dateutil==2.9.0.post0
23:04:31 regex==2024.11.6
23:04:31 requests==2.32.3
23:04:31 robotframework==7.2.2
23:04:31 robotframework-httplibrary==0.4.2
23:04:31 robotframework-pythonlibcore==4.4.1
23:04:31 robotframework-requests==0.9.7
23:04:31 robotframework-selenium2library==3.0.0
23:04:31 robotframework-seleniumlibrary==6.7.1
23:04:31 robotframework-sshlibrary==3.8.0
23:04:31 scapy==2.6.1
23:04:31 scp==0.15.0
23:04:31 selenium==4.29.0
23:04:31 six==1.17.0
23:04:31 sniffio==1.3.1
23:04:31 sortedcontainers==2.4.0
23:04:31 soupsieve==2.6
23:04:31 trio==0.29.0
23:04:31 trio-websocket==0.12.2
23:04:31 typing_extensions==4.12.2
23:04:31 urllib3==2.3.0
23:04:31 virtualenv==20.29.3
23:04:31 waitress==3.0.2
23:04:31 WebOb==1.8.9
23:04:31 websocket-client==1.8.0
23:04:31 WebTest==3.0.4
23:04:31 wsproto==1.2.0
23:04:31 [EnvInject] - Injecting environment variables from a build step.
23:04:31 [EnvInject] - Injecting as environment variables the properties file path 'env.properties'
23:04:31 [EnvInject] - Variables injected successfully.
23:04:31 [controller-csit-3node-clustering-ask-all-titanium] $ /bin/bash -l /tmp/jenkins13877182343643851578.sh
23:04:31 Setup pyenv:
23:04:32 system
23:04:32 3.8.13
23:04:32 3.9.13
23:04:32 3.10.13
23:04:32 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-titanium/.python-version)
23:04:32 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-Vbzb from file:/tmp/.os_lf_venv
23:04:33 lf-activate-venv(): INFO: Installing: python-heatclient python-openstackclient yq
23:04:39 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
23:04:39 lftools 0.37.11 requires urllib3<2.1.0, but you have urllib3 2.3.0 which is incompatible.
23:04:39 lf-activate-venv(): INFO: Adding /tmp/venv-Vbzb/bin to PATH
23:04:39 + ODL_SYSTEM=()
23:04:39 + TOOLS_SYSTEM=()
23:04:39 + OPENSTACK_SYSTEM=()
23:04:39 + OPENSTACK_CONTROLLERS=()
23:04:39 + mapfile -t ADDR
23:04:39 ++ openstack stack show -f json -c outputs releng-controller-csit-3node-clustering-ask-all-titanium-29
23:04:39 ++ jq -r '.outputs[] | select(.output_key | match("^vm_[0-9]+_ips$")) | .output_value | .[]'
23:04:41 + for i in "${ADDR[@]}"
23:04:41 ++ ssh 10.30.170.209 hostname -s
23:04:41 Warning: Permanently added '10.30.170.209' (ECDSA) to the list of known hosts.
23:04:42 + REMHOST=releng-19183-29-0-builder-0
23:04:42 + case ${REMHOST} in
23:04:42 + ODL_SYSTEM=("${ODL_SYSTEM[@]}" "${i}")
23:04:42 + for i in "${ADDR[@]}"
23:04:42 ++ ssh 10.30.170.81 hostname -s
23:04:42 Warning: Permanently added '10.30.170.81' (ECDSA) to the list of known hosts.
23:04:42 + REMHOST=releng-19183-29-0-builder-1
23:04:42 + case ${REMHOST} in
23:04:42 + ODL_SYSTEM=("${ODL_SYSTEM[@]}" "${i}")
23:04:42 + for i in "${ADDR[@]}"
23:04:42 ++ ssh 10.30.170.131 hostname -s
23:04:42 Warning: Permanently added '10.30.170.131' (ECDSA) to the list of known hosts.
23:04:42 + REMHOST=releng-19183-29-0-builder-2
23:04:42 + case ${REMHOST} in
23:04:42 + ODL_SYSTEM=("${ODL_SYSTEM[@]}" "${i}")
23:04:42 + echo NUM_ODL_SYSTEM=3
23:04:42 + echo NUM_TOOLS_SYSTEM=0
23:04:42 + '[' '' == yes ']'
23:04:42 + NUM_OPENSTACK_SYSTEM=0
23:04:42 + echo NUM_OPENSTACK_SYSTEM=0
23:04:42 + '[' 0 -eq 2 ']'
23:04:42 + echo ODL_SYSTEM_IP=10.30.170.209
23:04:42 ++ seq 0 2
23:04:42 + for i in $(seq 0 $(( ${#ODL_SYSTEM[@]} - 1 )))
23:04:42 + echo ODL_SYSTEM_1_IP=10.30.170.209
23:04:42 + for i in $(seq 0 $(( ${#ODL_SYSTEM[@]} - 1 )))
23:04:42 + echo ODL_SYSTEM_2_IP=10.30.170.81
23:04:42 + for i in $(seq 0 $(( ${#ODL_SYSTEM[@]} - 1 )))
23:04:42 + echo ODL_SYSTEM_3_IP=10.30.170.131
23:04:42 + echo TOOLS_SYSTEM_IP=
23:04:42 ++ seq 0 -1
23:04:42 + openstack_index=0
23:04:42 + NUM_OPENSTACK_CONTROL_NODES=1
23:04:42 + echo NUM_OPENSTACK_CONTROL_NODES=1
23:04:42 ++ seq 0 0
23:04:42 + for i in $(seq 0 $((NUM_OPENSTACK_CONTROL_NODES - 1)))
23:04:42 + echo OPENSTACK_CONTROL_NODE_1_IP=
23:04:42 + NUM_OPENSTACK_COMPUTE_NODES=-1
23:04:42 + echo NUM_OPENSTACK_COMPUTE_NODES=-1
23:04:42 + '[' -1 -ge 2 ']'
23:04:42 ++ seq 0 -2
23:04:42 + NUM_OPENSTACK_HAPROXY_NODES=0
23:04:42 + echo NUM_OPENSTACK_HAPROXY_NODES=0
23:04:42 ++ seq 0 -1
23:04:42 + echo 'Contents of slave_addresses.txt:'
23:04:42 Contents of slave_addresses.txt:
23:04:42 + cat slave_addresses.txt
23:04:42 NUM_ODL_SYSTEM=3
23:04:42 NUM_TOOLS_SYSTEM=0
23:04:42 NUM_OPENSTACK_SYSTEM=0
23:04:42 ODL_SYSTEM_IP=10.30.170.209
23:04:42 ODL_SYSTEM_1_IP=10.30.170.209
23:04:42 ODL_SYSTEM_2_IP=10.30.170.81
23:04:42 ODL_SYSTEM_3_IP=10.30.170.131
23:04:42 TOOLS_SYSTEM_IP=
23:04:42 NUM_OPENSTACK_CONTROL_NODES=1
23:04:42 OPENSTACK_CONTROL_NODE_1_IP=
23:04:42 NUM_OPENSTACK_COMPUTE_NODES=-1
23:04:42 NUM_OPENSTACK_HAPROXY_NODES=0
23:04:42 [EnvInject] - Injecting environment variables from a build step.
23:04:42 [EnvInject] - Injecting as environment variables the properties file path 'slave_addresses.txt'
23:04:42 [EnvInject] - Variables injected successfully.
23:04:42 [controller-csit-3node-clustering-ask-all-titanium] $ /bin/sh /tmp/jenkins11119839643229636149.sh
23:04:42 Preparing for JRE Version 21
23:04:42 Karaf artifact is karaf
23:04:42 Karaf project is integration
23:04:42 Java home is /usr/lib/jvm/java-21-openjdk-amd64
23:04:42 [EnvInject] - Injecting environment variables from a build step.
23:04:42 [EnvInject] - Injecting as environment variables the properties file path 'set_variables.env'
23:04:42 [EnvInject] - Variables injected successfully.
23:04:42 [controller-csit-3node-clustering-ask-all-titanium] $ /bin/bash /tmp/jenkins12752681957386683845.sh
23:04:43 2025-03-16 23:04:43 URL:https://git.opendaylight.org/gerrit/gitweb?p=integration/distribution.git;a=blob_plain;f=pom.xml;hb=master [2619/2619] -> "pom.xml" [1]
23:04:43 Bundle version is 0.22.0-SNAPSHOT
23:04:43 --2025-03-16 23:04:43-- https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.22.0-SNAPSHOT/maven-metadata.xml
23:04:43 Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6
23:04:43 Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected.
23:04:43 HTTP request sent, awaiting response... 200 OK
23:04:43 Length: 1416 (1.4K) [application/xml]
23:04:43 Saving to: ‘maven-metadata.xml’
23:04:43
23:04:43 0K . 100% 73.3M=0s
23:04:43
23:04:43 2025-03-16 23:04:43 (73.3 MB/s) - ‘maven-metadata.xml’ saved [1416/1416]
23:04:43
23:04:43 <?xml version="1.0" encoding="UTF-8"?>
23:04:43 <metadata modelVersion="1.1.0">
23:04:43 <groupId>org.opendaylight.integration</groupId>
23:04:43 <artifactId>karaf</artifactId>
23:04:43 <version>0.22.0-SNAPSHOT</version>
23:04:43 <versioning>
23:04:43 <snapshot>
23:04:43 <timestamp>20250316.175440</timestamp>
23:04:43 <buildNumber>1102</buildNumber>
23:04:43 </snapshot>
23:04:43 <lastUpdated>20250316175440</lastUpdated>
23:04:43 <snapshotVersions>
23:04:43 <snapshotVersion>
23:04:43 <extension>pom</extension>
23:04:43 <value>0.22.0-20250316.175440-1102</value>
23:04:43 <updated>20250316175440</updated>
23:04:43 </snapshotVersion>
23:04:43 <snapshotVersion>
23:04:43 <extension>tar.gz</extension>
23:04:43 <value>0.22.0-20250316.175440-1102</value>
23:04:43 <updated>20250316175440</updated>
23:04:43 </snapshotVersion>
23:04:43 <snapshotVersion>
23:04:43 <extension>zip</extension>
23:04:43 <value>0.22.0-20250316.175440-1102</value>
23:04:43 <updated>20250316175440</updated>
23:04:43 </snapshotVersion>
23:04:43 <snapshotVersion>
23:04:43 <classifier>cyclonedx</classifier>
23:04:43 <extension>xml</extension>
23:04:43 <value>0.22.0-20250316.175440-1102</value>
23:04:43 <updated>20250316175440</updated>
23:04:43 </snapshotVersion>
23:04:43 <snapshotVersion>
23:04:43 <classifier>cyclonedx</classifier>
23:04:43 <extension>json</extension>
23:04:43 <value>0.22.0-20250316.175440-1102</value>
23:04:43 <updated>20250316175440</updated>
23:04:43 </snapshotVersion>
23:04:43 </snapshotVersions>
23:04:43 </versioning>
23:04:43 </metadata>
23:04:43 Nexus timestamp is 0.22.0-20250316.175440-1102
23:04:43 Distribution bundle URL is https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.22.0-SNAPSHOT/karaf-0.22.0-20250316.175440-1102.zip
23:04:43 Distribution bundle is karaf-0.22.0-20250316.175440-1102.zip
23:04:43 Distribution bundle version is 0.22.0-SNAPSHOT
23:04:43 Distribution folder is karaf-0.22.0-SNAPSHOT
23:04:43 Nexus prefix is https://nexus.opendaylight.org
23:04:43 [EnvInject] - Injecting environment variables from a build step.
23:04:43 [EnvInject] - Injecting as environment variables the properties file path 'detect_variables.env'
23:04:43 [EnvInject] - Variables injected successfully.
23:04:43 [controller-csit-3node-clustering-ask-all-titanium] $ /bin/bash -l /tmp/jenkins5237352999356399594.sh
23:04:43 Setup pyenv:
23:04:43 system
23:04:43 3.8.13
23:04:43 3.9.13
23:04:43 3.10.13
23:04:43 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-titanium/.python-version)
23:04:44 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-Vbzb from file:/tmp/.os_lf_venv
23:04:45 lf-activate-venv(): INFO: Installing: python-heatclient python-openstackclient
23:04:48 lf-activate-venv(): INFO: Adding /tmp/venv-Vbzb/bin to PATH
23:04:48 Copying common-functions.sh to /tmp
23:04:50 Copying common-functions.sh to 10.30.170.209:/tmp
23:04:50 Warning: Permanently added '10.30.170.209' (ECDSA) to the list of known hosts.
23:04:50 Copying common-functions.sh to 10.30.170.81:/tmp
23:04:50 Warning: Permanently added '10.30.170.81' (ECDSA) to the list of known hosts.
23:04:50 Copying common-functions.sh to 10.30.170.131:/tmp
23:04:50 Warning: Permanently added '10.30.170.131' (ECDSA) to the list of known hosts.
23:04:51 [controller-csit-3node-clustering-ask-all-titanium] $ /bin/bash /tmp/jenkins6022837878182404006.sh
23:04:51 common-functions.sh is being sourced
23:04:51 common-functions environment:
23:04:51 MAVENCONF: /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:04:51 ACTUALFEATURES:
23:04:51 FEATURESCONF: /tmp/karaf-0.22.0-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:04:51 CUSTOMPROP: /tmp/karaf-0.22.0-SNAPSHOT/etc/custom.properties
23:04:51 LOGCONF: /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:04:51 MEMCONF: /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:04:51 CONTROLLERMEM: 2048m
23:04:51 AKKACONF: /tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/akka.conf
23:04:51 MODULESCONF: /tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/modules.conf
23:04:51 MODULESHARDSCONF: /tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/module-shards.conf
23:04:51 SUITES:
23:04:51
23:04:51 #################################################
23:04:51 ## Configure Cluster and Start ##
23:04:51 #################################################
23:04:51 ACTUALFEATURES: odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app
23:04:51 SPACE_SEPARATED_FEATURES: odl-integration-compatible-with-all odl-jolokia odl-restconf odl-clustering-test-app
23:04:51 Locating script plan to use...
23:04:51 script plan exists!!!
23:04:51 Changing the script plan path...
23:04:51 # Place the scripts in run order:
23:04:51 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/scripts/car_people_shard.sh
23:04:51 Executing /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/scripts/car_people_shard.sh...
23:04:51 Add car-people shards file
23:04:51 Copy shard config to member-1 with IP address 10.30.170.209
23:04:51 Warning: Permanently added '10.30.170.209' (ECDSA) to the list of known hosts.
23:04:51 Copy shard config to member-2 with IP address 10.30.170.81
23:04:51 Warning: Permanently added '10.30.170.81' (ECDSA) to the list of known hosts.
23:04:51 Copy shard config to member-3 with IP address 10.30.170.131
23:04:51 Warning: Permanently added '10.30.170.131' (ECDSA) to the list of known hosts.
23:04:51 Finished running script plans
23:04:51 Configuring member-1 with IP address 10.30.170.209
23:04:51 Warning: Permanently added '10.30.170.209' (ECDSA) to the list of known hosts.
23:04:52 Warning: Permanently added '10.30.170.209' (ECDSA) to the list of known hosts.
23:04:52 + source /tmp/common-functions.sh karaf-0.22.0-SNAPSHOT
23:04:52 ++ [[ /tmp/common-functions.sh == \/\t\m\p\/\c\o\n\f\i\g\u\r\a\t\i\o\n\-\s\c\r\i\p\t\.\s\h ]]
23:04:52 ++ echo 'common-functions.sh is being sourced'
23:04:52 common-functions.sh is being sourced
23:04:52 ++ BUNDLEFOLDER=karaf-0.22.0-SNAPSHOT
23:04:52 ++ export MAVENCONF=/tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:04:52 ++ MAVENCONF=/tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:04:52 ++ export FEATURESCONF=/tmp/karaf-0.22.0-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:04:52 ++ FEATURESCONF=/tmp/karaf-0.22.0-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:04:52 ++ export CUSTOMPROP=/tmp/karaf-0.22.0-SNAPSHOT/etc/custom.properties
23:04:52 ++ CUSTOMPROP=/tmp/karaf-0.22.0-SNAPSHOT/etc/custom.properties
23:04:52 ++ export LOGCONF=/tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:04:52 ++ LOGCONF=/tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:04:52 ++ export MEMCONF=/tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:04:52 ++ MEMCONF=/tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:04:52 ++ export CONTROLLERMEM=
23:04:52 ++ CONTROLLERMEM=
23:04:52 ++ export AKKACONF=/tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/akka.conf
23:04:52 ++ AKKACONF=/tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/akka.conf
23:04:52 ++ export MODULESCONF=/tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/modules.conf
23:04:52 ++ MODULESCONF=/tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/modules.conf
23:04:52 ++ export MODULESHARDSCONF=/tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/module-shards.conf
23:04:52 ++ MODULESHARDSCONF=/tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/module-shards.conf
23:04:52 ++ print_common_env
23:04:52 ++ cat
23:04:52 common-functions environment:
23:04:52 MAVENCONF: /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:04:52 ACTUALFEATURES:
23:04:52 FEATURESCONF: /tmp/karaf-0.22.0-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:04:52 CUSTOMPROP: /tmp/karaf-0.22.0-SNAPSHOT/etc/custom.properties
23:04:52 LOGCONF: /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:04:52 MEMCONF: /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:04:52 CONTROLLERMEM:
23:04:52 AKKACONF: /tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/akka.conf
23:04:52 MODULESCONF: /tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/modules.conf
23:04:52 MODULESHARDSCONF: /tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/module-shards.conf
23:04:52 SUITES:
23:04:52
23:04:52 ++ SSH='ssh -t -t'
23:04:52 ++ extra_services_cntl=' dnsmasq.service httpd.service libvirtd.service openvswitch.service ovs-vswitchd.service ovsdb-server.service rabbitmq-server.service '
23:04:52 ++ extra_services_cmp=' libvirtd.service openvswitch.service ovs-vswitchd.service ovsdb-server.service '
23:04:52 Changing to /tmp
23:04:52 + echo 'Changing to /tmp'
23:04:52 + cd /tmp
23:04:52 Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.22.0-SNAPSHOT/karaf-0.22.0-20250316.175440-1102.zip
23:04:52 + echo 'Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.22.0-SNAPSHOT/karaf-0.22.0-20250316.175440-1102.zip'
23:04:52 + wget --progress=dot:mega https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.22.0-SNAPSHOT/karaf-0.22.0-20250316.175440-1102.zip
23:04:52 --2025-03-16 23:04:52-- https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.22.0-SNAPSHOT/karaf-0.22.0-20250316.175440-1102.zip
23:04:52 Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6
23:04:52 Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected.
23:04:52 HTTP request sent, awaiting response... 200 OK
23:04:52 Length: 236097025 (225M) [application/zip]
23:04:52 Saving to: ‘karaf-0.22.0-20250316.175440-1102.zip’
23:04:52
23:04:52 0K ........ ........ ........ ........ ........ ........ 1% 58.3M 4s
23:04:52 3072K ........ ........ ........ ........ ........ ........ 2% 94.7M 3s
23:04:52 6144K ........ ........ ........ ........ ........ ........ 3% 102M 3s
23:04:52 9216K ........ ........ ........ ........ ........ ........ 5% 136M 2s
23:04:52 12288K ........ ........ ........ ........ ........ ........ 6% 127M 2s
23:04:52 15360K ........ ........ ........ ........ ........ ........ 7% 160M 2s
23:04:52 18432K ........ ........ ........ ........ ........ ........ 9% 152M 2s
23:04:52 21504K ........ ........ ........ ........ ........ ........ 10% 251M 2s
23:04:52 24576K ........ ........ ........ ........ ........ ........ 11% 227M 2s
23:04:52 27648K ........ ........ ........ ........ ........ ........ 13% 239M 2s
23:04:52 30720K ........ ........ ........ ........ ........ ........ 14% 263M 1s
23:04:52 33792K ........ ........ ........ ........ ........ ........ 15% 255M 1s
23:04:52 36864K ........ ........ ........ ........ ........ ........ 17% 259M 1s
23:04:52 39936K ........ ........ ........ ........ ........ ........ 18% 295M 1s
23:04:52 43008K ........ ........ ........ ........ ........ ........ 19% 347M 1s
23:04:52 46080K ........ ........ ........ ........ ........ ........ 21% 302M 1s
23:04:52 49152K ........ ........ ........ ........ ........ ........ 22% 364M 1s
23:04:52 52224K ........ ........ ........ ........ ........ ........ 23% 360M 1s
23:04:52 55296K ........ ........ ........ ........ ........ ........ 25% 355M 1s
23:04:52 58368K ........ ........ ........ ........ ........ ........ 26% 347M 1s
23:04:52 61440K ........ ........ ........ ........ ........ ........ 27% 222M 1s
23:04:52 64512K ........ ........ ........ ........ ........ ........ 29% 309M 1s
23:04:52 67584K ........ ........ ........ ........ ........ ........ 30% 336M 1s
23:04:52 70656K ........ ........ ........ ........ ........ ........ 31% 360M 1s
23:04:52 73728K ........ ........ ........ ........ ........ ........ 33% 342M 1s
23:04:52 76800K ........ ........ ........ ........ ........ ........ 34% 341M 1s
23:04:52 79872K ........ ........ ........ ........ ........ ........ 35% 345M 1s
23:04:52 82944K ........ ........ ........ ........ ........ ........ 37% 343M 1s
23:04:52 86016K ........ ........ ........ ........ ........ ........ 38% 313M 1s
23:04:52 89088K ........ ........ ........ ........ ........ ........ 39% 360M 1s
23:04:52 92160K ........ ........ ........ ........ ........ ........ 41% 347M 1s
23:04:52 95232K ........ ........ ........ ........ ........ ........ 42% 344M 1s
23:04:52 98304K ........ ........ ........ ........ ........ ........ 43% 333M 1s
23:04:52 101376K ........ ........ ........ ........ ........ ........ 45% 343M 1s
23:04:52 104448K ........ ........ ........ ........ ........ ........ 46% 341M 1s
23:04:52 107520K ........ ........ ........ ........ ........ ........ 47% 354M 1s
23:04:52 110592K ........ ........ ........ ........ ........ ........ 49% 342M 0s
23:04:52 113664K ........ ........ ........ ........ ........ ........ 50% 346M 0s
23:04:52 116736K ........ ........ ........ ........ ........ ........ 51% 336M 0s
23:04:52 119808K ........ ........ ........ ........ ........ ........ 53% 346M 0s
23:04:52 122880K ........ ........ ........ ........ ........ ........ 54% 320M 0s
23:04:52 125952K ........ ........ ........ ........ ........ ........ 55% 341M 0s
23:04:52 129024K ........ ........ ........ ........ ........ ........ 57% 322M 0s
23:04:52 132096K ........ ........ ........ ........ ........ ........ 58% 345M 0s
23:04:52 135168K ........ ........ ........ ........ ........ ........ 59% 338M 0s
23:04:52 138240K ........ ........ ........ ........ ........ ........ 61% 350M 0s
23:04:52 141312K ........ ........ ........ ........ ........ ........ 62% 332M 0s
23:04:52 144384K ........ ........ ........ ........ ........ ........ 63% 347M 0s
23:04:52 147456K ........ ........ ........ ........ ........ ........ 65% 339M 0s
23:04:52 150528K ........ ........ ........ ........ ........ ........ 66% 348M 0s
23:04:52 153600K ........ ........ ........ ........ ........ ........ 67% 332M 0s
23:04:52 156672K ........ ........ ........ ........ ........ ........ 69% 338M 0s
23:04:52 159744K ........ ........ ........ ........ ........ ........ 70% 325M 0s
23:04:52 162816K ........ ........ ........ ........ ........ ........ 71% 330M 0s
23:04:52 165888K ........ ........ ........ ........ ........ ........ 73% 326M 0s
23:04:52 168960K ........ ........ ........ ........ ........ ........ 74% 328M 0s
23:04:53 172032K ........ ........ ........ ........ ........ ........ 75% 322M 0s
23:04:53 175104K ........ ........ ........ ........ ........ ........ 77% 337M 0s
23:04:53 178176K ........ ........ ........ ........ ........ ........ 78% 313M 0s
23:04:53 181248K ........ ........ ........ ........ ........ ........ 79% 323M 0s
23:04:53 184320K ........ ........ ........ ........ ........ ........ 81% 317M 0s
23:04:53 187392K ........ ........ ........ ........ ........ ........ 82% 325M 0s
23:04:53 190464K ........ ........ ........ ........ ........ ........ 83% 322M 0s
23:04:53 193536K ........ ........ ........ ........ ........ ........ 85% 336M 0s
23:04:53 196608K ........ ........ ........ ........ ........ ........ 86% 328M 0s
23:04:53 199680K ........ ........ ........ ........ ........ ........ 87% 325M 0s
23:04:53 202752K ........ ........ ........ ........ ........ ........ 89% 333M 0s
23:04:53 205824K ........ ........ ........ ........ ........ ........ 90% 323M 0s
23:04:53 208896K ........ ........ ........ ........ ........ ........ 91% 330M 0s
23:04:53 211968K ........ ........ ........ ........ ........ ........ 93% 327M 0s
23:04:53 215040K ........ ........ ........ ........ ........ ........ 94% 318M 0s
23:04:53 218112K ........ ........ ........ ........ ........ ........ 95% 324M 0s
23:04:53 221184K ........ ........ ........ ........ ........ ........ 97% 308M 0s
23:04:53 224256K ........ ........ ........ ........ ........ ........ 98% 324M 0s
23:04:53 227328K ........ ........ ........ ........ ........ ........ 99% 296M 0s
23:04:53 230400K .. 100% 534M=0.8s
23:04:53
23:04:53 2025-03-16 23:04:53 (270 MB/s) - ‘karaf-0.22.0-20250316.175440-1102.zip’ saved [236097025/236097025]
23:04:53
23:04:53 Extracting the new controller...
23:04:53 + echo 'Extracting the new controller...'
23:04:53 + unzip -q karaf-0.22.0-20250316.175440-1102.zip
23:04:55 Adding external repositories...
23:04:55 + echo 'Adding external repositories...'
23:04:55 + sed -ie 's%org.ops4j.pax.url.mvn.repositories=%org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases%g' /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:04:55 + cat /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:04:55 ################################################################################
23:04:55 #
23:04:55 # Licensed to the Apache Software Foundation (ASF) under one or more
23:04:55 # contributor license agreements. See the NOTICE file distributed with
23:04:55 # this work for additional information regarding copyright ownership.
23:04:55 # The ASF licenses this file to You under the Apache License, Version 2.0
23:04:55 # (the "License"); you may not use this file except in compliance with
23:04:55 # the License. You may obtain a copy of the License at
23:04:55 #
23:04:55 # http://www.apache.org/licenses/LICENSE-2.0
23:04:55 #
23:04:55 # Unless required by applicable law or agreed to in writing, software
23:04:55 # distributed under the License is distributed on an "AS IS" BASIS,
23:04:55 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23:04:55 # See the License for the specific language governing permissions and
23:04:55 # limitations under the License.
23:04:55 #
23:04:55 ################################################################################
23:04:55
23:04:55 #
23:04:55 # If set to true, the following property will not allow any certificate to be used
23:04:55 # when accessing Maven repositories through SSL
23:04:55 #
23:04:55 #org.ops4j.pax.url.mvn.certificateCheck=
23:04:55
23:04:55 #
23:04:55 # Path to the local Maven settings file.
23:04:55 # The repositories defined in this file will be automatically added to the list
23:04:55 # of default repositories if the 'org.ops4j.pax.url.mvn.repositories' property
23:04:55 # below is not set.
23:04:55 # The following locations are checked for the existence of the settings.xml file
23:04:55 # * 1. looks for the specified url
23:04:55 # * 2. if not found looks for ${user.home}/.m2/settings.xml
23:04:55 # * 3. if not found looks for ${maven.home}/conf/settings.xml
23:04:55 # * 4. if not found looks for ${M2_HOME}/conf/settings.xml
23:04:55 #
23:04:55 #org.ops4j.pax.url.mvn.settings=
23:04:55
23:04:55 #
23:04:55 # Path to the local Maven repository which is used to avoid downloading
23:04:55 # artifacts when they already exist locally.
23:04:55 # The value of this property will be extracted from the settings.xml file
23:04:55 # above, or defaulted to:
23:04:55 # System.getProperty( "user.home" ) + "/.m2/repository"
23:04:55 #
23:04:55 org.ops4j.pax.url.mvn.localRepository=${karaf.home}/${karaf.default.repository}
23:04:55
23:04:55 #
23:04:55 # Default this to false. It's just weird to use undocumented repos
23:04:55 #
23:04:55 org.ops4j.pax.url.mvn.useFallbackRepositories=false
23:04:55
23:04:55 #
23:04:55 # Uncomment if you don't wanna use the proxy settings
23:04:55 # from the Maven conf/settings.xml file
23:04:55 #
23:04:55 # org.ops4j.pax.url.mvn.proxySupport=false
23:04:55
23:04:55 #
23:04:55 # Comma separated list of repositories scanned when resolving an artifact.
23:04:55 # Those repositories will be checked before iterating through the
23:04:55 # below list of repositories and even before the local repository
23:04:55 # A repository url can be appended with zero or more of the following flags:
23:04:55 # @snapshots : the repository contains snaphots
23:04:55 # @noreleases : the repository does not contain any released artifacts
23:04:55 #
23:04:55 # The following property value will add the system folder as a repo.
23:04:55 #
23:04:55 org.ops4j.pax.url.mvn.defaultRepositories=\
23:04:55 file:${karaf.home}/${karaf.default.repository}@id=system.repository@snapshots,\
23:04:55 file:${karaf.data}/kar@id=kar.repository@multi@snapshots,\
23:04:55 file:${karaf.base}/${karaf.default.repository}@id=child.system.repository@snapshots
23:04:55
23:04:55 # Use the default local repo (e.g.~/.m2/repository) as a "remote" repo
23:04:55 #org.ops4j.pax.url.mvn.defaultLocalRepoAsRemote=false
23:04:55
23:04:55 #
23:04:55 # Comma separated list of repositories scanned when resolving an artifact.
23:04:55 # The default list includes the following repositories:
23:04:55 # http://repo1.maven.org/maven2@id=central
23:04:55 # http://repository.springsource.com/maven/bundles/release@id=spring.ebr
23:04:55 # http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external
23:04:55 # http://zodiac.springsource.com/maven/bundles/release@id=gemini
23:04:55 # http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases
23:04:55 # https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases
23:04:55 # https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases
23:04:55 # To add repositories to the default ones, prepend '+' to the list of repositories
23:04:55 # to add.
23:04:55 # A repository url can be appended with zero or more of the following flags:
23:04:55 # @snapshots : the repository contains snapshots
23:04:55 # @noreleases : the repository does not contain any released artifacts
23:04:55 # @id=repository.id : the id for the repository, just like in the settings.xml this is optional but recommended
23:04:55 #
23:04:55 org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases
23:04:55
23:04:55 ### ^^^ No remote repositories. This is the only ODL change compared to Karaf defaults.Configuring the startup features...
23:04:55 + [[ True == \T\r\u\e ]]
23:04:55 + echo 'Configuring the startup features...'
23:04:55 + sed -ie 's/\(featuresBoot=\|featuresBoot =\)/featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app,/g' /tmp/karaf-0.22.0-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:04:55 + FEATURE_TEST_STRING=features-test
23:04:55 + FEATURE_TEST_VERSION=0.22.0-SNAPSHOT
23:04:55 + KARAF_VERSION=karaf4
23:04:55 + [[ integration == \i\n\t\e\g\r\a\t\i\o\n ]]
23:04:55 + sed -ie 's%\(featuresRepositories=\|featuresRepositories =\)%featuresRepositories = mvn:org.opendaylight.integration/features-test/0.22.0-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features,%g' /tmp/karaf-0.22.0-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:04:55 + [[ ! -z '' ]]
23:04:55 + cat /tmp/karaf-0.22.0-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:04:55 ################################################################################
23:04:55 #
23:04:55 # Licensed to the Apache Software Foundation (ASF) under one or more
23:04:55 # contributor license agreements. See the NOTICE file distributed with
23:04:55 # this work for additional information regarding copyright ownership.
23:04:55 # The ASF licenses this file to You under the Apache License, Version 2.0
23:04:55 # (the "License"); you may not use this file except in compliance with
23:04:55 # the License. You may obtain a copy of the License at
23:04:55 #
23:04:55 # http://www.apache.org/licenses/LICENSE-2.0
23:04:55 #
23:04:55 # Unless required by applicable law or agreed to in writing, software
23:04:55 # distributed under the License is distributed on an "AS IS" BASIS,
23:04:55 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23:04:55 # See the License for the specific language governing permissions and
23:04:55 # limitations under the License.
23:04:55 #
23:04:55 ################################################################################
23:04:55
23:04:55 #
23:04:55 # Comma separated list of features repositories to register by default
23:04:55 #
23:04:55 featuresRepositories = mvn:org.opendaylight.integration/features-test/0.22.0-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features, file:${karaf.etc}/1a8c7569-3af0-46ba-a718-f10d85096372.xml
23:04:55
23:04:55 #
23:04:55 # Comma separated list of features to install at startup
23:04:55 #
23:04:55 featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app, c5929223-d569-4724-9c23-39fc71743903
23:04:55
23:04:55 #
23:04:55 # Resource repositories (OBR) that the features resolver can use
23:04:55 # to resolve requirements/capabilities
23:04:55 #
23:04:55 # The format of the resourceRepositories is
23:04:55 # resourceRepositories=[xml:url|json:url],...
23:04:55 # for Instance:
23:04:55 #
23:04:55 #resourceRepositories=xml:http://host/path/to/index.xml
23:04:55 # or
23:04:55 #resourceRepositories=json:http://host/path/to/index.json
23:04:55 #
23:04:55
23:04:55 #
23:04:55 # Defines if the boot features are started in asynchronous mode (in a dedicated thread)
23:04:55 #
23:04:55 featuresBootAsynchronous=false
23:04:55
23:04:55 #
23:04:55 # Service requirements enforcement
23:04:55 #
23:04:55 # By default, the feature resolver checks the service requirements/capabilities of
23:04:55 # bundles for new features (xml schema >= 1.3.0) in order to automatically installs
23:04:55 # the required bundles.
23:04:55 # The following flag can have those values:
23:04:55 # - disable: service requirements are completely ignored
23:04:55 # - default: service requirements are ignored for old features
23:04:55 # - enforce: service requirements are always verified
23:04:55 #
23:04:55 #serviceRequirements=default
23:04:55
23:04:55 #
23:04:55 # Store cfg file for config element in feature
23:04:55 #
23:04:55 #configCfgStore=true
23:04:55
23:04:55 #
23:04:55 # Define if the feature service automatically refresh bundles
23:04:55 #
23:04:55 autoRefresh=true
23:04:55
23:04:55 #
23:04:55 # Configuration of features processing mechanism (overrides, blacklisting, modification of features)
23:04:55 # XML file defines instructions related to features processing
23:04:55 # versions.properties may declare properties to resolve placeholders in XML file
23:04:55 # both files are relative to ${karaf.etc}
23:04:55 #
23:04:55 #featureProcessing=org.apache.karaf.features.xml
23:04:55 #featureProcessingVersions=versions.properties
23:04:55 + configure_karaf_log karaf4 ''
23:04:55 + local -r karaf_version=karaf4
23:04:55 + local -r controllerdebugmap=
23:04:55 + local logapi=log4j
23:04:55 + grep log4j2 /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:04:55 log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
23:04:55 log4j2.rootLogger.level = INFO
23:04:55 #log4j2.rootLogger.type = asyncRoot
23:04:55 #log4j2.rootLogger.includeLocation = false
23:04:55 log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile
23:04:55 log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi
23:04:55 log4j2.rootLogger.appenderRef.Console.ref = Console
23:04:55 log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter
23:04:55 log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF}
23:04:55 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter
23:04:55 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair
23:04:55 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker
23:04:55 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL
23:04:55 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or
23:04:55 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY
23:04:55 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL
23:04:55 log4j2.logger.spifly.name = org.apache.aries.spifly
23:04:55 log4j2.logger.spifly.level = WARN
23:04:55 log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit
23:04:55 log4j2.logger.audit.level = INFO
23:04:55 log4j2.logger.audit.additivity = false
23:04:55 log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile
23:04:55 # Console appender not used by default (see log4j2.rootLogger.appenderRefs)
23:04:55 log4j2.appender.console.type = Console
23:04:55 log4j2.appender.console.name = Console
23:04:55 log4j2.appender.console.layout.type = PatternLayout
23:04:55 log4j2.appender.console.layout.pattern = ${log4j2.pattern}
23:04:55 log4j2.appender.rolling.type = RollingRandomAccessFile
23:04:55 log4j2.appender.rolling.name = RollingFile
23:04:55 log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
23:04:55 log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i
23:04:55 #log4j2.appender.rolling.immediateFlush = false
23:04:55 log4j2.appender.rolling.append = true
23:04:55 log4j2.appender.rolling.layout.type = PatternLayout
23:04:55 log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
23:04:55 log4j2.appender.rolling.policies.type = Policies
23:04:55 log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
23:04:55 log4j2.appender.rolling.policies.size.size = 64MB
23:04:55 log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
23:04:55 log4j2.appender.rolling.strategy.max = 7
23:04:55 log4j2.appender.audit.type = RollingRandomAccessFile
23:04:55 log4j2.appender.audit.name = AuditRollingFile
23:04:55 log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log
23:04:55 log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i
23:04:55 log4j2.appender.audit.append = true
23:04:55 log4j2.appender.audit.layout.type = PatternLayout
23:04:55 log4j2.appender.audit.layout.pattern = ${log4j2.pattern}
23:04:55 log4j2.appender.audit.policies.type = Policies
23:04:55 log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy
23:04:55 log4j2.appender.audit.policies.size.size = 8MB
23:04:55 log4j2.appender.audit.strategy.type = DefaultRolloverStrategy
23:04:55 log4j2.appender.audit.strategy.max = 7
23:04:55 log4j2.appender.osgi.type = PaxOsgi
23:04:55 log4j2.appender.osgi.name = PaxOsgi
23:04:55 log4j2.appender.osgi.filter = *
23:04:55 #log4j2.logger.aether.name = shaded.org.eclipse.aether
23:04:55 #log4j2.logger.aether.level = TRACE
23:04:55 #log4j2.logger.http-headers.name = shaded.org.apache.http.headers
23:04:55 #log4j2.logger.http-headers.level = DEBUG
23:04:55 #log4j2.logger.maven.name = org.ops4j.pax.url.mvn
23:04:55 #log4j2.logger.maven.level = TRACE
23:04:55 + logapi=log4j2
23:04:55 + echo 'Configuring the karaf log... karaf_version: karaf4, logapi: log4j2'
23:04:55 Configuring the karaf log... karaf_version: karaf4, logapi: log4j2
23:04:55 + '[' log4j2 == log4j2 ']'
23:04:55 + sed -ie 's/log4j2.appender.rolling.policies.size.size = 64MB/log4j2.appender.rolling.policies.size.size = 1GB/g' /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:04:55 + orgmodule=org.opendaylight.yangtools.yang.parser.repo.YangTextSchemaContextResolver
23:04:55 + orgmodule_=org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver
23:04:55 + echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN'
23:04:55 + echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN'
23:04:55 controllerdebugmap:
23:04:55 + unset IFS
23:04:55 + echo 'controllerdebugmap: '
23:04:55 + '[' -n '' ']'
23:04:55 + echo 'cat /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg'
23:04:55 cat /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:04:55 + cat /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:04:55 ################################################################################
23:04:55 #
23:04:55 # Licensed to the Apache Software Foundation (ASF) under one or more
23:04:55 # contributor license agreements. See the NOTICE file distributed with
23:04:55 # this work for additional information regarding copyright ownership.
23:04:55 # The ASF licenses this file to You under the Apache License, Version 2.0
23:04:55 # (the "License"); you may not use this file except in compliance with
23:04:55 # the License. You may obtain a copy of the License at
23:04:55 #
23:04:55 # http://www.apache.org/licenses/LICENSE-2.0
23:04:55 #
23:04:55 # Unless required by applicable law or agreed to in writing, software
23:04:55 # distributed under the License is distributed on an "AS IS" BASIS,
23:04:55 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23:04:55 # See the License for the specific language governing permissions and
23:04:55 # limitations under the License.
23:04:55 #
23:04:55 ################################################################################
23:04:55
23:04:55 # Common pattern layout for appenders
23:04:55 log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
23:04:55
23:04:55 # Root logger
23:04:55 log4j2.rootLogger.level = INFO
23:04:55 # uncomment to use asynchronous loggers, which require mvn:com.lmax/disruptor/3.3.2 library
23:04:55 #log4j2.rootLogger.type = asyncRoot
23:04:55 #log4j2.rootLogger.includeLocation = false
23:04:55 log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile
23:04:55 log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi
23:04:55 log4j2.rootLogger.appenderRef.Console.ref = Console
23:04:55 log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter
23:04:55 log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF}
23:04:55
23:04:55 # Filters for logs marked by org.opendaylight.odlparent.Markers
23:04:55 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter
23:04:55 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair
23:04:55 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker
23:04:55 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL
23:04:55 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or
23:04:55 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY
23:04:55 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL
23:04:55
23:04:55 # Loggers configuration
23:04:55
23:04:55 # Spifly logger
23:04:55 log4j2.logger.spifly.name = org.apache.aries.spifly
23:04:55 log4j2.logger.spifly.level = WARN
23:04:55
23:04:55 # Security audit logger
23:04:55 log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit
23:04:55 log4j2.logger.audit.level = INFO
23:04:55 log4j2.logger.audit.additivity = false
23:04:55 log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile
23:04:55
23:04:55 # Appenders configuration
23:04:55
23:04:55 # Console appender not used by default (see log4j2.rootLogger.appenderRefs)
23:04:55 log4j2.appender.console.type = Console
23:04:55 log4j2.appender.console.name = Console
23:04:55 log4j2.appender.console.layout.type = PatternLayout
23:04:55 log4j2.appender.console.layout.pattern = ${log4j2.pattern}
23:04:55
23:04:55 # Rolling file appender
23:04:55 log4j2.appender.rolling.type = RollingRandomAccessFile
23:04:55 log4j2.appender.rolling.name = RollingFile
23:04:55 log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
23:04:55 log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i
23:04:55 # uncomment to not force a disk flush
23:04:55 #log4j2.appender.rolling.immediateFlush = false
23:04:55 log4j2.appender.rolling.append = true
23:04:55 log4j2.appender.rolling.layout.type = PatternLayout
23:04:55 log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
23:04:55 log4j2.appender.rolling.policies.type = Policies
23:04:55 log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
23:04:55 log4j2.appender.rolling.policies.size.size = 1GB
23:04:55 log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
23:04:55 log4j2.appender.rolling.strategy.max = 7
23:04:55
23:04:55 # Audit file appender
23:04:55 log4j2.appender.audit.type = RollingRandomAccessFile
23:04:55 log4j2.appender.audit.name = AuditRollingFile
23:04:55 log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log
23:04:55 log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i
23:04:55 log4j2.appender.audit.append = true
23:04:55 log4j2.appender.audit.layout.type = PatternLayout
23:04:55 log4j2.appender.audit.layout.pattern = ${log4j2.pattern}
23:04:55 log4j2.appender.audit.policies.type = Policies
23:04:55 log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy
23:04:55 log4j2.appender.audit.policies.size.size = 8MB
23:04:55 log4j2.appender.audit.strategy.type = DefaultRolloverStrategy
23:04:55 log4j2.appender.audit.strategy.max = 7
23:04:55
23:04:55 # OSGi appender
23:04:55 log4j2.appender.osgi.type = PaxOsgi
23:04:55 log4j2.appender.osgi.name = PaxOsgi
23:04:55 log4j2.appender.osgi.filter = *
23:04:55
23:04:55 # help with identification of maven-related problems with pax-url-aether
23:04:55 #log4j2.logger.aether.name = shaded.org.eclipse.aether
23:04:55 #log4j2.logger.aether.level = TRACE
23:04:55 #log4j2.logger.http-headers.name = shaded.org.apache.http.headers
23:04:55 #log4j2.logger.http-headers.level = DEBUG
23:04:55 #log4j2.logger.maven.name = org.ops4j.pax.url.mvn
23:04:55 #log4j2.logger.maven.level = TRACE
23:04:55 log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN
23:04:55 log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN
23:04:55 Configure
23:04:55 java home: /usr/lib/jvm/java-21-openjdk-amd64
23:04:55 max memory: 3072m
23:04:55 memconf: /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:04:55 + set_java_vars /usr/lib/jvm/java-21-openjdk-amd64 3072m /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:04:55 + local -r java_home=/usr/lib/jvm/java-21-openjdk-amd64
23:04:55 + local -r controllermem=3072m
23:04:55 + local -r memconf=/tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:04:55 + echo Configure
23:04:55 + echo ' java home: /usr/lib/jvm/java-21-openjdk-amd64'
23:04:55 + echo ' max memory: 3072m'
23:04:55 + echo ' memconf: /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv'
23:04:55 + sed -ie 's%^# export JAVA_HOME%export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-21-openjdk-amd64}%g' /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:04:55 + sed -ie 's/JAVA_MAX_MEM="2048m"/JAVA_MAX_MEM=3072m/g' /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:04:55 cat /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:04:55 + echo 'cat /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv'
23:04:55 + cat /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:04:55 #!/bin/sh
23:04:55 #
23:04:55 # Licensed to the Apache Software Foundation (ASF) under one or more
23:04:55 # contributor license agreements. See the NOTICE file distributed with
23:04:55 # this work for additional information regarding copyright ownership.
23:04:55 # The ASF licenses this file to You under the Apache License, Version 2.0
23:04:55 # (the "License"); you may not use this file except in compliance with
23:04:55 # the License. You may obtain a copy of the License at
23:04:55 #
23:04:55 # http://www.apache.org/licenses/LICENSE-2.0
23:04:55 #
23:04:55 # Unless required by applicable law or agreed to in writing, software
23:04:55 # distributed under the License is distributed on an "AS IS" BASIS,
23:04:55 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23:04:55 # See the License for the specific language governing permissions and
23:04:55 # limitations under the License.
23:04:55 #
23:04:55
23:04:55 #
23:04:55 # handle specific scripts; the SCRIPT_NAME is exactly the name of the Karaf
23:04:55 # script: client, instance, shell, start, status, stop, karaf
23:04:55 #
23:04:55 # if [ "${KARAF_SCRIPT}" == "SCRIPT_NAME" ]; then
23:04:55 # Actions go here...
23:04:55 # fi
23:04:55
23:04:55 #
23:04:55 # general settings which should be applied for all scripts go here; please keep
23:04:55 # in mind that it is possible that scripts might be executed more than once, e.g.
23:04:55 # in example of the start script where the start script is executed first and the
23:04:55 # karaf script afterwards.
23:04:55 #
23:04:55
23:04:55 #
23:04:55 # The following section shows the possible configuration options for the default
23:04:55 # karaf scripts
23:04:55 #
23:04:55 export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-21-openjdk-amd64} # Location of Java installation
23:04:55 # export JAVA_OPTS # Generic JVM options, for instance, where you can pass the memory configuration
23:04:55 # export JAVA_NON_DEBUG_OPTS # Additional non-debug JVM options
23:04:55 # export EXTRA_JAVA_OPTS # Additional JVM options
23:04:55 # export KARAF_HOME # Karaf home folder
23:04:55 # export KARAF_DATA # Karaf data folder
23:04:55 # export KARAF_BASE # Karaf base folder
23:04:55 # export KARAF_ETC # Karaf etc folder
23:04:55 # export KARAF_LOG # Karaf log folder
23:04:55 # export KARAF_SYSTEM_OPTS # First citizen Karaf options
23:04:55 # export KARAF_OPTS # Additional available Karaf options
23:04:55 # export KARAF_DEBUG # Enable debug mode
23:04:55 # export KARAF_REDIRECT # Enable/set the std/err redirection when using bin/start
23:04:55 # export KARAF_NOROOT # Prevent execution as root if set to true
23:04:55 Set Java version
23:04:55 + echo 'Set Java version'
23:04:55 + sudo /usr/sbin/alternatives --install /usr/bin/java java /usr/lib/jvm/java-21-openjdk-amd64/bin/java 1
23:04:55 sudo: /usr/sbin/alternatives: command not found
23:04:55 + sudo /usr/sbin/alternatives --set java /usr/lib/jvm/java-21-openjdk-amd64/bin/java
23:04:55 sudo: /usr/sbin/alternatives: command not found
23:04:55 + echo 'JDK default version ...'
23:04:55 JDK default version ...
23:04:55 + java -version
23:04:55 openjdk version "11.0.25" 2024-10-15
23:04:55 OpenJDK Runtime Environment (build 11.0.25+9-post-Ubuntu-1ubuntu120.04)
23:04:55 OpenJDK 64-Bit Server VM (build 11.0.25+9-post-Ubuntu-1ubuntu120.04, mixed mode, sharing)
23:04:55 Set JAVA_HOME
23:04:55 + echo 'Set JAVA_HOME'
23:04:55 + export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
23:04:55 + JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
23:04:55 ++ readlink -e /usr/lib/jvm/java-21-openjdk-amd64/bin/java
23:04:55 + JAVA_RESOLVED=/usr/lib/jvm/java-21-openjdk-amd64/bin/java
23:04:55 Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-21-openjdk-amd64/bin/java
23:04:55 + echo 'Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-21-openjdk-amd64/bin/java'
23:04:55 Listing all open ports on controller system...
23:04:55 + echo 'Listing all open ports on controller system...'
23:04:55 + netstat -pnatu
23:04:55 /tmp/configuration-script.sh: line 40: netstat: command not found
23:04:55 Custom shard config exists!!!
23:04:55 Copying the shard config...
23:04:55 + '[' -f /tmp/custom_shard_config.txt ']'
23:04:55 + echo 'Custom shard config exists!!!'
23:04:55 + echo 'Copying the shard config...'
23:04:55 + cp /tmp/custom_shard_config.txt /tmp/karaf-0.22.0-SNAPSHOT/bin/
23:04:55 Configuring cluster
23:04:55 + echo 'Configuring cluster'
23:04:55 + /tmp/karaf-0.22.0-SNAPSHOT/bin/configure_cluster.sh 1 10.30.170.209 10.30.170.81 10.30.170.131
23:04:55 ################################################
23:04:55 ## Configure Cluster ##
23:04:55 ################################################
23:04:55 NOTE: Cluster configuration files not found. Copying from
23:04:55 /tmp/karaf-0.22.0-SNAPSHOT/system/org/opendaylight/controller/sal-clustering-config/10.0.2
23:04:55 Configuring unique name in akka.conf
23:04:55 Configuring hostname in akka.conf
23:04:55 Configuring data and rpc seed nodes in akka.conf
23:04:55 modules = [
23:04:55
23:04:55 {
23:04:55 name = "inventory"
23:04:55 namespace = "urn:opendaylight:inventory"
23:04:55 shard-strategy = "module"
23:04:55 },
23:04:55 {
23:04:55 name = "topology"
23:04:55 namespace = "urn:TBD:params:xml:ns:yang:network-topology"
23:04:55 shard-strategy = "module"
23:04:55 },
23:04:55 {
23:04:55 name = "toaster"
23:04:55 namespace = "http://netconfcentral.org/ns/toaster"
23:04:55 shard-strategy = "module"
23:04:55 },
23:04:55 {
23:04:55 name = "car"
23:04:55 namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car"
23:04:55 shard-strategy = "module"
23:04:55 },
23:04:55 {
23:04:55 name = "people"
23:04:55 namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people"
23:04:55 shard-strategy = "module"
23:04:55 },
23:04:55 {
23:04:55 name = "car-people"
23:04:55 namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people"
23:04:55 shard-strategy = "module"
23:04:55 }
23:04:55 ]
23:04:55 Configuring replication type in module-shards.conf
23:04:55 ################################################
23:04:55 ## NOTE: Manually restart controller to ##
23:04:55 ## apply configuration. ##
23:04:55 ################################################
23:04:55 Dump akka.conf
23:04:55 + echo 'Dump akka.conf'
23:04:55 + cat /tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/akka.conf
23:04:55
23:04:55 odl-cluster-data {
23:04:55 akka {
23:04:55 remote {
23:04:55 artery {
23:04:55 enabled = on
23:04:55 transport = tcp
23:04:55 canonical.hostname = "10.30.170.209"
23:04:55 canonical.port = 2550
23:04:55 }
23:04:55 }
23:04:55
23:04:55 cluster {
23:04:55 # Using artery.
23:04:55 seed-nodes = ["akka://opendaylight-cluster-data@10.30.170.209:2550",
23:04:55 "akka://opendaylight-cluster-data@10.30.170.81:2550",
23:04:55 "akka://opendaylight-cluster-data@10.30.170.131:2550"]
23:04:55
23:04:55 roles = ["member-1"]
23:04:55
23:04:55 # when under load we might trip a false positive on the failure detector
23:04:55 # failure-detector {
23:04:55 # heartbeat-interval = 4 s
23:04:55 # acceptable-heartbeat-pause = 16s
23:04:55 # }
23:04:55 }
23:04:55
23:04:55 persistence {
23:04:55 # By default the snapshots/journal directories live in KARAF_HOME. You can choose to put it somewhere else by
23:04:55 # modifying the following two properties. The directory location specified may be a relative or absolute path.
23:04:55 # The relative path is always relative to KARAF_HOME.
23:04:55
23:04:55 # snapshot-store.local.dir = "target/snapshots"
23:04:55
23:04:55 # Use lz4 compression for LocalSnapshotStore snapshots
23:04:55 snapshot-store.local.use-lz4-compression = false
23:04:55 # Size of blocks for lz4 compression: 64KB, 256KB, 1MB or 4MB
23:04:55 snapshot-store.local.lz4-blocksize = 256KB
23:04:55 }
23:04:55 disable-default-actor-system-quarantined-event-handling = "false"
23:04:55 }
23:04:55 }
23:04:55 Dump modules.conf
23:04:55 + echo 'Dump modules.conf'
23:04:55 + cat /tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/modules.conf
23:04:55 modules = [
23:04:55
23:04:55 {
23:04:55 name = "inventory"
23:04:55 namespace = "urn:opendaylight:inventory"
23:04:55 shard-strategy = "module"
23:04:55 },
23:04:55 {
23:04:55 name = "topology"
23:04:55 namespace = "urn:TBD:params:xml:ns:yang:network-topology"
23:04:55 shard-strategy = "module"
23:04:55 },
23:04:55 {
23:04:55 name = "toaster"
23:04:55 namespace = "http://netconfcentral.org/ns/toaster"
23:04:55 shard-strategy = "module"
23:04:55 },
23:04:55 {
23:04:55 name = "car"
23:04:55 namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car"
23:04:55 shard-strategy = "module"
23:04:55 },
23:04:55 {
23:04:55 name = "people"
23:04:55 namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people"
23:04:55 shard-strategy = "module"
23:04:55 },
23:04:55 {
23:04:55 name = "car-people"
23:04:55 namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people"
23:04:55 shard-strategy = "module"
23:04:55 }
23:04:55 ]
23:04:55 Dump module-shards.conf
23:04:55 + echo 'Dump module-shards.conf'
23:04:55 + cat /tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/module-shards.conf
23:04:55 module-shards = [
23:04:55 {
23:04:55 name = "default"
23:04:55 shards = [
23:04:55 {
23:04:55 name = "default"
23:04:55 replicas = ["member-1",
23:04:55 "member-2",
23:04:55 "member-3"]
23:04:55 }
23:04:55 ]
23:04:55 },
23:04:55 {
23:04:55 name = "inventory"
23:04:55 shards = [
23:04:55 {
23:04:55 name="inventory"
23:04:55 replicas = ["member-1",
23:04:55 "member-2",
23:04:55 "member-3"]
23:04:55 }
23:04:55 ]
23:04:55 },
23:04:55 {
23:04:55 name = "topology"
23:04:55 shards = [
23:04:55 {
23:04:55 name="topology"
23:04:55 replicas = ["member-1",
23:04:55 "member-2",
23:04:55 "member-3"]
23:04:55 }
23:04:55 ]
23:04:55 },
23:04:55 {
23:04:55 name = "toaster"
23:04:55 shards = [
23:04:55 {
23:04:55 name="toaster"
23:04:55 replicas = ["member-1",
23:04:55 "member-2",
23:04:55 "member-3"]
23:04:55 }
23:04:55 ]
23:04:55 },
23:04:55 {
23:04:55 name = "car"
23:04:55 shards = [
23:04:55 {
23:04:55 name="car"
23:04:55 replicas = ["member-1",
23:04:55 "member-2",
23:04:55 "member-3"]
23:04:55 }
23:04:55 ]
23:04:55 },
23:04:55 {
23:04:55 name = "people"
23:04:55 shards = [
23:04:55 {
23:04:55 name="people"
23:04:55 replicas = ["member-1",
23:04:55 "member-2",
23:04:55 "member-3"]
23:04:55 }
23:04:55 ]
23:04:55 },
23:04:55 {
23:04:55 name = "car-people"
23:04:55 shards = [
23:04:55 {
23:04:55 name="car-people"
23:04:55 replicas = ["member-1",
23:04:55 "member-2",
23:04:55 "member-3"]
23:04:55 }
23:04:55 ]
23:04:55 }
23:04:55 ]
23:04:55 Configuring member-2 with IP address 10.30.170.81
23:04:55 Warning: Permanently added '10.30.170.81' (ECDSA) to the list of known hosts.
23:04:55 Warning: Permanently added '10.30.170.81' (ECDSA) to the list of known hosts.
23:04:56 + source /tmp/common-functions.sh karaf-0.22.0-SNAPSHOT
23:04:56 common-functions.sh is being sourced
23:04:56 ++ [[ /tmp/common-functions.sh == \/\t\m\p\/\c\o\n\f\i\g\u\r\a\t\i\o\n\-\s\c\r\i\p\t\.\s\h ]]
23:04:56 ++ echo 'common-functions.sh is being sourced'
23:04:56 ++ BUNDLEFOLDER=karaf-0.22.0-SNAPSHOT
23:04:56 ++ export MAVENCONF=/tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:04:56 ++ MAVENCONF=/tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:04:56 ++ export FEATURESCONF=/tmp/karaf-0.22.0-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:04:56 ++ FEATURESCONF=/tmp/karaf-0.22.0-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:04:56 ++ export CUSTOMPROP=/tmp/karaf-0.22.0-SNAPSHOT/etc/custom.properties
23:04:56 ++ CUSTOMPROP=/tmp/karaf-0.22.0-SNAPSHOT/etc/custom.properties
23:04:56 ++ export LOGCONF=/tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:04:56 ++ LOGCONF=/tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:04:56 ++ export MEMCONF=/tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:04:56 ++ MEMCONF=/tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:04:56 ++ export CONTROLLERMEM=
23:04:56 ++ CONTROLLERMEM=
23:04:56 ++ export AKKACONF=/tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/akka.conf
23:04:56 ++ AKKACONF=/tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/akka.conf
23:04:56 ++ export MODULESCONF=/tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/modules.conf
23:04:56 ++ MODULESCONF=/tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/modules.conf
23:04:56 ++ export MODULESHARDSCONF=/tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/module-shards.conf
23:04:56 ++ MODULESHARDSCONF=/tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/module-shards.conf
23:04:56 ++ print_common_env
23:04:56 ++ cat
23:04:56 common-functions environment:
23:04:56 MAVENCONF: /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:04:56 ACTUALFEATURES:
23:04:56 FEATURESCONF: /tmp/karaf-0.22.0-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:04:56 CUSTOMPROP: /tmp/karaf-0.22.0-SNAPSHOT/etc/custom.properties
23:04:56 LOGCONF: /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:04:56 MEMCONF: /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:04:56 CONTROLLERMEM:
23:04:56 AKKACONF: /tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/akka.conf
23:04:56 MODULESCONF: /tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/modules.conf
23:04:56 MODULESHARDSCONF: /tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/module-shards.conf
23:04:56 SUITES:
23:04:56
23:04:56 ++ SSH='ssh -t -t'
23:04:56 ++ extra_services_cntl=' dnsmasq.service httpd.service libvirtd.service openvswitch.service ovs-vswitchd.service ovsdb-server.service rabbitmq-server.service '
23:04:56 ++ extra_services_cmp=' libvirtd.service openvswitch.service ovs-vswitchd.service ovsdb-server.service '
23:04:56 Changing to /tmp
23:04:56 Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.22.0-SNAPSHOT/karaf-0.22.0-20250316.175440-1102.zip
23:04:56 + echo 'Changing to /tmp'
23:04:56 + cd /tmp
23:04:56 + echo 'Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.22.0-SNAPSHOT/karaf-0.22.0-20250316.175440-1102.zip'
23:04:56 + wget --progress=dot:mega https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.22.0-SNAPSHOT/karaf-0.22.0-20250316.175440-1102.zip
23:04:56 --2025-03-16 23:04:56-- https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.22.0-SNAPSHOT/karaf-0.22.0-20250316.175440-1102.zip
23:04:56 Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6
23:04:56 Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected.
23:04:56 HTTP request sent, awaiting response... 200 OK
23:04:56 Length: 236097025 (225M) [application/zip]
23:04:56 Saving to: ‘karaf-0.22.0-20250316.175440-1102.zip’
23:04:56
23:04:56 0K ........ ........ ........ ........ ........ ........ 1% 59.9M 4s
23:04:56 3072K ........ ........ ........ ........ ........ ........ 2% 129M 3s
23:04:56 6144K ........ ........ ........ ........ ........ ........ 3% 143M 2s
23:04:56 9216K ........ ........ ........ ........ ........ ........ 5% 173M 2s
23:04:56 12288K ........ ........ ........ ........ ........ ........ 6% 141M 2s
23:04:56 15360K ........ ........ ........ ........ ........ ........ 7% 153M 2s
23:04:56 18432K ........ ........ ........ ........ ........ ........ 9% 185M 2s
23:04:56 21504K ........ ........ ........ ........ ........ ........ 10% 204M 2s
23:04:56 24576K ........ ........ ........ ........ ........ ........ 11% 282M 1s
23:04:56 27648K ........ ........ ........ ........ ........ ........ 13% 286M 1s
23:04:56 30720K ........ ........ ........ ........ ........ ........ 14% 287M 1s
23:04:56 33792K ........ ........ ........ ........ ........ ........ 15% 296M 1s
23:04:56 36864K ........ ........ ........ ........ ........ ........ 17% 339M 1s
23:04:56 39936K ........ ........ ........ ........ ........ ........ 18% 316M 1s
23:04:56 43008K ........ ........ ........ ........ ........ ........ 19% 328M 1s
23:04:56 46080K ........ ........ ........ ........ ........ ........ 21% 361M 1s
23:04:56 49152K ........ ........ ........ ........ ........ ........ 22% 371M 1s
23:04:56 52224K ........ ........ ........ ........ ........ ........ 23% 332M 1s
23:04:56 55296K ........ ........ ........ ........ ........ ........ 25% 304M 1s
23:04:56 58368K ........ ........ ........ ........ ........ ........ 26% 355M 1s
23:04:56 61440K ........ ........ ........ ........ ........ ........ 27% 269M 1s
23:04:56 64512K ........ ........ ........ ........ ........ ........ 29% 256M 1s
23:04:56 67584K ........ ........ ........ ........ ........ ........ 30% 282M 1s
23:04:56 70656K ........ ........ ........ ........ ........ ........ 31% 352M 1s
23:04:56 73728K ........ ........ ........ ........ ........ ........ 33% 356M 1s
23:04:56 76800K ........ ........ ........ ........ ........ ........ 34% 297M 1s
23:04:56 79872K ........ ........ ........ ........ ........ ........ 35% 166M 1s
23:04:56 82944K ........ ........ ........ ........ ........ ........ 37% 297M 1s
23:04:56 86016K ........ ........ ........ ........ ........ ........ 38% 328M 1s
23:04:56 89088K ........ ........ ........ ........ ........ ........ 39% 352M 1s
23:04:56 92160K ........ ........ ........ ........ ........ ........ 41% 364M 1s
23:04:56 95232K ........ ........ ........ ........ ........ ........ 42% 291M 1s
23:04:56 98304K ........ ........ ........ ........ ........ ........ 43% 303M 1s
23:04:56 101376K ........ ........ ........ ........ ........ ........ 45% 347M 1s
23:04:56 104448K ........ ........ ........ ........ ........ ........ 46% 183M 1s
23:04:56 107520K ........ ........ ........ ........ ........ ........ 47% 224M 1s
23:04:56 110592K ........ ........ ........ ........ ........ ........ 49% 287M 0s
23:04:56 113664K ........ ........ ........ ........ ........ ........ 50% 228M 0s
23:04:56 116736K ........ ........ ........ ........ ........ ........ 51% 200M 0s
23:04:56 119808K ........ ........ ........ ........ ........ ........ 53% 407M 0s
23:04:56 122880K ........ ........ ........ ........ ........ ........ 54% 385M 0s
23:04:56 125952K ........ ........ ........ ........ ........ ........ 55% 363M 0s
23:04:56 129024K ........ ........ ........ ........ ........ ........ 57% 372M 0s
23:04:56 132096K ........ ........ ........ ........ ........ ........ 58% 398M 0s
23:04:56 135168K ........ ........ ........ ........ ........ ........ 59% 376M 0s
23:04:56 138240K ........ ........ ........ ........ ........ ........ 61% 334M 0s
23:04:56 141312K ........ ........ ........ ........ ........ ........ 62% 332M 0s
23:04:56 144384K ........ ........ ........ ........ ........ ........ 63% 315M 0s
23:04:56 147456K ........ ........ ........ ........ ........ ........ 65% 327M 0s
23:04:56 150528K ........ ........ ........ ........ ........ ........ 66% 254M 0s
23:04:56 153600K ........ ........ ........ ........ ........ ........ 67% 246M 0s
23:04:56 156672K ........ ........ ........ ........ ........ ........ 69% 254M 0s
23:04:56 159744K ........ ........ ........ ........ ........ ........ 70% 328M 0s
23:04:56 162816K ........ ........ ........ ........ ........ ........ 71% 412M 0s
23:04:56 165888K ........ ........ ........ ........ ........ ........ 73% 348M 0s
23:04:56 168960K ........ ........ ........ ........ ........ ........ 74% 355M 0s
23:04:56 172032K ........ ........ ........ ........ ........ ........ 75% 399M 0s
23:04:56 175104K ........ ........ ........ ........ ........ ........ 77% 416M 0s
23:04:56 178176K ........ ........ ........ ........ ........ ........ 78% 368M 0s
23:04:56 181248K ........ ........ ........ ........ ........ ........ 79% 304M 0s
23:04:56 184320K ........ ........ ........ ........ ........ ........ 81% 336M 0s
23:04:56 187392K ........ ........ ........ ........ ........ ........ 82% 365M 0s
23:04:56 190464K ........ ........ ........ ........ ........ ........ 83% 403M 0s
23:04:56 193536K ........ ........ ........ ........ ........ ........ 85% 330M 0s
23:04:56 196608K ........ ........ ........ ........ ........ ........ 86% 259M 0s
23:04:56 199680K ........ ........ ........ ........ ........ ........ 87% 323M 0s
23:04:56 202752K ........ ........ ........ ........ ........ ........ 89% 243M 0s
23:04:56 205824K ........ ........ ........ ........ ........ ........ 90% 533M 0s
23:04:56 208896K ........ ........ ........ ........ ........ ........ 91% 413M 0s
23:04:56 211968K ........ ........ ........ ........ ........ ........ 93% 304M 0s
23:04:56 215040K ........ ........ ........ ........ ........ ........ 94% 149M 0s
23:04:56 218112K ........ ........ ........ ........ ........ ........ 95% 260M 0s
23:04:56 221184K ........ ........ ........ ........ ........ ........ 97% 468M 0s
23:04:56 224256K ........ ........ ........ ........ ........ ........ 98% 441M 0s
23:04:56 227328K ........ ........ ........ ........ ........ ........ 99% 286M 0s
23:04:56 230400K .. 100% 237M=0.8s
23:04:56
23:04:56 2025-03-16 23:04:56 (268 MB/s) - ‘karaf-0.22.0-20250316.175440-1102.zip’ saved [236097025/236097025]
23:04:56
23:04:56 Extracting the new controller...
23:04:56 + echo 'Extracting the new controller...'
23:04:56 + unzip -q karaf-0.22.0-20250316.175440-1102.zip
23:04:58 Adding external repositories...
23:04:58 + echo 'Adding external repositories...'
23:04:58 + sed -ie 's%org.ops4j.pax.url.mvn.repositories=%org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases%g' /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:04:58 + cat /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:04:58 ################################################################################
23:04:58 #
23:04:58 # Licensed to the Apache Software Foundation (ASF) under one or more
23:04:58 # contributor license agreements. See the NOTICE file distributed with
23:04:58 # this work for additional information regarding copyright ownership.
23:04:58 # The ASF licenses this file to You under the Apache License, Version 2.0
23:04:58 # (the "License"); you may not use this file except in compliance with
23:04:58 # the License. You may obtain a copy of the License at
23:04:58 #
23:04:58 # http://www.apache.org/licenses/LICENSE-2.0
23:04:58 #
23:04:58 # Unless required by applicable law or agreed to in writing, software
23:04:58 # distributed under the License is distributed on an "AS IS" BASIS,
23:04:58 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23:04:58 # See the License for the specific language governing permissions and
23:04:58 # limitations under the License.
23:04:58 #
23:04:58 ################################################################################
23:04:58
23:04:58 #
23:04:58 # If set to true, the following property will not allow any certificate to be used
23:04:58 # when accessing Maven repositories through SSL
23:04:58 #
23:04:58 #org.ops4j.pax.url.mvn.certificateCheck=
23:04:58
23:04:58 #
23:04:58 # Path to the local Maven settings file.
23:04:58 # The repositories defined in this file will be automatically added to the list
23:04:58 # of default repositories if the 'org.ops4j.pax.url.mvn.repositories' property
23:04:58 # below is not set.
23:04:58 # The following locations are checked for the existence of the settings.xml file
23:04:58 # * 1. looks for the specified url
23:04:58 # * 2. if not found looks for ${user.home}/.m2/settings.xml
23:04:58 # * 3. if not found looks for ${maven.home}/conf/settings.xml
23:04:58 # * 4. if not found looks for ${M2_HOME}/conf/settings.xml
23:04:58 #
23:04:58 #org.ops4j.pax.url.mvn.settings=
23:04:58
23:04:58 #
23:04:58 # Path to the local Maven repository which is used to avoid downloading
23:04:58 # artifacts when they already exist locally.
23:04:58 # The value of this property will be extracted from the settings.xml file
23:04:58 # above, or defaulted to:
23:04:58 # System.getProperty( "user.home" ) + "/.m2/repository"
23:04:58 #
23:04:58 org.ops4j.pax.url.mvn.localRepository=${karaf.home}/${karaf.default.repository}
23:04:58
23:04:58 #
23:04:58 # Default this to false. It's just weird to use undocumented repos
23:04:58 #
23:04:58 org.ops4j.pax.url.mvn.useFallbackRepositories=false
23:04:58
23:04:58 #
23:04:58 # Uncomment if you don't wanna use the proxy settings
23:04:58 # from the Maven conf/settings.xml file
23:04:58 #
23:04:58 # org.ops4j.pax.url.mvn.proxySupport=false
23:04:58
23:04:58 #
23:04:58 # Comma separated list of repositories scanned when resolving an artifact.
23:04:58 # Those repositories will be checked before iterating through the
23:04:58 # below list of repositories and even before the local repository
23:04:58 # A repository url can be appended with zero or more of the following flags:
23:04:58 # @snapshots : the repository contains snaphots
23:04:58 # @noreleases : the repository does not contain any released artifacts
23:04:58 #
23:04:58 # The following property value will add the system folder as a repo.
23:04:58 #
23:04:58 org.ops4j.pax.url.mvn.defaultRepositories=\
23:04:58 file:${karaf.home}/${karaf.default.repository}@id=system.repository@snapshots,\
23:04:58 file:${karaf.data}/kar@id=kar.repository@multi@snapshots,\
23:04:58 file:${karaf.base}/${karaf.default.repository}@id=child.system.repository@snapshots
23:04:58
23:04:58 # Use the default local repo (e.g.~/.m2/repository) as a "remote" repo
23:04:58 #org.ops4j.pax.url.mvn.defaultLocalRepoAsRemote=false
23:04:58
23:04:58 #
23:04:58 # Comma separated list of repositories scanned when resolving an artifact.
23:04:58 # The default list includes the following repositories:
23:04:58 # http://repo1.maven.org/maven2@id=central
23:04:58 # http://repository.springsource.com/maven/bundles/release@id=spring.ebr
23:04:58 # http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external
23:04:58 # http://zodiac.springsource.com/maven/bundles/release@id=gemini
23:04:58 # http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases
23:04:58 # https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases
23:04:58 # https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases
23:04:58 # To add repositories to the default ones, prepend '+' to the list of repositories
23:04:58 # to add.
23:04:58 # A repository url can be appended with zero or more of the following flags:
23:04:58 # @snapshots : the repository contains snapshots
23:04:58 # @noreleases : the repository does not contain any released artifacts
23:04:58 # @id=repository.id : the id for the repository, just like in the settings.xml this is optional but recommended
23:04:58 #
23:04:58 org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases
23:04:58
23:04:58 ### ^^^ No remote repositories. This is the only ODL change compared to Karaf defaults.Configuring the startup features...
23:04:58 + [[ True == \T\r\u\e ]]
23:04:58 + echo 'Configuring the startup features...'
23:04:58 + sed -ie 's/\(featuresBoot=\|featuresBoot =\)/featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app,/g' /tmp/karaf-0.22.0-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:04:58 + FEATURE_TEST_STRING=features-test
23:04:58 + FEATURE_TEST_VERSION=0.22.0-SNAPSHOT
23:04:58 + KARAF_VERSION=karaf4
23:04:58 + [[ integration == \i\n\t\e\g\r\a\t\i\o\n ]]
23:04:58 + sed -ie 's%\(featuresRepositories=\|featuresRepositories =\)%featuresRepositories = mvn:org.opendaylight.integration/features-test/0.22.0-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features,%g' /tmp/karaf-0.22.0-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:04:58 + [[ ! -z '' ]]
23:04:58 + cat /tmp/karaf-0.22.0-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:04:58 ################################################################################
23:04:58 #
23:04:58 # Licensed to the Apache Software Foundation (ASF) under one or more
23:04:58 # contributor license agreements. See the NOTICE file distributed with
23:04:58 # this work for additional information regarding copyright ownership.
23:04:58 # The ASF licenses this file to You under the Apache License, Version 2.0
23:04:58 # (the "License"); you may not use this file except in compliance with
23:04:58 # the License. You may obtain a copy of the License at
23:04:58 #
23:04:58 # http://www.apache.org/licenses/LICENSE-2.0
23:04:58 #
23:04:58 # Unless required by applicable law or agreed to in writing, software
23:04:58 # distributed under the License is distributed on an "AS IS" BASIS,
23:04:58 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23:04:58 # See the License for the specific language governing permissions and
23:04:58 # limitations under the License.
23:04:58 #
23:04:58 ################################################################################
23:04:58
23:04:58 #
23:04:58 # Comma separated list of features repositories to register by default
23:04:58 #
23:04:58 featuresRepositories = mvn:org.opendaylight.integration/features-test/0.22.0-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features, file:${karaf.etc}/1a8c7569-3af0-46ba-a718-f10d85096372.xml
23:04:58
23:04:58 #
23:04:58 # Comma separated list of features to install at startup
23:04:58 #
23:04:58 featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app, c5929223-d569-4724-9c23-39fc71743903
23:04:58
23:04:58 #
23:04:58 # Resource repositories (OBR) that the features resolver can use
23:04:58 # to resolve requirements/capabilities
23:04:58 #
23:04:58 # The format of the resourceRepositories is
23:04:58 # resourceRepositories=[xml:url|json:url],...
23:04:58 # for Instance:
23:04:58 #
23:04:58 #resourceRepositories=xml:http://host/path/to/index.xml
23:04:58 # or
23:04:58 #resourceRepositories=json:http://host/path/to/index.json
23:04:58 #
23:04:58
23:04:58 #
23:04:58 # Defines if the boot features are started in asynchronous mode (in a dedicated thread)
23:04:58 #
23:04:58 featuresBootAsynchronous=false
23:04:58
23:04:58 #
23:04:58 # Service requirements enforcement
23:04:58 #
23:04:58 # By default, the feature resolver checks the service requirements/capabilities of
23:04:58 # bundles for new features (xml schema >= 1.3.0) in order to automatically installs
23:04:58 # the required bundles.
23:04:58 # The following flag can have those values:
23:04:58 # - disable: service requirements are completely ignored
23:04:58 # - default: service requirements are ignored for old features
23:04:58 # - enforce: service requirements are always verified
23:04:58 #
23:04:58 #serviceRequirements=default
23:04:58
23:04:58 #
23:04:58 # Store cfg file for config element in feature
23:04:58 #
23:04:58 #configCfgStore=true
23:04:58
23:04:58 #
23:04:58 # Define if the feature service automatically refresh bundles
23:04:58 #
23:04:58 autoRefresh=true
23:04:58
23:04:58 #
23:04:58 # Configuration of features processing mechanism (overrides, blacklisting, modification of features)
23:04:58 # XML file defines instructions related to features processing
23:04:58 # versions.properties may declare properties to resolve placeholders in XML file
23:04:58 # both files are relative to ${karaf.etc}
23:04:58 #
23:04:58 #featureProcessing=org.apache.karaf.features.xml
23:04:58 #featureProcessingVersions=versions.properties
23:04:58 + configure_karaf_log karaf4 ''
23:04:58 + local -r karaf_version=karaf4
23:04:58 + local -r controllerdebugmap=
23:04:58 + local logapi=log4j
23:04:58 + grep log4j2 /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:04:58 log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
23:04:58 log4j2.rootLogger.level = INFO
23:04:58 #log4j2.rootLogger.type = asyncRoot
23:04:58 #log4j2.rootLogger.includeLocation = false
23:04:58 log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile
23:04:58 log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi
23:04:58 log4j2.rootLogger.appenderRef.Console.ref = Console
23:04:58 log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter
23:04:58 log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF}
23:04:58 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter
23:04:58 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair
23:04:58 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker
23:04:58 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL
23:04:58 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or
23:04:58 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY
23:04:58 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL
23:04:58 log4j2.logger.spifly.name = org.apache.aries.spifly
23:04:58 log4j2.logger.spifly.level = WARN
23:04:58 log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit
23:04:58 log4j2.logger.audit.level = INFO
23:04:58 log4j2.logger.audit.additivity = false
23:04:58 log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile
23:04:58 # Console appender not used by default (see log4j2.rootLogger.appenderRefs)
23:04:58 log4j2.appender.console.type = Console
23:04:58 log4j2.appender.console.name = Console
23:04:58 log4j2.appender.console.layout.type = PatternLayout
23:04:58 log4j2.appender.console.layout.pattern = ${log4j2.pattern}
23:04:58 log4j2.appender.rolling.type = RollingRandomAccessFile
23:04:58 log4j2.appender.rolling.name = RollingFile
23:04:58 log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
23:04:58 log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i
23:04:58 #log4j2.appender.rolling.immediateFlush = false
23:04:58 log4j2.appender.rolling.append = true
23:04:58 log4j2.appender.rolling.layout.type = PatternLayout
23:04:58 log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
23:04:58 log4j2.appender.rolling.policies.type = Policies
23:04:58 log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
23:04:58 log4j2.appender.rolling.policies.size.size = 64MB
23:04:58 log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
23:04:58 log4j2.appender.rolling.strategy.max = 7
23:04:58 log4j2.appender.audit.type = RollingRandomAccessFile
23:04:58 log4j2.appender.audit.name = AuditRollingFile
23:04:58 log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log
23:04:58 log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i
23:04:58 log4j2.appender.audit.append = true
23:04:58 log4j2.appender.audit.layout.type = PatternLayout
23:04:58 log4j2.appender.audit.layout.pattern = ${log4j2.pattern}
23:04:58 log4j2.appender.audit.policies.type = Policies
23:04:58 log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy
23:04:58 log4j2.appender.audit.policies.size.size = 8MB
23:04:58 log4j2.appender.audit.strategy.type = DefaultRolloverStrategy
23:04:58 log4j2.appender.audit.strategy.max = 7
23:04:58 log4j2.appender.osgi.type = PaxOsgi
23:04:58 log4j2.appender.osgi.name = PaxOsgi
23:04:58 log4j2.appender.osgi.filter = *
23:04:58 #log4j2.logger.aether.name = shaded.org.eclipse.aether
23:04:58 #log4j2.logger.aether.level = TRACE
23:04:58 #log4j2.logger.http-headers.name = shaded.org.apache.http.headers
23:04:58 #log4j2.logger.http-headers.level = DEBUG
23:04:58 #log4j2.logger.maven.name = org.ops4j.pax.url.mvn
23:04:58 #log4j2.logger.maven.level = TRACE
23:04:58 Configuring the karaf log... karaf_version: karaf4, logapi: log4j2
23:04:58 + logapi=log4j2
23:04:58 + echo 'Configuring the karaf log... karaf_version: karaf4, logapi: log4j2'
23:04:58 + '[' log4j2 == log4j2 ']'
23:04:58 + sed -ie 's/log4j2.appender.rolling.policies.size.size = 64MB/log4j2.appender.rolling.policies.size.size = 1GB/g' /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:04:58 controllerdebugmap:
23:04:58 cat /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:04:58 + orgmodule=org.opendaylight.yangtools.yang.parser.repo.YangTextSchemaContextResolver
23:04:58 + orgmodule_=org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver
23:04:58 + echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN'
23:04:58 + echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN'
23:04:58 + unset IFS
23:04:58 + echo 'controllerdebugmap: '
23:04:58 + '[' -n '' ']'
23:04:58 + echo 'cat /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg'
23:04:58 + cat /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:04:58 ################################################################################
23:04:58 #
23:04:58 # Licensed to the Apache Software Foundation (ASF) under one or more
23:04:58 # contributor license agreements. See the NOTICE file distributed with
23:04:58 # this work for additional information regarding copyright ownership.
23:04:58 # The ASF licenses this file to You under the Apache License, Version 2.0
23:04:58 # (the "License"); you may not use this file except in compliance with
23:04:58 # the License. You may obtain a copy of the License at
23:04:58 #
23:04:58 # http://www.apache.org/licenses/LICENSE-2.0
23:04:58 #
23:04:58 # Unless required by applicable law or agreed to in writing, software
23:04:58 # distributed under the License is distributed on an "AS IS" BASIS,
23:04:58 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23:04:58 # See the License for the specific language governing permissions and
23:04:58 # limitations under the License.
23:04:58 #
23:04:58 ################################################################################
23:04:58
23:04:58 # Common pattern layout for appenders
23:04:58 log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
23:04:58
23:04:58 # Root logger
23:04:58 log4j2.rootLogger.level = INFO
23:04:58 # uncomment to use asynchronous loggers, which require mvn:com.lmax/disruptor/3.3.2 library
23:04:58 #log4j2.rootLogger.type = asyncRoot
23:04:58 #log4j2.rootLogger.includeLocation = false
23:04:58 log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile
23:04:58 log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi
23:04:58 log4j2.rootLogger.appenderRef.Console.ref = Console
23:04:58 log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter
23:04:58 log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF}
23:04:58
23:04:58 # Filters for logs marked by org.opendaylight.odlparent.Markers
23:04:58 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter
23:04:58 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair
23:04:58 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker
23:04:58 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL
23:04:58 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or
23:04:58 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY
23:04:58 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL
23:04:58
23:04:58 # Loggers configuration
23:04:58
23:04:58 # Spifly logger
23:04:58 log4j2.logger.spifly.name = org.apache.aries.spifly
23:04:58 log4j2.logger.spifly.level = WARN
23:04:58
23:04:58 # Security audit logger
23:04:58 log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit
23:04:58 log4j2.logger.audit.level = INFO
23:04:58 log4j2.logger.audit.additivity = false
23:04:58 log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile
23:04:58
23:04:58 # Appenders configuration
23:04:58
23:04:58 # Console appender not used by default (see log4j2.rootLogger.appenderRefs)
23:04:58 log4j2.appender.console.type = Console
23:04:58 log4j2.appender.console.name = Console
23:04:58 log4j2.appender.console.layout.type = PatternLayout
23:04:58 log4j2.appender.console.layout.pattern = ${log4j2.pattern}
23:04:58
23:04:58 # Rolling file appender
23:04:58 log4j2.appender.rolling.type = RollingRandomAccessFile
23:04:58 log4j2.appender.rolling.name = RollingFile
23:04:58 log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
23:04:58 log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i
23:04:58 # uncomment to not force a disk flush
23:04:58 #log4j2.appender.rolling.immediateFlush = false
23:04:58 log4j2.appender.rolling.append = true
23:04:58 log4j2.appender.rolling.layout.type = PatternLayout
23:04:58 log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
23:04:58 log4j2.appender.rolling.policies.type = Policies
23:04:58 log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
23:04:58 log4j2.appender.rolling.policies.size.size = 1GB
23:04:58 log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
23:04:58 log4j2.appender.rolling.strategy.max = 7
23:04:58
23:04:58 # Audit file appender
23:04:58 log4j2.appender.audit.type = RollingRandomAccessFile
23:04:58 log4j2.appender.audit.name = AuditRollingFile
23:04:58 log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log
23:04:58 log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i
23:04:58 log4j2.appender.audit.append = true
23:04:58 log4j2.appender.audit.layout.type = PatternLayout
23:04:58 log4j2.appender.audit.layout.pattern = ${log4j2.pattern}
23:04:58 log4j2.appender.audit.policies.type = Policies
23:04:58 log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy
23:04:58 log4j2.appender.audit.policies.size.size = 8MB
23:04:58 log4j2.appender.audit.strategy.type = DefaultRolloverStrategy
23:04:58 log4j2.appender.audit.strategy.max = 7
23:04:58
23:04:58 # OSGi appender
23:04:58 log4j2.appender.osgi.type = PaxOsgi
23:04:58 log4j2.appender.osgi.name = PaxOsgi
23:04:58 log4j2.appender.osgi.filter = *
23:04:58
23:04:58 # help with identification of maven-related problems with pax-url-aether
23:04:58 #log4j2.logger.aether.name = shaded.org.eclipse.aether
23:04:58 #log4j2.logger.aether.level = TRACE
23:04:58 #log4j2.logger.http-headers.name = shaded.org.apache.http.headers
23:04:58 #log4j2.logger.http-headers.level = DEBUG
23:04:58 #log4j2.logger.maven.name = org.ops4j.pax.url.mvn
23:04:58 #log4j2.logger.maven.level = TRACE
23:04:58 log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN
23:04:58 log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN
23:04:58 Configure
23:04:58 java home: /usr/lib/jvm/java-21-openjdk-amd64
23:04:58 max memory: 3072m
23:04:58 memconf: /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:04:58 + set_java_vars /usr/lib/jvm/java-21-openjdk-amd64 3072m /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:04:58 + local -r java_home=/usr/lib/jvm/java-21-openjdk-amd64
23:04:58 + local -r controllermem=3072m
23:04:58 + local -r memconf=/tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:04:58 + echo Configure
23:04:58 + echo ' java home: /usr/lib/jvm/java-21-openjdk-amd64'
23:04:58 + echo ' max memory: 3072m'
23:04:58 + echo ' memconf: /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv'
23:04:58 + sed -ie 's%^# export JAVA_HOME%export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-21-openjdk-amd64}%g' /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:04:58 + sed -ie 's/JAVA_MAX_MEM="2048m"/JAVA_MAX_MEM=3072m/g' /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:04:58 cat /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:04:58 + echo 'cat /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv'
23:04:58 + cat /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:04:58 #!/bin/sh
23:04:58 #
23:04:58 # Licensed to the Apache Software Foundation (ASF) under one or more
23:04:58 # contributor license agreements. See the NOTICE file distributed with
23:04:58 # this work for additional information regarding copyright ownership.
23:04:58 # The ASF licenses this file to You under the Apache License, Version 2.0
23:04:58 # (the "License"); you may not use this file except in compliance with
23:04:58 # the License. You may obtain a copy of the License at
23:04:58 #
23:04:58 # http://www.apache.org/licenses/LICENSE-2.0
23:04:58 #
23:04:58 # Unless required by applicable law or agreed to in writing, software
23:04:58 # distributed under the License is distributed on an "AS IS" BASIS,
23:04:58 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23:04:58 # See the License for the specific language governing permissions and
23:04:58 # limitations under the License.
23:04:58 #
23:04:58
23:04:58 #
23:04:58 # handle specific scripts; the SCRIPT_NAME is exactly the name of the Karaf
23:04:58 # script: client, instance, shell, start, status, stop, karaf
23:04:58 #
23:04:58 # if [ "${KARAF_SCRIPT}" == "SCRIPT_NAME" ]; then
23:04:58 # Actions go here...
23:04:58 # fi
23:04:58
23:04:58 #
23:04:58 # general settings which should be applied for all scripts go here; please keep
23:04:58 # in mind that it is possible that scripts might be executed more than once, e.g.
23:04:58 # in example of the start script where the start script is executed first and the
23:04:58 # karaf script afterwards.
23:04:58 #
23:04:58
23:04:58 #
23:04:58 # The following section shows the possible configuration options for the default
23:04:58 # karaf scripts
23:04:58 #
23:04:58 export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-21-openjdk-amd64} # Location of Java installation
23:04:58 # export JAVA_OPTS # Generic JVM options, for instance, where you can pass the memory configuration
23:04:58 # export JAVA_NON_DEBUG_OPTS # Additional non-debug JVM options
23:04:58 # export EXTRA_JAVA_OPTS # Additional JVM options
23:04:58 # export KARAF_HOME # Karaf home folder
23:04:58 # export KARAF_DATA # Karaf data folder
23:04:58 # export KARAF_BASE # Karaf base folder
23:04:58 # export KARAF_ETC # Karaf etc folder
23:04:58 # export KARAF_LOG # Karaf log folder
23:04:58 # export KARAF_SYSTEM_OPTS # First citizen Karaf options
23:04:58 # export KARAF_OPTS # Additional available Karaf options
23:04:58 # export KARAF_DEBUG # Enable debug mode
23:04:58 # export KARAF_REDIRECT # Enable/set the std/err redirection when using bin/start
23:04:58 # export KARAF_NOROOT # Prevent execution as root if set to true
23:04:58 Set Java version
23:04:58 + echo 'Set Java version'
23:04:58 + sudo /usr/sbin/alternatives --install /usr/bin/java java /usr/lib/jvm/java-21-openjdk-amd64/bin/java 1
23:04:58 sudo: /usr/sbin/alternatives: command not found
23:04:58 + sudo /usr/sbin/alternatives --set java /usr/lib/jvm/java-21-openjdk-amd64/bin/java
23:04:58 sudo: /usr/sbin/alternatives: command not found
23:04:58 JDK default version ...
23:04:58 + echo 'JDK default version ...'
23:04:58 + java -version
23:04:59 openjdk version "11.0.25" 2024-10-15
23:04:59 OpenJDK Runtime Environment (build 11.0.25+9-post-Ubuntu-1ubuntu120.04)
23:04:59 OpenJDK 64-Bit Server VM (build 11.0.25+9-post-Ubuntu-1ubuntu120.04, mixed mode, sharing)
23:04:59 Set JAVA_HOME
23:04:59 + echo 'Set JAVA_HOME'
23:04:59 + export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
23:04:59 + JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
23:04:59 ++ readlink -e /usr/lib/jvm/java-21-openjdk-amd64/bin/java
23:04:59 Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-21-openjdk-amd64/bin/java
23:04:59 Listing all open ports on controller system...
23:04:59 + JAVA_RESOLVED=/usr/lib/jvm/java-21-openjdk-amd64/bin/java
23:04:59 + echo 'Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-21-openjdk-amd64/bin/java'
23:04:59 + echo 'Listing all open ports on controller system...'
23:04:59 + netstat -pnatu
23:04:59 /tmp/configuration-script.sh: line 40: netstat: command not found
23:04:59 Custom shard config exists!!!
23:04:59 Copying the shard config...
23:04:59 + '[' -f /tmp/custom_shard_config.txt ']'
23:04:59 + echo 'Custom shard config exists!!!'
23:04:59 + echo 'Copying the shard config...'
23:04:59 + cp /tmp/custom_shard_config.txt /tmp/karaf-0.22.0-SNAPSHOT/bin/
23:04:59 Configuring cluster
23:04:59 + echo 'Configuring cluster'
23:04:59 + /tmp/karaf-0.22.0-SNAPSHOT/bin/configure_cluster.sh 2 10.30.170.209 10.30.170.81 10.30.170.131
23:04:59 ################################################
23:04:59 ## Configure Cluster ##
23:04:59 ################################################
23:04:59 NOTE: Cluster configuration files not found. Copying from
23:04:59 /tmp/karaf-0.22.0-SNAPSHOT/system/org/opendaylight/controller/sal-clustering-config/10.0.2
23:04:59 Configuring unique name in akka.conf
23:04:59 Configuring hostname in akka.conf
23:04:59 Configuring data and rpc seed nodes in akka.conf
23:04:59 modules = [
23:04:59
23:04:59 {
23:04:59 name = "inventory"
23:04:59 namespace = "urn:opendaylight:inventory"
23:04:59 shard-strategy = "module"
23:04:59 },
23:04:59 {
23:04:59 name = "topology"
23:04:59 namespace = "urn:TBD:params:xml:ns:yang:network-topology"
23:04:59 shard-strategy = "module"
23:04:59 },
23:04:59 {
23:04:59 name = "toaster"
23:04:59 namespace = "http://netconfcentral.org/ns/toaster"
23:04:59 shard-strategy = "module"
23:04:59 },
23:04:59 {
23:04:59 name = "car"
23:04:59 namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car"
23:04:59 shard-strategy = "module"
23:04:59 },
23:04:59 {
23:04:59 name = "people"
23:04:59 namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people"
23:04:59 shard-strategy = "module"
23:04:59 },
23:04:59 {
23:04:59 name = "car-people"
23:04:59 namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people"
23:04:59 shard-strategy = "module"
23:04:59 }
23:04:59 ]
23:04:59 Configuring replication type in module-shards.conf
23:04:59 ################################################
23:04:59 ## NOTE: Manually restart controller to ##
23:04:59 ## apply configuration. ##
23:04:59 ################################################
23:04:59 Dump akka.conf
23:04:59 + echo 'Dump akka.conf'
23:04:59 + cat /tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/akka.conf
23:04:59
23:04:59 odl-cluster-data {
23:04:59 akka {
23:04:59 remote {
23:04:59 artery {
23:04:59 enabled = on
23:04:59 transport = tcp
23:04:59 canonical.hostname = "10.30.170.81"
23:04:59 canonical.port = 2550
23:04:59 }
23:04:59 }
23:04:59
23:04:59 cluster {
23:04:59 # Using artery.
23:04:59 seed-nodes = ["akka://opendaylight-cluster-data@10.30.170.209:2550",
23:04:59 "akka://opendaylight-cluster-data@10.30.170.81:2550",
23:04:59 "akka://opendaylight-cluster-data@10.30.170.131:2550"]
23:04:59
23:04:59 roles = ["member-2"]
23:04:59
23:04:59 # when under load we might trip a false positive on the failure detector
23:04:59 # failure-detector {
23:04:59 # heartbeat-interval = 4 s
23:04:59 # acceptable-heartbeat-pause = 16s
23:04:59 # }
23:04:59 }
23:04:59
23:04:59 persistence {
23:04:59 # By default the snapshots/journal directories live in KARAF_HOME. You can choose to put it somewhere else by
23:04:59 # modifying the following two properties. The directory location specified may be a relative or absolute path.
23:04:59 # The relative path is always relative to KARAF_HOME.
23:04:59
23:04:59 # snapshot-store.local.dir = "target/snapshots"
23:04:59
23:04:59 # Use lz4 compression for LocalSnapshotStore snapshots
23:04:59 snapshot-store.local.use-lz4-compression = false
23:04:59 # Size of blocks for lz4 compression: 64KB, 256KB, 1MB or 4MB
23:04:59 snapshot-store.local.lz4-blocksize = 256KB
23:04:59 }
23:04:59 disable-default-actor-system-quarantined-event-handling = "false"
23:04:59 }
23:04:59 }
23:04:59 Dump modules.conf
23:04:59 + echo 'Dump modules.conf'
23:04:59 + cat /tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/modules.conf
23:04:59 modules = [
23:04:59
23:04:59 {
23:04:59 name = "inventory"
23:04:59 namespace = "urn:opendaylight:inventory"
23:04:59 shard-strategy = "module"
23:04:59 },
23:04:59 {
23:04:59 name = "topology"
23:04:59 namespace = "urn:TBD:params:xml:ns:yang:network-topology"
23:04:59 shard-strategy = "module"
23:04:59 },
23:04:59 {
23:04:59 name = "toaster"
23:04:59 namespace = "http://netconfcentral.org/ns/toaster"
23:04:59 shard-strategy = "module"
23:04:59 },
23:04:59 {
23:04:59 name = "car"
23:04:59 namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car"
23:04:59 shard-strategy = "module"
23:04:59 },
23:04:59 {
23:04:59 name = "people"
23:04:59 namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people"
23:04:59 shard-strategy = "module"
23:04:59 },
23:04:59 {
23:04:59 name = "car-people"
23:04:59 namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people"
23:04:59 shard-strategy = "module"
23:04:59 }
23:04:59 ]
23:04:59 Dump module-shards.conf
23:04:59 + echo 'Dump module-shards.conf'
23:04:59 + cat /tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/module-shards.conf
23:04:59 module-shards = [
23:04:59 {
23:04:59 name = "default"
23:04:59 shards = [
23:04:59 {
23:04:59 name = "default"
23:04:59 replicas = ["member-1",
23:04:59 "member-2",
23:04:59 "member-3"]
23:04:59 }
23:04:59 ]
23:04:59 },
23:04:59 {
23:04:59 name = "inventory"
23:04:59 shards = [
23:04:59 {
23:04:59 name="inventory"
23:04:59 replicas = ["member-1",
23:04:59 "member-2",
23:04:59 "member-3"]
23:04:59 }
23:04:59 ]
23:04:59 },
23:04:59 {
23:04:59 name = "topology"
23:04:59 shards = [
23:04:59 {
23:04:59 name="topology"
23:04:59 replicas = ["member-1",
23:04:59 "member-2",
23:04:59 "member-3"]
23:04:59 }
23:04:59 ]
23:04:59 },
23:04:59 {
23:04:59 name = "toaster"
23:04:59 shards = [
23:04:59 {
23:04:59 name="toaster"
23:04:59 replicas = ["member-1",
23:04:59 "member-2",
23:04:59 "member-3"]
23:04:59 }
23:04:59 ]
23:04:59 },
23:04:59 {
23:04:59 name = "car"
23:04:59 shards = [
23:04:59 {
23:04:59 name="car"
23:04:59 replicas = ["member-1",
23:04:59 "member-2",
23:04:59 "member-3"]
23:04:59 }
23:04:59 ]
23:04:59 },
23:04:59 {
23:04:59 name = "people"
23:04:59 shards = [
23:04:59 {
23:04:59 name="people"
23:04:59 replicas = ["member-1",
23:04:59 "member-2",
23:04:59 "member-3"]
23:04:59 }
23:04:59 ]
23:04:59 },
23:04:59 {
23:04:59 name = "car-people"
23:04:59 shards = [
23:04:59 {
23:04:59 name="car-people"
23:04:59 replicas = ["member-1",
23:04:59 "member-2",
23:04:59 "member-3"]
23:04:59 }
23:04:59 ]
23:04:59 }
23:04:59 ]
23:04:59 Configuring member-3 with IP address 10.30.170.131
23:04:59 Warning: Permanently added '10.30.170.131' (ECDSA) to the list of known hosts.
23:04:59 Warning: Permanently added '10.30.170.131' (ECDSA) to the list of known hosts.
23:04:59 + source /tmp/common-functions.sh karaf-0.22.0-SNAPSHOT
23:04:59 common-functions.sh is being sourced
23:04:59 ++ [[ /tmp/common-functions.sh == \/\t\m\p\/\c\o\n\f\i\g\u\r\a\t\i\o\n\-\s\c\r\i\p\t\.\s\h ]]
23:04:59 ++ echo 'common-functions.sh is being sourced'
23:04:59 ++ BUNDLEFOLDER=karaf-0.22.0-SNAPSHOT
23:04:59 ++ export MAVENCONF=/tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:04:59 ++ MAVENCONF=/tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:04:59 ++ export FEATURESCONF=/tmp/karaf-0.22.0-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:04:59 ++ FEATURESCONF=/tmp/karaf-0.22.0-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:04:59 ++ export CUSTOMPROP=/tmp/karaf-0.22.0-SNAPSHOT/etc/custom.properties
23:04:59 ++ CUSTOMPROP=/tmp/karaf-0.22.0-SNAPSHOT/etc/custom.properties
23:04:59 ++ export LOGCONF=/tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:04:59 ++ LOGCONF=/tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:04:59 ++ export MEMCONF=/tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:04:59 ++ MEMCONF=/tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:04:59 ++ export CONTROLLERMEM=
23:04:59 ++ CONTROLLERMEM=
23:04:59 ++ export AKKACONF=/tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/akka.conf
23:04:59 ++ AKKACONF=/tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/akka.conf
23:04:59 ++ export MODULESCONF=/tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/modules.conf
23:04:59 ++ MODULESCONF=/tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/modules.conf
23:04:59 ++ export MODULESHARDSCONF=/tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/module-shards.conf
23:04:59 ++ MODULESHARDSCONF=/tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/module-shards.conf
23:04:59 ++ print_common_env
23:04:59 ++ cat
23:04:59 common-functions environment:
23:04:59 MAVENCONF: /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:04:59 ACTUALFEATURES:
23:04:59 FEATURESCONF: /tmp/karaf-0.22.0-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:04:59 CUSTOMPROP: /tmp/karaf-0.22.0-SNAPSHOT/etc/custom.properties
23:04:59 LOGCONF: /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:04:59 MEMCONF: /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:04:59 CONTROLLERMEM:
23:04:59 AKKACONF: /tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/akka.conf
23:04:59 MODULESCONF: /tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/modules.conf
23:04:59 MODULESHARDSCONF: /tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/module-shards.conf
23:04:59 SUITES:
23:04:59
23:04:59 ++ SSH='ssh -t -t'
23:04:59 ++ extra_services_cntl=' dnsmasq.service httpd.service libvirtd.service openvswitch.service ovs-vswitchd.service ovsdb-server.service rabbitmq-server.service '
23:04:59 ++ extra_services_cmp=' libvirtd.service openvswitch.service ovs-vswitchd.service ovsdb-server.service '
23:04:59 Changing to /tmp
23:04:59 Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.22.0-SNAPSHOT/karaf-0.22.0-20250316.175440-1102.zip
23:04:59 + echo 'Changing to /tmp'
23:04:59 + cd /tmp
23:04:59 + echo 'Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.22.0-SNAPSHOT/karaf-0.22.0-20250316.175440-1102.zip'
23:04:59 + wget --progress=dot:mega https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.22.0-SNAPSHOT/karaf-0.22.0-20250316.175440-1102.zip
23:04:59 --2025-03-16 23:04:59-- https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.22.0-SNAPSHOT/karaf-0.22.0-20250316.175440-1102.zip
23:04:59 Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6
23:04:59 Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected.
23:04:59 HTTP request sent, awaiting response... 200 OK
23:04:59 Length: 236097025 (225M) [application/zip]
23:04:59 Saving to: ‘karaf-0.22.0-20250316.175440-1102.zip’
23:04:59
23:04:59 0K ........ ........ ........ ........ ........ ........ 1% 11.1M 20s
23:05:00 3072K ........ ........ ........ ........ ........ ........ 2% 99.3M 11s
23:05:00 6144K ........ ........ ........ ........ ........ ........ 3% 101M 8s
23:05:00 9216K ........ ........ ........ ........ ........ ........ 5% 109M 6s
23:05:00 12288K ........ ........ ........ ........ ........ ........ 6% 124M 5s
23:05:00 15360K ........ ........ ........ ........ ........ ........ 7% 131M 5s
23:05:00 18432K ........ ........ ........ ........ ........ ........ 9% 99.9M 4s
23:05:00 21504K ........ ........ ........ ........ ........ ........ 10% 79.7M 4s
23:05:00 24576K ........ ........ ........ ........ ........ ........ 11% 127M 4s
23:05:00 27648K ........ ........ ........ ........ ........ ........ 13% 140M 3s
23:05:00 30720K ........ ........ ........ ........ ........ ........ 14% 141M 3s
23:05:00 33792K ........ ........ ........ ........ ........ ........ 15% 120M 3s
23:05:00 36864K ........ ........ ........ ........ ........ ........ 17% 142M 3s
23:05:00 39936K ........ ........ ........ ........ ........ ........ 18% 134M 3s
23:05:00 43008K ........ ........ ........ ........ ........ ........ 19% 137M 3s
23:05:00 46080K ........ ........ ........ ........ ........ ........ 21% 114M 2s
23:05:00 49152K ........ ........ ........ ........ ........ ........ 22% 142M 2s
23:05:00 52224K ........ ........ ........ ........ ........ ........ 23% 142M 2s
23:05:00 55296K ........ ........ ........ ........ ........ ........ 25% 147M 2s
23:05:00 58368K ........ ........ ........ ........ ........ ........ 26% 141M 2s
23:05:00 61440K ........ ........ ........ ........ ........ ........ 27% 126M 2s
23:05:00 64512K ........ ........ ........ ........ ........ ........ 29% 110M 2s
23:05:00 67584K ........ ........ ........ ........ ........ ........ 30% 135M 2s
23:05:00 70656K ........ ........ ........ ........ ........ ........ 31% 130M 2s
23:05:00 73728K ........ ........ ........ ........ ........ ........ 33% 134M 2s
23:05:00 76800K ........ ........ ........ ........ ........ ........ 34% 117M 2s
23:05:00 79872K ........ ........ ........ ........ ........ ........ 35% 144M 2s
23:05:00 82944K ........ ........ ........ ........ ........ ........ 37% 120M 2s
23:05:00 86016K ........ ........ ........ ........ ........ ........ 38% 114M 2s
23:05:00 89088K ........ ........ ........ ........ ........ ........ 39% 122M 1s
23:05:00 92160K ........ ........ ........ ........ ........ ........ 41% 144M 1s
23:05:00 95232K ........ ........ ........ ........ ........ ........ 42% 149M 1s
23:05:00 98304K ........ ........ ........ ........ ........ ........ 43% 137M 1s
23:05:00 101376K ........ ........ ........ ........ ........ ........ 45% 147M 1s
23:05:00 104448K ........ ........ ........ ........ ........ ........ 46% 134M 1s
23:05:00 107520K ........ ........ ........ ........ ........ ........ 47% 115M 1s
23:05:00 110592K ........ ........ ........ ........ ........ ........ 49% 169M 1s
23:05:00 113664K ........ ........ ........ ........ ........ ........ 50% 110M 1s
23:05:00 116736K ........ ........ ........ ........ ........ ........ 51% 141M 1s
23:05:01 119808K ........ ........ ........ ........ ........ ........ 53% 130M 1s
23:05:01 122880K ........ ........ ........ ........ ........ ........ 54% 186M 1s
23:05:01 125952K ........ ........ ........ ........ ........ ........ 55% 215M 1s
23:05:01 129024K ........ ........ ........ ........ ........ ........ 57% 249M 1s
23:05:01 132096K ........ ........ ........ ........ ........ ........ 58% 306M 1s
23:05:01 135168K ........ ........ ........ ........ ........ ........ 59% 304M 1s
23:05:01 138240K ........ ........ ........ ........ ........ ........ 61% 304M 1s
23:05:01 141312K ........ ........ ........ ........ ........ ........ 62% 296M 1s
23:05:01 144384K ........ ........ ........ ........ ........ ........ 63% 292M 1s
23:05:01 147456K ........ ........ ........ ........ ........ ........ 65% 297M 1s
23:05:01 150528K ........ ........ ........ ........ ........ ........ 66% 282M 1s
23:05:01 153600K ........ ........ ........ ........ ........ ........ 67% 284M 1s
23:05:01 156672K ........ ........ ........ ........ ........ ........ 69% 298M 1s
23:05:01 159744K ........ ........ ........ ........ ........ ........ 70% 278M 1s
23:05:01 162816K ........ ........ ........ ........ ........ ........ 71% 312M 1s
23:05:01 165888K ........ ........ ........ ........ ........ ........ 73% 301M 0s
23:05:01 168960K ........ ........ ........ ........ ........ ........ 74% 295M 0s
23:05:01 172032K ........ ........ ........ ........ ........ ........ 75% 308M 0s
23:05:01 175104K ........ ........ ........ ........ ........ ........ 77% 312M 0s
23:05:01 178176K ........ ........ ........ ........ ........ ........ 78% 313M 0s
23:05:01 181248K ........ ........ ........ ........ ........ ........ 79% 280M 0s
23:05:01 184320K ........ ........ ........ ........ ........ ........ 81% 284M 0s
23:05:01 187392K ........ ........ ........ ........ ........ ........ 82% 289M 0s
23:05:01 190464K ........ ........ ........ ........ ........ ........ 83% 297M 0s
23:05:01 193536K ........ ........ ........ ........ ........ ........ 85% 294M 0s
23:05:01 196608K ........ ........ ........ ........ ........ ........ 86% 289M 0s
23:05:01 199680K ........ ........ ........ ........ ........ ........ 87% 300M 0s
23:05:01 202752K ........ ........ ........ ........ ........ ........ 89% 290M 0s
23:05:01 205824K ........ ........ ........ ........ ........ ........ 90% 279M 0s
23:05:01 208896K ........ ........ ........ ........ ........ ........ 91% 279M 0s
23:05:01 211968K ........ ........ ........ ........ ........ ........ 93% 287M 0s
23:05:01 215040K ........ ........ ........ ........ ........ ........ 94% 265M 0s
23:05:01 218112K ........ ........ ........ ........ ........ ........ 95% 266M 0s
23:05:01 221184K ........ ........ ........ ........ ........ ........ 97% 278M 0s
23:05:01 224256K ........ ........ ........ ........ ........ ........ 98% 278M 0s
23:05:01 227328K ........ ........ ........ ........ ........ ........ 99% 264M 0s
23:05:01 230400K .. 100% 398M=1.6s
23:05:01
23:05:01 2025-03-16 23:05:01 (143 MB/s) - ‘karaf-0.22.0-20250316.175440-1102.zip’ saved [236097025/236097025]
23:05:01
23:05:01 Extracting the new controller...
23:05:01 + echo 'Extracting the new controller...'
23:05:01 + unzip -q karaf-0.22.0-20250316.175440-1102.zip
23:05:03 Adding external repositories...
23:05:03 + echo 'Adding external repositories...'
23:05:03 + sed -ie 's%org.ops4j.pax.url.mvn.repositories=%org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases%g' /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:05:03 + cat /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:05:03 ################################################################################
23:05:03 #
23:05:03 # Licensed to the Apache Software Foundation (ASF) under one or more
23:05:03 # contributor license agreements. See the NOTICE file distributed with
23:05:03 # this work for additional information regarding copyright ownership.
23:05:03 # The ASF licenses this file to You under the Apache License, Version 2.0
23:05:03 # (the "License"); you may not use this file except in compliance with
23:05:03 # the License. You may obtain a copy of the License at
23:05:03 #
23:05:03 # http://www.apache.org/licenses/LICENSE-2.0
23:05:03 #
23:05:03 # Unless required by applicable law or agreed to in writing, software
23:05:03 # distributed under the License is distributed on an "AS IS" BASIS,
23:05:03 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23:05:03 # See the License for the specific language governing permissions and
23:05:03 # limitations under the License.
23:05:03 #
23:05:03 ################################################################################
23:05:03
23:05:03 #
23:05:03 # If set to true, the following property will not allow any certificate to be used
23:05:03 # when accessing Maven repositories through SSL
23:05:03 #
23:05:03 #org.ops4j.pax.url.mvn.certificateCheck=
23:05:03
23:05:03 #
23:05:03 # Path to the local Maven settings file.
23:05:03 # The repositories defined in this file will be automatically added to the list
23:05:03 # of default repositories if the 'org.ops4j.pax.url.mvn.repositories' property
23:05:03 # below is not set.
23:05:03 # The following locations are checked for the existence of the settings.xml file
23:05:03 # * 1. looks for the specified url
23:05:03 # * 2. if not found looks for ${user.home}/.m2/settings.xml
23:05:03 # * 3. if not found looks for ${maven.home}/conf/settings.xml
23:05:03 # * 4. if not found looks for ${M2_HOME}/conf/settings.xml
23:05:03 #
23:05:03 #org.ops4j.pax.url.mvn.settings=
23:05:03
23:05:03 #
23:05:03 # Path to the local Maven repository which is used to avoid downloading
23:05:03 # artifacts when they already exist locally.
23:05:03 # The value of this property will be extracted from the settings.xml file
23:05:03 # above, or defaulted to:
23:05:03 # System.getProperty( "user.home" ) + "/.m2/repository"
23:05:03 #
23:05:03 org.ops4j.pax.url.mvn.localRepository=${karaf.home}/${karaf.default.repository}
23:05:03
23:05:03 #
23:05:03 # Default this to false. It's just weird to use undocumented repos
23:05:03 #
23:05:03 org.ops4j.pax.url.mvn.useFallbackRepositories=false
23:05:03
23:05:03 #
23:05:03 # Uncomment if you don't wanna use the proxy settings
23:05:03 # from the Maven conf/settings.xml file
23:05:03 #
23:05:03 # org.ops4j.pax.url.mvn.proxySupport=false
23:05:03
23:05:03 #
23:05:03 # Comma separated list of repositories scanned when resolving an artifact.
23:05:03 # Those repositories will be checked before iterating through the
23:05:03 # below list of repositories and even before the local repository
23:05:03 # A repository url can be appended with zero or more of the following flags:
23:05:03 # @snapshots : the repository contains snaphots
23:05:03 # @noreleases : the repository does not contain any released artifacts
23:05:03 #
23:05:03 # The following property value will add the system folder as a repo.
23:05:03 #
23:05:03 org.ops4j.pax.url.mvn.defaultRepositories=\
23:05:03 file:${karaf.home}/${karaf.default.repository}@id=system.repository@snapshots,\
23:05:03 file:${karaf.data}/kar@id=kar.repository@multi@snapshots,\
23:05:03 file:${karaf.base}/${karaf.default.repository}@id=child.system.repository@snapshots
23:05:03
23:05:03 # Use the default local repo (e.g.~/.m2/repository) as a "remote" repo
23:05:03 #org.ops4j.pax.url.mvn.defaultLocalRepoAsRemote=false
23:05:03
23:05:03 #
23:05:03 # Comma separated list of repositories scanned when resolving an artifact.
23:05:03 # The default list includes the following repositories:
23:05:03 # http://repo1.maven.org/maven2@id=central
23:05:03 # http://repository.springsource.com/maven/bundles/release@id=spring.ebr
23:05:03 # http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external
23:05:03 # http://zodiac.springsource.com/maven/bundles/release@id=gemini
23:05:03 # http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases
23:05:03 # https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases
23:05:03 # https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases
23:05:03 # To add repositories to the default ones, prepend '+' to the list of repositories
23:05:03 # to add.
23:05:03 # A repository url can be appended with zero or more of the following flags:
23:05:03 # @snapshots : the repository contains snapshots
23:05:03 # @noreleases : the repository does not contain any released artifacts
23:05:03 # @id=repository.id : the id for the repository, just like in the settings.xml this is optional but recommended
23:05:03 #
23:05:03 org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases
23:05:03
23:05:03 ### ^^^ No remote repositories. This is the only ODL change compared to Karaf defaults.Configuring the startup features...
23:05:03 + [[ True == \T\r\u\e ]]
23:05:03 + echo 'Configuring the startup features...'
23:05:03 + sed -ie 's/\(featuresBoot=\|featuresBoot =\)/featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app,/g' /tmp/karaf-0.22.0-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:05:03 + FEATURE_TEST_STRING=features-test
23:05:03 + FEATURE_TEST_VERSION=0.22.0-SNAPSHOT
23:05:03 + KARAF_VERSION=karaf4
23:05:03 + [[ integration == \i\n\t\e\g\r\a\t\i\o\n ]]
23:05:03 + sed -ie 's%\(featuresRepositories=\|featuresRepositories =\)%featuresRepositories = mvn:org.opendaylight.integration/features-test/0.22.0-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features,%g' /tmp/karaf-0.22.0-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:05:03 + [[ ! -z '' ]]
23:05:03 + cat /tmp/karaf-0.22.0-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:05:03 ################################################################################
23:05:03 #
23:05:03 # Licensed to the Apache Software Foundation (ASF) under one or more
23:05:03 # contributor license agreements. See the NOTICE file distributed with
23:05:03 # this work for additional information regarding copyright ownership.
23:05:03 # The ASF licenses this file to You under the Apache License, Version 2.0
23:05:03 # (the "License"); you may not use this file except in compliance with
23:05:03 # the License. You may obtain a copy of the License at
23:05:03 #
23:05:03 # http://www.apache.org/licenses/LICENSE-2.0
23:05:03 #
23:05:03 # Unless required by applicable law or agreed to in writing, software
23:05:03 # distributed under the License is distributed on an "AS IS" BASIS,
23:05:03 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23:05:03 # See the License for the specific language governing permissions and
23:05:03 # limitations under the License.
23:05:03 #
23:05:03 ################################################################################
23:05:03
23:05:03 #
23:05:03 # Comma separated list of features repositories to register by default
23:05:03 #
23:05:03 featuresRepositories = mvn:org.opendaylight.integration/features-test/0.22.0-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features, file:${karaf.etc}/1a8c7569-3af0-46ba-a718-f10d85096372.xml
23:05:03
23:05:03 #
23:05:03 # Comma separated list of features to install at startup
23:05:03 #
23:05:03 featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app, c5929223-d569-4724-9c23-39fc71743903
23:05:03
23:05:03 #
23:05:03 # Resource repositories (OBR) that the features resolver can use
23:05:03 # to resolve requirements/capabilities
23:05:03 #
23:05:03 # The format of the resourceRepositories is
23:05:03 # resourceRepositories=[xml:url|json:url],...
23:05:03 # for Instance:
23:05:03 #
23:05:03 #resourceRepositories=xml:http://host/path/to/index.xml
23:05:03 # or
23:05:03 #resourceRepositories=json:http://host/path/to/index.json
23:05:03 #
23:05:03
23:05:03 #
23:05:03 # Defines if the boot features are started in asynchronous mode (in a dedicated thread)
23:05:03 #
23:05:03 featuresBootAsynchronous=false
23:05:03
23:05:03 #
23:05:03 # Service requirements enforcement
23:05:03 #
23:05:03 # By default, the feature resolver checks the service requirements/capabilities of
23:05:03 # bundles for new features (xml schema >= 1.3.0) in order to automatically installs
23:05:03 # the required bundles.
23:05:03 # The following flag can have those values:
23:05:03 # - disable: service requirements are completely ignored
23:05:03 # - default: service requirements are ignored for old features
23:05:03 # - enforce: service requirements are always verified
23:05:03 #
23:05:03 #serviceRequirements=default
23:05:03
23:05:03 #
23:05:03 # Store cfg file for config element in feature
23:05:03 #
23:05:03 #configCfgStore=true
23:05:03
23:05:03 #
23:05:03 # Define if the feature service automatically refresh bundles
23:05:03 #
23:05:03 autoRefresh=true
23:05:03
23:05:03 #
23:05:03 # Configuration of features processing mechanism (overrides, blacklisting, modification of features)
23:05:03 # XML file defines instructions related to features processing
23:05:03 # versions.properties may declare properties to resolve placeholders in XML file
23:05:03 # both files are relative to ${karaf.etc}
23:05:03 #
23:05:03 #featureProcessing=org.apache.karaf.features.xml
23:05:03 #featureProcessingVersions=versions.properties
23:05:03 + configure_karaf_log karaf4 ''
23:05:03 + local -r karaf_version=karaf4
23:05:03 + local -r controllerdebugmap=
23:05:03 + local logapi=log4j
23:05:03 + grep log4j2 /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:05:03 log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
23:05:03 log4j2.rootLogger.level = INFO
23:05:03 #log4j2.rootLogger.type = asyncRoot
23:05:03 #log4j2.rootLogger.includeLocation = false
23:05:03 log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile
23:05:03 log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi
23:05:03 log4j2.rootLogger.appenderRef.Console.ref = Console
23:05:03 log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter
23:05:03 log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF}
23:05:03 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter
23:05:03 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair
23:05:03 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker
23:05:03 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL
23:05:03 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or
23:05:03 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY
23:05:03 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL
23:05:03 log4j2.logger.spifly.name = org.apache.aries.spifly
23:05:03 log4j2.logger.spifly.level = WARN
23:05:03 log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit
23:05:03 log4j2.logger.audit.level = INFO
23:05:03 log4j2.logger.audit.additivity = false
23:05:03 log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile
23:05:03 # Console appender not used by default (see log4j2.rootLogger.appenderRefs)
23:05:03 log4j2.appender.console.type = Console
23:05:03 log4j2.appender.console.name = Console
23:05:03 log4j2.appender.console.layout.type = PatternLayout
23:05:03 log4j2.appender.console.layout.pattern = ${log4j2.pattern}
23:05:03 log4j2.appender.rolling.type = RollingRandomAccessFile
23:05:03 log4j2.appender.rolling.name = RollingFile
23:05:03 log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
23:05:03 log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i
23:05:03 #log4j2.appender.rolling.immediateFlush = false
23:05:03 log4j2.appender.rolling.append = true
23:05:03 log4j2.appender.rolling.layout.type = PatternLayout
23:05:03 log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
23:05:03 log4j2.appender.rolling.policies.type = Policies
23:05:03 log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
23:05:03 log4j2.appender.rolling.policies.size.size = 64MB
23:05:03 log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
23:05:03 log4j2.appender.rolling.strategy.max = 7
23:05:03 log4j2.appender.audit.type = RollingRandomAccessFile
23:05:03 log4j2.appender.audit.name = AuditRollingFile
23:05:03 log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log
23:05:03 log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i
23:05:03 log4j2.appender.audit.append = true
23:05:03 log4j2.appender.audit.layout.type = PatternLayout
23:05:03 log4j2.appender.audit.layout.pattern = ${log4j2.pattern}
23:05:03 log4j2.appender.audit.policies.type = Policies
23:05:03 log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy
23:05:03 log4j2.appender.audit.policies.size.size = 8MB
23:05:03 log4j2.appender.audit.strategy.type = DefaultRolloverStrategy
23:05:03 log4j2.appender.audit.strategy.max = 7
23:05:03 log4j2.appender.osgi.type = PaxOsgi
23:05:03 log4j2.appender.osgi.name = PaxOsgi
23:05:03 log4j2.appender.osgi.filter = *
23:05:03 #log4j2.logger.aether.name = shaded.org.eclipse.aether
23:05:03 #log4j2.logger.aether.level = TRACE
23:05:03 #log4j2.logger.http-headers.name = shaded.org.apache.http.headers
23:05:03 #log4j2.logger.http-headers.level = DEBUG
23:05:03 #log4j2.logger.maven.name = org.ops4j.pax.url.mvn
23:05:03 #log4j2.logger.maven.level = TRACE
23:05:03 + logapi=log4j2
23:05:03 Configuring the karaf log... karaf_version: karaf4, logapi: log4j2
23:05:03 + echo 'Configuring the karaf log... karaf_version: karaf4, logapi: log4j2'
23:05:03 + '[' log4j2 == log4j2 ']'
23:05:03 + sed -ie 's/log4j2.appender.rolling.policies.size.size = 64MB/log4j2.appender.rolling.policies.size.size = 1GB/g' /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:05:03 + orgmodule=org.opendaylight.yangtools.yang.parser.repo.YangTextSchemaContextResolver
23:05:03 + orgmodule_=org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver
23:05:03 + echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN'
23:05:03 controllerdebugmap:
23:05:03 + echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN'
23:05:03 + unset IFS
23:05:03 + echo 'controllerdebugmap: '
23:05:03 + '[' -n '' ']'
23:05:03 cat /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:05:03 + echo 'cat /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg'
23:05:03 + cat /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:05:03 ################################################################################
23:05:03 #
23:05:03 # Licensed to the Apache Software Foundation (ASF) under one or more
23:05:03 # contributor license agreements. See the NOTICE file distributed with
23:05:03 # this work for additional information regarding copyright ownership.
23:05:03 # The ASF licenses this file to You under the Apache License, Version 2.0
23:05:03 # (the "License"); you may not use this file except in compliance with
23:05:03 # the License. You may obtain a copy of the License at
23:05:03 #
23:05:03 # http://www.apache.org/licenses/LICENSE-2.0
23:05:03 #
23:05:03 # Unless required by applicable law or agreed to in writing, software
23:05:03 # distributed under the License is distributed on an "AS IS" BASIS,
23:05:03 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23:05:03 # See the License for the specific language governing permissions and
23:05:03 # limitations under the License.
23:05:03 #
23:05:03 ################################################################################
23:05:03
23:05:03 # Common pattern layout for appenders
23:05:03 log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
23:05:03
23:05:03 # Root logger
23:05:03 log4j2.rootLogger.level = INFO
23:05:03 # uncomment to use asynchronous loggers, which require mvn:com.lmax/disruptor/3.3.2 library
23:05:03 #log4j2.rootLogger.type = asyncRoot
23:05:03 #log4j2.rootLogger.includeLocation = false
23:05:03 log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile
23:05:03 log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi
23:05:03 log4j2.rootLogger.appenderRef.Console.ref = Console
23:05:03 log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter
23:05:03 log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF}
23:05:03
23:05:03 # Filters for logs marked by org.opendaylight.odlparent.Markers
23:05:03 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter
23:05:03 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair
23:05:03 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker
23:05:03 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL
23:05:03 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or
23:05:03 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY
23:05:03 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL
23:05:03
23:05:03 # Loggers configuration
23:05:03
23:05:03 # Spifly logger
23:05:03 log4j2.logger.spifly.name = org.apache.aries.spifly
23:05:03 log4j2.logger.spifly.level = WARN
23:05:03
23:05:03 # Security audit logger
23:05:03 log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit
23:05:03 log4j2.logger.audit.level = INFO
23:05:03 log4j2.logger.audit.additivity = false
23:05:03 log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile
23:05:03
23:05:03 # Appenders configuration
23:05:03
23:05:03 # Console appender not used by default (see log4j2.rootLogger.appenderRefs)
23:05:03 log4j2.appender.console.type = Console
23:05:03 log4j2.appender.console.name = Console
23:05:03 log4j2.appender.console.layout.type = PatternLayout
23:05:03 log4j2.appender.console.layout.pattern = ${log4j2.pattern}
23:05:03
23:05:03 # Rolling file appender
23:05:03 log4j2.appender.rolling.type = RollingRandomAccessFile
23:05:03 log4j2.appender.rolling.name = RollingFile
23:05:03 log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
23:05:03 log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i
23:05:03 # uncomment to not force a disk flush
23:05:03 #log4j2.appender.rolling.immediateFlush = false
23:05:03 log4j2.appender.rolling.append = true
23:05:03 log4j2.appender.rolling.layout.type = PatternLayout
23:05:03 log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
23:05:03 log4j2.appender.rolling.policies.type = Policies
23:05:03 log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
23:05:03 log4j2.appender.rolling.policies.size.size = 1GB
23:05:03 log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
23:05:03 log4j2.appender.rolling.strategy.max = 7
23:05:03
23:05:03 # Audit file appender
23:05:03 log4j2.appender.audit.type = RollingRandomAccessFile
23:05:03 log4j2.appender.audit.name = AuditRollingFile
23:05:03 log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log
23:05:03 log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i
23:05:03 log4j2.appender.audit.append = true
23:05:03 log4j2.appender.audit.layout.type = PatternLayout
23:05:03 log4j2.appender.audit.layout.pattern = ${log4j2.pattern}
23:05:03 log4j2.appender.audit.policies.type = Policies
23:05:03 log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy
23:05:03 log4j2.appender.audit.policies.size.size = 8MB
23:05:03 log4j2.appender.audit.strategy.type = DefaultRolloverStrategy
23:05:03 log4j2.appender.audit.strategy.max = 7
23:05:03
23:05:03 # OSGi appender
23:05:03 log4j2.appender.osgi.type = PaxOsgi
23:05:03 log4j2.appender.osgi.name = PaxOsgi
23:05:03 log4j2.appender.osgi.filter = *
23:05:03
23:05:03 # help with identification of maven-related problems with pax-url-aether
23:05:03 #log4j2.logger.aether.name = shaded.org.eclipse.aether
23:05:03 #log4j2.logger.aether.level = TRACE
23:05:03 #log4j2.logger.http-headers.name = shaded.org.apache.http.headers
23:05:03 #log4j2.logger.http-headers.level = DEBUG
23:05:03 #log4j2.logger.maven.name = org.ops4j.pax.url.mvn
23:05:03 #log4j2.logger.maven.level = TRACE
23:05:03 log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN
23:05:03 log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN
23:05:03 Configure
23:05:03 java home: /usr/lib/jvm/java-21-openjdk-amd64
23:05:03 max memory: 3072m
23:05:03 memconf: /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:05:03 + set_java_vars /usr/lib/jvm/java-21-openjdk-amd64 3072m /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:05:03 + local -r java_home=/usr/lib/jvm/java-21-openjdk-amd64
23:05:03 + local -r controllermem=3072m
23:05:03 + local -r memconf=/tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:05:03 + echo Configure
23:05:03 + echo ' java home: /usr/lib/jvm/java-21-openjdk-amd64'
23:05:03 + echo ' max memory: 3072m'
23:05:03 + echo ' memconf: /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv'
23:05:03 + sed -ie 's%^# export JAVA_HOME%export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-21-openjdk-amd64}%g' /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:05:03 + sed -ie 's/JAVA_MAX_MEM="2048m"/JAVA_MAX_MEM=3072m/g' /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:05:03 cat /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:05:03 + echo 'cat /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv'
23:05:03 + cat /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:05:03 #!/bin/sh
23:05:03 #
23:05:03 # Licensed to the Apache Software Foundation (ASF) under one or more
23:05:03 # contributor license agreements. See the NOTICE file distributed with
23:05:03 # this work for additional information regarding copyright ownership.
23:05:03 # The ASF licenses this file to You under the Apache License, Version 2.0
23:05:03 # (the "License"); you may not use this file except in compliance with
23:05:03 # the License. You may obtain a copy of the License at
23:05:03 #
23:05:03 # http://www.apache.org/licenses/LICENSE-2.0
23:05:03 #
23:05:03 # Unless required by applicable law or agreed to in writing, software
23:05:03 # distributed under the License is distributed on an "AS IS" BASIS,
23:05:03 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23:05:03 # See the License for the specific language governing permissions and
23:05:03 # limitations under the License.
23:05:03 #
23:05:03
23:05:03 #
23:05:03 # handle specific scripts; the SCRIPT_NAME is exactly the name of the Karaf
23:05:03 # script: client, instance, shell, start, status, stop, karaf
23:05:03 #
23:05:03 # if [ "${KARAF_SCRIPT}" == "SCRIPT_NAME" ]; then
23:05:03 # Actions go here...
23:05:03 # fi
23:05:03
23:05:03 #
23:05:03 # general settings which should be applied for all scripts go here; please keep
23:05:03 # in mind that it is possible that scripts might be executed more than once, e.g.
23:05:03 # in example of the start script where the start script is executed first and the
23:05:03 # karaf script afterwards.
23:05:03 #
23:05:03
23:05:03 #
23:05:03 # The following section shows the possible configuration options for the default
23:05:03 # karaf scripts
23:05:03 #
23:05:03 export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-21-openjdk-amd64} # Location of Java installation
23:05:03 # export JAVA_OPTS # Generic JVM options, for instance, where you can pass the memory configuration
23:05:03 # export JAVA_NON_DEBUG_OPTS # Additional non-debug JVM options
23:05:03 # export EXTRA_JAVA_OPTS # Additional JVM options
23:05:03 # export KARAF_HOME # Karaf home folder
23:05:03 # export KARAF_DATA # Karaf data folder
23:05:03 # export KARAF_BASE # Karaf base folder
23:05:03 # export KARAF_ETC # Karaf etc folder
23:05:03 # export KARAF_LOG # Karaf log folder
23:05:03 # export KARAF_SYSTEM_OPTS # First citizen Karaf options
23:05:03 # export KARAF_OPTS # Additional available Karaf options
23:05:03 # export KARAF_DEBUG # Enable debug mode
23:05:03 # export KARAF_REDIRECT # Enable/set the std/err redirection when using bin/start
23:05:03 # export KARAF_NOROOT # Prevent execution as root if set to true
23:05:03 Set Java version
23:05:03 + echo 'Set Java version'
23:05:03 + sudo /usr/sbin/alternatives --install /usr/bin/java java /usr/lib/jvm/java-21-openjdk-amd64/bin/java 1
23:05:03 sudo: /usr/sbin/alternatives: command not found
23:05:03 + sudo /usr/sbin/alternatives --set java /usr/lib/jvm/java-21-openjdk-amd64/bin/java
23:05:03 sudo: /usr/sbin/alternatives: command not found
23:05:03 JDK default version ...
23:05:03 + echo 'JDK default version ...'
23:05:03 + java -version
23:05:03 openjdk version "11.0.25" 2024-10-15
23:05:03 OpenJDK Runtime Environment (build 11.0.25+9-post-Ubuntu-1ubuntu120.04)
23:05:03 OpenJDK 64-Bit Server VM (build 11.0.25+9-post-Ubuntu-1ubuntu120.04, mixed mode, sharing)
23:05:03 Set JAVA_HOME
23:05:03 + echo 'Set JAVA_HOME'
23:05:03 + export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
23:05:03 + JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
23:05:03 ++ readlink -e /usr/lib/jvm/java-21-openjdk-amd64/bin/java
23:05:03 Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-21-openjdk-amd64/bin/java
23:05:03 + JAVA_RESOLVED=/usr/lib/jvm/java-21-openjdk-amd64/bin/java
23:05:03 + echo 'Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-21-openjdk-amd64/bin/java'
23:05:03 Listing all open ports on controller system...
23:05:03 + echo 'Listing all open ports on controller system...'
23:05:03 + netstat -pnatu
23:05:03 /tmp/configuration-script.sh: line 40: netstat: command not found
23:05:03 + '[' -f /tmp/custom_shard_config.txt ']'
23:05:03 + echo 'Custom shard config exists!!!'
23:05:03 Custom shard config exists!!!
23:05:03 Copying the shard config...
23:05:03 + echo 'Copying the shard config...'
23:05:03 + cp /tmp/custom_shard_config.txt /tmp/karaf-0.22.0-SNAPSHOT/bin/
23:05:03 Configuring cluster
23:05:03 + echo 'Configuring cluster'
23:05:03 + /tmp/karaf-0.22.0-SNAPSHOT/bin/configure_cluster.sh 3 10.30.170.209 10.30.170.81 10.30.170.131
23:05:03 ################################################
23:05:03 ## Configure Cluster ##
23:05:03 ################################################
23:05:03 NOTE: Cluster configuration files not found. Copying from
23:05:03 /tmp/karaf-0.22.0-SNAPSHOT/system/org/opendaylight/controller/sal-clustering-config/10.0.2
23:05:03 Configuring unique name in akka.conf
23:05:03 Configuring hostname in akka.conf
23:05:03 Configuring data and rpc seed nodes in akka.conf
23:05:03 modules = [
23:05:03
23:05:03 {
23:05:03 name = "inventory"
23:05:03 namespace = "urn:opendaylight:inventory"
23:05:03 shard-strategy = "module"
23:05:03 },
23:05:03 {
23:05:03 name = "topology"
23:05:03 namespace = "urn:TBD:params:xml:ns:yang:network-topology"
23:05:03 shard-strategy = "module"
23:05:03 },
23:05:03 {
23:05:03 name = "toaster"
23:05:03 namespace = "http://netconfcentral.org/ns/toaster"
23:05:03 shard-strategy = "module"
23:05:03 },
23:05:03 {
23:05:03 name = "car"
23:05:03 namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car"
23:05:03 shard-strategy = "module"
23:05:03 },
23:05:03 {
23:05:03 name = "people"
23:05:03 namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people"
23:05:03 shard-strategy = "module"
23:05:03 },
23:05:03 {
23:05:03 name = "car-people"
23:05:03 namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people"
23:05:03 shard-strategy = "module"
23:05:03 }
23:05:03 ]
23:05:03 Configuring replication type in module-shards.conf
23:05:03 ################################################
23:05:03 ## NOTE: Manually restart controller to ##
23:05:03 ## apply configuration. ##
23:05:03 ################################################
23:05:03 Dump akka.conf
23:05:03 + echo 'Dump akka.conf'
23:05:03 + cat /tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/akka.conf
23:05:03
23:05:03 odl-cluster-data {
23:05:03 akka {
23:05:03 remote {
23:05:03 artery {
23:05:03 enabled = on
23:05:03 transport = tcp
23:05:03 canonical.hostname = "10.30.170.131"
23:05:03 canonical.port = 2550
23:05:03 }
23:05:03 }
23:05:03
23:05:03 cluster {
23:05:03 # Using artery.
23:05:03 seed-nodes = ["akka://opendaylight-cluster-data@10.30.170.209:2550",
23:05:03 "akka://opendaylight-cluster-data@10.30.170.81:2550",
23:05:03 "akka://opendaylight-cluster-data@10.30.170.131:2550"]
23:05:03
23:05:03 roles = ["member-3"]
23:05:03
23:05:03 # when under load we might trip a false positive on the failure detector
23:05:03 # failure-detector {
23:05:03 # heartbeat-interval = 4 s
23:05:03 # acceptable-heartbeat-pause = 16s
23:05:03 # }
23:05:03 }
23:05:03
23:05:03 persistence {
23:05:03 # By default the snapshots/journal directories live in KARAF_HOME. You can choose to put it somewhere else by
23:05:03 # modifying the following two properties. The directory location specified may be a relative or absolute path.
23:05:03 # The relative path is always relative to KARAF_HOME.
23:05:03
23:05:03 # snapshot-store.local.dir = "target/snapshots"
23:05:03
23:05:03 # Use lz4 compression for LocalSnapshotStore snapshots
23:05:03 snapshot-store.local.use-lz4-compression = false
23:05:03 # Size of blocks for lz4 compression: 64KB, 256KB, 1MB or 4MB
23:05:03 snapshot-store.local.lz4-blocksize = 256KB
23:05:03 }
23:05:03 disable-default-actor-system-quarantined-event-handling = "false"
23:05:03 }
23:05:03 }
23:05:03 Dump modules.conf
23:05:03 + echo 'Dump modules.conf'
23:05:03 + cat /tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/modules.conf
23:05:03 modules = [
23:05:03
23:05:03 {
23:05:03 name = "inventory"
23:05:03 namespace = "urn:opendaylight:inventory"
23:05:03 shard-strategy = "module"
23:05:03 },
23:05:03 {
23:05:03 name = "topology"
23:05:03 namespace = "urn:TBD:params:xml:ns:yang:network-topology"
23:05:03 shard-strategy = "module"
23:05:03 },
23:05:03 {
23:05:03 name = "toaster"
23:05:03 namespace = "http://netconfcentral.org/ns/toaster"
23:05:03 shard-strategy = "module"
23:05:03 },
23:05:03 {
23:05:03 name = "car"
23:05:03 namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car"
23:05:03 shard-strategy = "module"
23:05:03 },
23:05:03 {
23:05:03 name = "people"
23:05:03 namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people"
23:05:03 shard-strategy = "module"
23:05:03 },
23:05:03 {
23:05:03 name = "car-people"
23:05:03 namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people"
23:05:03 shard-strategy = "module"
23:05:03 }
23:05:03 ]
23:05:03 Dump module-shards.conf
23:05:03 + echo 'Dump module-shards.conf'
23:05:03 + cat /tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/module-shards.conf
23:05:03 module-shards = [
23:05:03 {
23:05:03 name = "default"
23:05:03 shards = [
23:05:03 {
23:05:03 name = "default"
23:05:03 replicas = ["member-1",
23:05:03 "member-2",
23:05:03 "member-3"]
23:05:03 }
23:05:03 ]
23:05:03 },
23:05:03 {
23:05:03 name = "inventory"
23:05:03 shards = [
23:05:03 {
23:05:03 name="inventory"
23:05:03 replicas = ["member-1",
23:05:03 "member-2",
23:05:03 "member-3"]
23:05:03 }
23:05:03 ]
23:05:03 },
23:05:03 {
23:05:03 name = "topology"
23:05:03 shards = [
23:05:03 {
23:05:03 name="topology"
23:05:03 replicas = ["member-1",
23:05:03 "member-2",
23:05:03 "member-3"]
23:05:03 }
23:05:03 ]
23:05:03 },
23:05:03 {
23:05:03 name = "toaster"
23:05:03 shards = [
23:05:03 {
23:05:03 name="toaster"
23:05:03 replicas = ["member-1",
23:05:03 "member-2",
23:05:03 "member-3"]
23:05:03 }
23:05:03 ]
23:05:03 },
23:05:03 {
23:05:03 name = "car"
23:05:03 shards = [
23:05:03 {
23:05:03 name="car"
23:05:03 replicas = ["member-1",
23:05:03 "member-2",
23:05:03 "member-3"]
23:05:03 }
23:05:03 ]
23:05:03 },
23:05:03 {
23:05:03 name = "people"
23:05:03 shards = [
23:05:03 {
23:05:03 name="people"
23:05:03 replicas = ["member-1",
23:05:03 "member-2",
23:05:03 "member-3"]
23:05:03 }
23:05:03 ]
23:05:03 },
23:05:03 {
23:05:03 name = "car-people"
23:05:03 shards = [
23:05:03 {
23:05:03 name="car-people"
23:05:03 replicas = ["member-1",
23:05:03 "member-2",
23:05:03 "member-3"]
23:05:03 }
23:05:03 ]
23:05:03 }
23:05:03 ]
23:05:03 Locating config plan to use...
23:05:03 Finished running config plans
23:05:03 Starting member-1 with IP address 10.30.170.209
23:05:03 Warning: Permanently added '10.30.170.209' (ECDSA) to the list of known hosts.
23:05:03 Warning: Permanently added '10.30.170.209' (ECDSA) to the list of known hosts.
23:05:04 Redirecting karaf console output to karaf_console.log
23:05:04 Starting controller...
23:05:04 start: Redirecting Karaf output to /tmp/karaf-0.22.0-SNAPSHOT/data/log/karaf_console.log
23:05:04 Starting member-2 with IP address 10.30.170.81
23:05:04 Warning: Permanently added '10.30.170.81' (ECDSA) to the list of known hosts.
23:05:04 Warning: Permanently added '10.30.170.81' (ECDSA) to the list of known hosts.
23:05:05 Redirecting karaf console output to karaf_console.log
23:05:05 Starting controller...
23:05:05 start: Redirecting Karaf output to /tmp/karaf-0.22.0-SNAPSHOT/data/log/karaf_console.log
23:05:05 Starting member-3 with IP address 10.30.170.131
23:05:05 Warning: Permanently added '10.30.170.131' (ECDSA) to the list of known hosts.
23:05:05 Warning: Permanently added '10.30.170.131' (ECDSA) to the list of known hosts.
23:05:05 Redirecting karaf console output to karaf_console.log
23:05:05 Starting controller...
23:05:05 start: Redirecting Karaf output to /tmp/karaf-0.22.0-SNAPSHOT/data/log/karaf_console.log
23:05:05 [controller-csit-3node-clustering-ask-all-titanium] $ /bin/bash /tmp/jenkins16063541296799358040.sh
23:05:05 common-functions.sh is being sourced
23:05:05 common-functions environment:
23:05:05 MAVENCONF: /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:05:05 ACTUALFEATURES:
23:05:05 FEATURESCONF: /tmp/karaf-0.22.0-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:05:05 CUSTOMPROP: /tmp/karaf-0.22.0-SNAPSHOT/etc/custom.properties
23:05:05 LOGCONF: /tmp/karaf-0.22.0-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:05:05 MEMCONF: /tmp/karaf-0.22.0-SNAPSHOT/bin/setenv
23:05:05 CONTROLLERMEM: 2048m
23:05:05 AKKACONF: /tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/akka.conf
23:05:05 MODULESCONF: /tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/modules.conf
23:05:05 MODULESHARDSCONF: /tmp/karaf-0.22.0-SNAPSHOT/configuration/initial/module-shards.conf
23:05:05 SUITES:
23:05:05
23:05:05 + echo '#################################################'
23:05:05 #################################################
23:05:05 + echo '## Verify Cluster is UP ##'
23:05:05 ## Verify Cluster is UP ##
23:05:05 + echo '#################################################'
23:05:05 #################################################
23:05:05 + create_post_startup_script
23:05:05 + cat
23:05:05 + copy_and_run_post_startup_script
23:05:05 + seed_index=1
23:05:05 ++ seq 1 3
23:05:05 + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
23:05:05 + CONTROLLERIP=ODL_SYSTEM_1_IP
23:05:05 + echo 'Execute the post startup script on controller 10.30.170.209'
23:05:05 Execute the post startup script on controller 10.30.170.209
23:05:05 + scp /w/workspace/controller-csit-3node-clustering-ask-all-titanium/post-startup-script.sh 10.30.170.209:/tmp/
23:05:05 Warning: Permanently added '10.30.170.209' (ECDSA) to the list of known hosts.
23:05:06 + ssh 10.30.170.209 'bash /tmp/post-startup-script.sh 1'
23:05:06 Warning: Permanently added '10.30.170.209' (ECDSA) to the list of known hosts.
23:05:06 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:05:11 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:05:16 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:05:21 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:05:26 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:05:31 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:05:36 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:05:41 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:05:46 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:05:51 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:05:56 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:06:01 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:06:06 Waiting up to 3 minutes for controller to come up, checking every 5 seconds...
23:06:11 2025-03-16T23:05:58,609 | INFO | SystemReadyService-0 | SimpleSystemReadyMonitor | 292 - org.opendaylight.infrautils.ready-api - 7.0.3 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.]
23:06:11 Controller is UP
23:06:11 2025-03-16T23:05:58,609 | INFO | SystemReadyService-0 | SimpleSystemReadyMonitor | 292 - org.opendaylight.infrautils.ready-api - 7.0.3 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.]
23:06:11 Listing all open ports on controller system...
23:06:11 /tmp/post-startup-script.sh: line 51: netstat: command not found
23:06:11 looking for "BindException: Address already in use" in log file
23:06:11 looking for "server is unhealthy" in log file
23:06:11 + '[' 1 == 0 ']'
23:06:11 + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
23:06:11 + CONTROLLERIP=ODL_SYSTEM_2_IP
23:06:11 + echo 'Execute the post startup script on controller 10.30.170.81'
23:06:11 Execute the post startup script on controller 10.30.170.81
23:06:11 + scp /w/workspace/controller-csit-3node-clustering-ask-all-titanium/post-startup-script.sh 10.30.170.81:/tmp/
23:06:11 Warning: Permanently added '10.30.170.81' (ECDSA) to the list of known hosts.
23:06:12 + ssh 10.30.170.81 'bash /tmp/post-startup-script.sh 2'
23:06:12 Warning: Permanently added '10.30.170.81' (ECDSA) to the list of known hosts.
23:06:12 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:06:17 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:06:22 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:06:27 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:06:32 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:06:37 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:06:42 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:06:47 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:06:52 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:06:57 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:07:02 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:07:07 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:07:12 Waiting up to 3 minutes for controller to come up, checking every 5 seconds...
23:07:17 2025-03-16T23:05:58,544 | INFO | SystemReadyService-0 | SimpleSystemReadyMonitor | 292 - org.opendaylight.infrautils.ready-api - 7.0.3 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.]
23:07:17 Controller is UP
23:07:17 2025-03-16T23:05:58,544 | INFO | SystemReadyService-0 | SimpleSystemReadyMonitor | 292 - org.opendaylight.infrautils.ready-api - 7.0.3 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.]
23:07:17 Listing all open ports on controller system...
23:07:17 /tmp/post-startup-script.sh: line 51: netstat: command not found
23:07:17 looking for "BindException: Address already in use" in log file
23:07:17 looking for "server is unhealthy" in log file
23:07:17 + '[' 2 == 0 ']'
23:07:17 + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
23:07:17 + CONTROLLERIP=ODL_SYSTEM_3_IP
23:07:17 + echo 'Execute the post startup script on controller 10.30.170.131'
23:07:17 Execute the post startup script on controller 10.30.170.131
23:07:17 + scp /w/workspace/controller-csit-3node-clustering-ask-all-titanium/post-startup-script.sh 10.30.170.131:/tmp/
23:07:17 Warning: Permanently added '10.30.170.131' (ECDSA) to the list of known hosts.
23:07:17 + ssh 10.30.170.131 'bash /tmp/post-startup-script.sh 3'
23:07:17 Warning: Permanently added '10.30.170.131' (ECDSA) to the list of known hosts.
23:07:18 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:07:23 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:07:28 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:07:33 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:07:38 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:07:43 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:07:48 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:07:53 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:07:58 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:08:03 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:08:08 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:08:13 /tmp/post-startup-script.sh: line 4: netstat: command not found
23:08:18 Waiting up to 3 minutes for controller to come up, checking every 5 seconds...
23:08:23 2025-03-16T23:05:58,037 | INFO | SystemReadyService-0 | SimpleSystemReadyMonitor | 292 - org.opendaylight.infrautils.ready-api - 7.0.3 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.]
23:08:23 Controller is UP
23:08:23 2025-03-16T23:05:58,037 | INFO | SystemReadyService-0 | SimpleSystemReadyMonitor | 292 - org.opendaylight.infrautils.ready-api - 7.0.3 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.]
23:08:23 Listing all open ports on controller system...
23:08:23 /tmp/post-startup-script.sh: line 51: netstat: command not found
23:08:23 looking for "BindException: Address already in use" in log file
23:08:23 looking for "server is unhealthy" in log file
23:08:23 + '[' 0 == 0 ']'
23:08:23 + seed_index=1
23:08:23 + dump_controller_threads
23:08:23 ++ seq 1 3
23:08:23 + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
23:08:23 + CONTROLLERIP=ODL_SYSTEM_1_IP
23:08:23 + echo 'Let'\''s take the karaf thread dump'
23:08:23 Let's take the karaf thread dump
23:08:23 + ssh 10.30.170.209 'sudo ps aux'
23:08:23 Warning: Permanently added '10.30.170.209' (ECDSA) to the list of known hosts.
23:08:23 ++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-ask-all-titanium/ps_before.log
23:08:23 ++ grep -v grep
23:08:23 ++ cut -f2 '-d '
23:08:23 ++ tr -s ' '
23:08:23 + pid=28758
23:08:23 + echo 'karaf main: org.apache.karaf.main.Main, pid:28758'
23:08:23 karaf main: org.apache.karaf.main.Main, pid:28758
23:08:23 + ssh 10.30.170.209 '/usr/lib/jvm/java-21-openjdk-amd64/bin/jstack -l 28758'
23:08:23 Warning: Permanently added '10.30.170.209' (ECDSA) to the list of known hosts.
23:08:24 + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
23:08:24 + CONTROLLERIP=ODL_SYSTEM_2_IP
23:08:24 + echo 'Let'\''s take the karaf thread dump'
23:08:24 Let's take the karaf thread dump
23:08:24 + ssh 10.30.170.81 'sudo ps aux'
23:08:24 Warning: Permanently added '10.30.170.81' (ECDSA) to the list of known hosts.
23:08:24 ++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-ask-all-titanium/ps_before.log
23:08:24 ++ grep -v grep
23:08:24 ++ tr -s ' '
23:08:24 ++ cut -f2 '-d '
23:08:24 + pid=28775
23:08:24 + echo 'karaf main: org.apache.karaf.main.Main, pid:28775'
23:08:24 karaf main: org.apache.karaf.main.Main, pid:28775
23:08:24 + ssh 10.30.170.81 '/usr/lib/jvm/java-21-openjdk-amd64/bin/jstack -l 28775'
23:08:24 Warning: Permanently added '10.30.170.81' (ECDSA) to the list of known hosts.
23:08:25 + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
23:08:25 + CONTROLLERIP=ODL_SYSTEM_3_IP
23:08:25 + echo 'Let'\''s take the karaf thread dump'
23:08:25 Let's take the karaf thread dump
23:08:25 + ssh 10.30.170.131 'sudo ps aux'
23:08:25 Warning: Permanently added '10.30.170.131' (ECDSA) to the list of known hosts.
23:08:26 ++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-ask-all-titanium/ps_before.log
23:08:26 ++ grep -v grep
23:08:26 ++ cut -f2 '-d '
23:08:26 ++ tr -s ' '
23:08:26 + pid=28770
23:08:26 + echo 'karaf main: org.apache.karaf.main.Main, pid:28770'
23:08:26 karaf main: org.apache.karaf.main.Main, pid:28770
23:08:26 + ssh 10.30.170.131 '/usr/lib/jvm/java-21-openjdk-amd64/bin/jstack -l 28770'
23:08:26 Warning: Permanently added '10.30.170.131' (ECDSA) to the list of known hosts.
23:08:26 + '[' 0 -gt 0 ']'
23:08:26 + echo 'Generating controller variables...'
23:08:26 Generating controller variables...
23:08:26 ++ seq 1 3
23:08:26 + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
23:08:26 + CONTROLLERIP=ODL_SYSTEM_1_IP
23:08:26 + odl_variables=' -v ODL_SYSTEM_1_IP:10.30.170.209'
23:08:26 + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
23:08:26 + CONTROLLERIP=ODL_SYSTEM_2_IP
23:08:26 + odl_variables=' -v ODL_SYSTEM_1_IP:10.30.170.209 -v ODL_SYSTEM_2_IP:10.30.170.81'
23:08:26 + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
23:08:26 + CONTROLLERIP=ODL_SYSTEM_3_IP
23:08:26 + odl_variables=' -v ODL_SYSTEM_1_IP:10.30.170.209 -v ODL_SYSTEM_2_IP:10.30.170.81 -v ODL_SYSTEM_3_IP:10.30.170.131'
23:08:26 + echo 'Generating mininet variables...'
23:08:26 Generating mininet variables...
23:08:26 ++ seq 1 0
23:08:26 + get_test_suites SUITES
23:08:26 + local __suite_list=SUITES
23:08:26 + echo 'Locating test plan to use...'
23:08:26 Locating test plan to use...
23:08:26 + testplan_filepath=/w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/testplans/controller-clustering-ask-titanium.txt
23:08:26 + '[' '!' -f /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/testplans/controller-clustering-ask-titanium.txt ']'
23:08:26 + testplan_filepath=/w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/testplans/controller-clustering-ask.txt
23:08:26 + '[' disabled '!=' disabled ']'
23:08:26 + echo 'Changing the testplan path...'
23:08:26 Changing the testplan path...
23:08:26 + sed s:integration:/w/workspace/controller-csit-3node-clustering-ask-all-titanium: /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/testplans/controller-clustering-ask.txt
23:08:26 + cat testplan.txt
23:08:26 # Place the suites in run order:
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_precedence.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_partition_and_heal.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/action_provider_precedence.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/action_provider_partition_and_heal.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/cluster_singleton/master_stability.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/singleton_service/global_rpc_kill.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/singleton_service/global_rpc_freeze.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/singleton_service/global_rpc_isolate.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot
23:08:26 /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot
23:08:26 + '[' -z '' ']'
23:08:26 ++ grep -E -v '(^[[:space:]]*#|^[[:space:]]*$)' testplan.txt
23:08:26 ++ tr '\012' ' '
23:08:26 + suite_list='/w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/action_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/action_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/singleton_service/global_rpc_kill.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/singleton_service/global_rpc_freeze.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/singleton_service/global_rpc_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot '
23:08:26 + eval 'SUITES='\''/w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/action_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/action_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/singleton_service/global_rpc_kill.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/singleton_service/global_rpc_freeze.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/singleton_service/global_rpc_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot '\'''
23:08:26 ++ SUITES='/w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/action_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/action_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/singleton_service/global_rpc_kill.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/singleton_service/global_rpc_freeze.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/singleton_service/global_rpc_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot '
23:08:26 + echo 'Starting Robot test suites /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/action_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/action_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/singleton_service/global_rpc_kill.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/singleton_service/global_rpc_freeze.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/singleton_service/global_rpc_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot ...'
23:08:26 Starting Robot test suites /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/action_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/action_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/singleton_service/global_rpc_kill.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/singleton_service/global_rpc_freeze.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/singleton_service/global_rpc_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot ...
23:08:26 + robot -N controller-clustering-ask.txt --removekeywords wuks -e exclude -e skip_if_titanium -v BUNDLEFOLDER:karaf-0.22.0-SNAPSHOT -v BUNDLE_URL:https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.22.0-SNAPSHOT/karaf-0.22.0-20250316.175440-1102.zip -v CONTROLLER:10.30.170.209 -v CONTROLLER1:10.30.170.81 -v CONTROLLER2:10.30.170.131 -v CONTROLLER_USER:jenkins -v JAVA_HOME:/usr/lib/jvm/java-21-openjdk-amd64 -v JDKVERSION:openjdk21 -v JENKINS_WORKSPACE:/w/workspace/controller-csit-3node-clustering-ask-all-titanium -v MININET: -v MININET1: -v MININET2: -v MININET_USER:jenkins -v NEXUSURL_PREFIX:https://nexus.opendaylight.org -v NUM_ODL_SYSTEM:3 -v NUM_TOOLS_SYSTEM:0 -v ODL_STREAM:titanium -v ODL_SYSTEM_IP:10.30.170.209 -v ODL_SYSTEM_1_IP:10.30.170.209 -v ODL_SYSTEM_2_IP:10.30.170.81 -v ODL_SYSTEM_3_IP:10.30.170.131 -v ODL_SYSTEM_USER:jenkins -v TOOLS_SYSTEM_IP: -v TOOLS_SYSTEM_USER:jenkins -v USER_HOME:/home/jenkins -v IS_KARAF_APPL:True -v WORKSPACE:/tmp /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/action_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_rpc_broker/action_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/singleton_service/global_rpc_kill.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/singleton_service/global_rpc_freeze.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/singleton_service/global_rpc_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-titanium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot
23:08:26 ==============================================================================
23:08:26 controller-clustering-ask.txt
23:08:26 ==============================================================================
23:08:27 controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te...
23:08:27 ==============================================================================
23:08:32 Stop_All_Members :: Stop every odl node. | PASS |
23:12:43 ------------------------------------------------------------------------------
23:12:43 Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS |
23:12:53 ------------------------------------------------------------------------------
23:12:53 Start_All_And_Sync :: Start each member and wait for sync. | FAIL |
23:13:54 Got rc: 1 or stderr was not empty: sudo: netstat: command not found
23:13:54 ------------------------------------------------------------------------------
23:13:54 controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL |
23:13:54 3 tests, 2 passed, 1 failed
23:13:54 ==============================================================================
23:13:54 controller-clustering-ask.txt.Rpc Provider Precedence :: DOMRpcBroker testi...
23:13:54 ==============================================================================
23:14:00 Register_Rpc_On_Each_Node :: Register global rpc on each node of t... | PASS |
23:14:01 ------------------------------------------------------------------------------
23:14:01 Invoke_Rpc_On_Each_Node :: Verify that the rpc response comes from... | PASS |
23:14:02 ------------------------------------------------------------------------------
23:14:02 Unregister_Rpc_On_Node :: Unregister the rpc on one of the cluster... | PASS |
23:14:03 ------------------------------------------------------------------------------
23:14:03 Invoke_Rpc_On_Node_With_Unregistered_Rpc :: Invoke rcp on the node... | PASS |
23:14:03 ------------------------------------------------------------------------------
23:14:03 Invoke_Rpc_On_Remaining_Nodes :: Verify that the rpc response come... | PASS |
23:14:04 ------------------------------------------------------------------------------
23:14:04 Reregister_Rpc_On_Node :: Reregister the rpc. | PASS |
23:14:04 ------------------------------------------------------------------------------
23:14:04 Invoke_Rpc_On_Each_Node_Again :: Verify that the rpc response come... | PASS |
23:14:05 ------------------------------------------------------------------------------
23:14:05 Unregister_Rpc_On_Each_Node :: Unregister rpc on every node. | PASS |
23:14:06 ------------------------------------------------------------------------------
23:14:06 controller-clustering-ask.txt.Rpc Provider Precedence :: DOMRpcBro... | PASS |
23:14:06 8 tests, 8 passed, 0 failed
23:14:06 ==============================================================================
23:14:06 controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te...
23:14:06 ==============================================================================
23:14:08 Stop_All_Members :: Stop every odl node. | PASS |
23:18:21 ------------------------------------------------------------------------------
23:18:21 Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS |
23:18:31 ------------------------------------------------------------------------------
23:18:31 Start_All_And_Sync :: Start each member and wait for sync. | FAIL |
23:19:33 Got rc: 1 or stderr was not empty: sudo: netstat: command not found
23:19:33 ------------------------------------------------------------------------------
23:19:33 controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL |
23:19:33 3 tests, 2 passed, 1 failed
23:19:33 ==============================================================================
23:19:33 controller-clustering-ask.txt.Rpc Provider Partition And Heal :: DOMRpcBrok...
23:19:33 ==============================================================================
23:19:37 Register_Rpc_On_Two_Nodes :: Register rpc on two nodes of the odl ... | PASS |
23:19:38 ------------------------------------------------------------------------------
23:19:38 Invoke_Rpc_On_Each_Node :: Invoke get-constant rpc on every node o... | PASS |
23:19:39 ------------------------------------------------------------------------------
23:19:39 Isolate_One_Node :: Isolate one node with registered rpc. From the... | PASS |
23:19:41 ------------------------------------------------------------------------------
23:19:41 Invoke_Rpc_On_Nonisolated_Nodes :: Invoke rpc on non-islolated nodes. | PASS |
23:20:03 ------------------------------------------------------------------------------
23:20:03 Rejoin_Isolated_Member :: Rejoin isolated node | FAIL |
23:21:05 ... click for list of related bugs or create a new one if needed (with the
23:21:05 "controller_clustering_ask_txt_rpc_provider_partition_and_heal_rejoin_isolated_member"
23:21:05 reference somewhere inside)
23:21:05 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_rpc_provider_partition_and_heal_rejoin_isolated_member&order=bug_status"
23:21:05
23:21:05 Keyword 'Check_Cluster_Is_In_Sync' failed after retrying for 1 minute. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore
23:21:05 ------------------------------------------------------------------------------
23:21:05 Invoke_Rpc_On_Each_Node_Again :: Invoke rpc get-constant on every ... | PASS |
23:21:26 ------------------------------------------------------------------------------
23:21:26 Unregister_Rpc_On_Each_Node :: Inregister rpc on both nodes. | FAIL |
23:21:26 ... click for list of related bugs or create a new one if needed (with the
23:21:26 "controller_clustering_ask_txt_rpc_provider_partition_and_heal_unregister_rpc_on_each_node"
23:21:26 reference somewhere inside)
23:21:26 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_rpc_provider_partition_and_heal_unregister_rpc_on_each_node&order=bug_status"
23:21:26
23:21:26 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/operations/odl-mdsal-lowlevel-control:unregister-constant
23:21:26 ------------------------------------------------------------------------------
23:21:26 controller-clustering-ask.txt.Rpc Provider Partition And Heal :: D... | FAIL |
23:21:26 7 tests, 5 passed, 2 failed
23:21:26 ==============================================================================
23:21:26 controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te...
23:21:26 ==============================================================================
23:21:29 Stop_All_Members :: Stop every odl node. | FAIL |
23:27:33 Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
23:27:33 ------------------------------------------------------------------------------
23:27:33 Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS |
23:27:44 ------------------------------------------------------------------------------
23:27:44 Start_All_And_Sync :: Start each member and wait for sync. | FAIL |
23:33:49 Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore
23:33:49 ------------------------------------------------------------------------------
23:33:49 controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL |
23:33:49 3 tests, 1 passed, 2 failed
23:33:49 ==============================================================================
23:33:49 controller-clustering-ask.txt.Action Provider Precedence :: DOMRpcBroker te...
23:33:49 ==============================================================================
23:33:54 Register_Rpc_On_Each_Node :: Register routed rpc on each node of t... | FAIL |
23:33:54 ... click for list of related bugs or create a new one if needed (with the
23:33:54 "controller_clustering_ask_txt_action_provider_precedence_register_rpc_on_each_node"
23:33:54 reference somewhere inside)
23:33:54 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_precedence_register_rpc_on_each_node&order=bug_status"
23:33:54
23:33:54 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/operations/odl-mdsal-lowlevel-control:register-bound-constant
23:33:54 ------------------------------------------------------------------------------
23:33:54 Invoke_Rpc_On_Each_Node :: Verify that the rpc response comes from... | FAIL |
23:33:55 ... click for list of related bugs or create a new one if needed (with the
23:33:55 "controller_clustering_ask_txt_action_provider_precedence_invoke_rpc_on_each_node"
23:33:55 reference somewhere inside)
23:33:55 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_precedence_invoke_rpc_on_each_node&order=bug_status"
23:33:55
23:33:55 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/operations/odl-mdsal-lowlevel-target:get-contexted-constant
23:33:55 ------------------------------------------------------------------------------
23:33:55 Unregister_Rpc_On_Node :: Unregister the rpc on one of the cluster... | FAIL |
23:33:55 ... click for list of related bugs or create a new one if needed (with the
23:33:55 "controller_clustering_ask_txt_action_provider_precedence_unregister_rpc_on_node"
23:33:55 reference somewhere inside)
23:33:55 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_precedence_unregister_rpc_on_node&order=bug_status"
23:33:55
23:33:55 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/operations/odl-mdsal-lowlevel-control:unregister-bound-constant
23:33:55 ------------------------------------------------------------------------------
23:33:55 Invoke_Rpc_On_Node_With_Unregistered_Rpc :: Invoke rcp on the node... | FAIL |
23:33:56 ... click for list of related bugs or create a new one if needed (with the
23:33:56 "controller_clustering_ask_txt_action_provider_precedence_invoke_rpc_on_node_with_unregistered_rpc"
23:33:56 reference somewhere inside)
23:33:56 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_precedence_invoke_rpc_on_node_with_unregistered_rpc&order=bug_status"
23:33:56
23:33:56 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/operations/odl-mdsal-lowlevel-target:get-contexted-constant
23:33:56 ------------------------------------------------------------------------------
23:33:56 Invoke_Rpc_On_Remaining_Nodes :: Verify that the rpc response come... | PASS |
23:33:56 ------------------------------------------------------------------------------
23:33:56 Reregister_Rpc_On_Node :: Reregister the rpc. | FAIL |
23:33:56 ... click for list of related bugs or create a new one if needed (with the
23:33:56 "controller_clustering_ask_txt_action_provider_precedence_reregister_rpc_on_node"
23:33:56 reference somewhere inside)
23:33:56 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_precedence_reregister_rpc_on_node&order=bug_status"
23:33:56
23:33:56 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/operations/odl-mdsal-lowlevel-control:register-bound-constant
23:33:56 ------------------------------------------------------------------------------
23:33:56 Invoke_Rpc_On_Each_Node_Again :: Verify that the rpc response come... | FAIL |
23:33:57 ... click for list of related bugs or create a new one if needed (with the
23:33:57 "controller_clustering_ask_txt_action_provider_precedence_invoke_rpc_on_each_node_again"
23:33:57 reference somewhere inside)
23:33:57 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_precedence_invoke_rpc_on_each_node_again&order=bug_status"
23:33:57
23:33:57 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/operations/odl-mdsal-lowlevel-target:get-contexted-constant
23:33:57 ------------------------------------------------------------------------------
23:33:57 Unregister_Rpc_On_Each_Node :: Unregister rpc on every node. | FAIL |
23:33:57 ... click for list of related bugs or create a new one if needed (with the
23:33:57 "controller_clustering_ask_txt_action_provider_precedence_unregister_rpc_on_each_node"
23:33:57 reference somewhere inside)
23:33:57 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_precedence_unregister_rpc_on_each_node&order=bug_status"
23:33:57
23:33:57 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/operations/odl-mdsal-lowlevel-control:unregister-bound-constant
23:33:57 ------------------------------------------------------------------------------
23:33:57 controller-clustering-ask.txt.Action Provider Precedence :: DOMRpc... | FAIL |
23:33:57 8 tests, 1 passed, 7 failed
23:33:57 ==============================================================================
23:33:57 controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te...
23:33:57 ==============================================================================
23:34:00 Stop_All_Members :: Stop every odl node. | FAIL |
23:40:04 Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
23:40:04 ------------------------------------------------------------------------------
23:40:04 Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS |
23:40:14 ------------------------------------------------------------------------------
23:40:14 Start_All_And_Sync :: Start each member and wait for sync. | FAIL |
23:46:19 Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore
23:46:19 ------------------------------------------------------------------------------
23:46:19 controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL |
23:46:19 3 tests, 1 passed, 2 failed
23:46:19 ==============================================================================
23:46:20 controller-clustering-ask.txt.Action Provider Partition And Heal :: DOMRpcB...
23:46:20 ==============================================================================
23:46:24 Register_Rpc_On_Two_Nodes :: Register rpc on two nodes of the odl ... | FAIL |
23:46:24 ... click for list of related bugs or create a new one if needed (with the
23:46:24 "controller_clustering_ask_txt_action_provider_partition_and_heal_register_rpc_on_two_nodes"
23:46:24 reference somewhere inside)
23:46:24 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_partition_and_heal_register_rpc_on_two_nodes&order=bug_status"
23:46:24
23:46:24 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/operations/odl-mdsal-lowlevel-control:register-bound-constant
23:46:24 ------------------------------------------------------------------------------
23:46:24 Invoke_Rpc_On_Each_Node :: Invoke get-contexted-constant rpc on ev... | FAIL |
23:46:36 ... click for list of related bugs or create a new one if needed (with the
23:46:36 "controller_clustering_ask_txt_action_provider_partition_and_heal_invoke_rpc_on_each_node"
23:46:36 reference somewhere inside)
23:46:36 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_partition_and_heal_invoke_rpc_on_each_node&order=bug_status"
23:46:36
23:46:36 Keyword 'Verify_Contexted_Constant_On_Active_Nodes' failed after retrying for 10 seconds. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/operations/odl-mdsal-lowlevel-target:get-contexted-constant
23:46:36 ------------------------------------------------------------------------------
23:46:36 Isolate_One_Node :: Isolate one node with registered rpc. | FAIL |
23:46:36 ... click for list of related bugs or create a new one if needed (with the
23:46:36 "controller_clustering_ask_txt_action_provider_partition_and_heal_isolate_one_node"
23:46:36 reference somewhere inside)
23:46:36 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_partition_and_heal_isolate_one_node&order=bug_status"
23:46:36
23:46:36 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.131:8181/rests/operations/odl-mdsal-lowlevel-target:get-contexted-constant
23:46:36 ------------------------------------------------------------------------------
23:46:36 Invoke_Rpc_On_Remaining_Nodes :: Invoke rpc on non-islolated nodes... | FAIL |
23:47:51 ... click for list of related bugs or create a new one if needed (with the
23:47:51 "controller_clustering_ask_txt_action_provider_partition_and_heal_invoke_rpc_on_remaining_nodes"
23:47:51 reference somewhere inside)
23:47:51 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_partition_and_heal_invoke_rpc_on_remaining_nodes&order=bug_status"
23:47:51
23:47:51 Keyword 'DrbCommons.Verify_Contexted_Constant_On_Active_Nodes' failed after retrying for 1 minute. The last error was: Keyword 'Verify_Contexted_Constant_On_Active_Nodes' failed after retrying for 10 seconds. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/operations/odl-mdsal-lowlevel-target:get-contexted-constant
23:47:51 ------------------------------------------------------------------------------
23:47:51 Rejoin_Isolated_Member :: Rejoin isolated node | FAIL |
23:47:51 ... click for list of related bugs or create a new one if needed (with the
23:47:51 "controller_clustering_ask_txt_action_provider_partition_and_heal_rejoin_isolated_member"
23:47:51 reference somewhere inside)
23:47:51 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_partition_and_heal_rejoin_isolated_member&order=bug_status"
23:47:51
23:47:51 Variable '${isolated_idx}' not found.
23:47:51 ------------------------------------------------------------------------------
23:47:51 Invoke_Rpc_On_Each_Node_Again :: Invoke rpc get-contexted-constant... | FAIL |
23:48:02 ... click for list of related bugs or create a new one if needed (with the
23:48:02 "controller_clustering_ask_txt_action_provider_partition_and_heal_invoke_rpc_on_each_node_again"
23:48:02 reference somewhere inside)
23:48:02 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_partition_and_heal_invoke_rpc_on_each_node_again&order=bug_status"
23:48:02
23:48:02 Keyword 'Verify_Contexted_Constant_On_Active_Nodes' failed after retrying for 10 seconds. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/operations/odl-mdsal-lowlevel-target:get-contexted-constant
23:48:02 ------------------------------------------------------------------------------
23:48:02 Unregister_Rpc_On_Each_Node :: Inregister rpc on both nodes. | FAIL |
23:48:03 ... click for list of related bugs or create a new one if needed (with the
23:48:03 "controller_clustering_ask_txt_action_provider_partition_and_heal_unregister_rpc_on_each_node"
23:48:03 reference somewhere inside)
23:48:03 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_partition_and_heal_unregister_rpc_on_each_node&order=bug_status"
23:48:03
23:48:03 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/operations/odl-mdsal-lowlevel-control:unregister-bound-constant
23:48:03 ------------------------------------------------------------------------------
23:48:03 controller-clustering-ask.txt.Action Provider Partition And Heal :... | FAIL |
23:48:03 7 tests, 0 passed, 7 failed
23:48:03 ==============================================================================
23:48:03 controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te...
23:48:03 ==============================================================================
23:48:06 Stop_All_Members :: Stop every odl node. | FAIL |
23:54:08 Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
23:54:08 ------------------------------------------------------------------------------
23:54:08 Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS |
23:54:19 ------------------------------------------------------------------------------
23:54:19 Start_All_And_Sync :: Start each member and wait for sync. | FAIL |
00:00:25 Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore
00:00:25 ------------------------------------------------------------------------------
00:00:25 controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL |
00:00:25 3 tests, 1 passed, 2 failed
00:00:25 ==============================================================================
00:00:25 controller-clustering-ask.txt.Master Stability :: Cluster Singleton testing...
00:00:25 ==============================================================================
00:00:28 Register_Singleton_Constant_On_Each_Node_And_Verify :: Register a ... | FAIL |
00:00:29 ... click for list of related bugs or create a new one if needed (with the
00:00:29 "controller_clustering_ask_txt_master_stability_register_singleton_constant_on_each_node_and_verify"
00:00:29 reference somewhere inside)
00:00:29 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_master_stability_register_singleton_constant_on_each_node_and_verify&order=bug_status"
00:00:29
00:00:29 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/operations/odl-mdsal-lowlevel-control:register-singleton-constant
00:00:29 ------------------------------------------------------------------------------
00:00:29 Unregister_Singleton_Constant_On_Non_Master_Node :: Unregister the... | FAIL |
00:00:29 ... click for list of related bugs or create a new one if needed (with the
00:00:29 "controller_clustering_ask_txt_master_stability_unregister_singleton_constant_on_non_master_node"
00:00:29 reference somewhere inside)
00:00:29 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_master_stability_unregister_singleton_constant_on_non_master_node&order=bug_status"
00:00:29
00:00:29 Variable '@{cs_candidates}' not found. Did you mean:
00:00:29 @{cs_exp_candidates}
00:00:29 ------------------------------------------------------------------------------
00:00:29 Monitor_Stability_While_Unregistered :: Verify that the owner rema... | FAIL |
00:00:30 ... click for list of related bugs or create a new one if needed (with the
00:00:30 "controller_clustering_ask_txt_master_stability_monitor_stability_while_unregistered"
00:00:30 reference somewhere inside)
00:00:30 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_master_stability_monitor_stability_while_unregistered&order=bug_status"
00:00:30
00:00:30 Variable '${cs_owner}' not found.
00:00:30 ------------------------------------------------------------------------------
00:00:30 Reregister_Singleton_Constant :: Re-register the unregistered cand... | FAIL |
00:00:30 ... click for list of related bugs or create a new one if needed (with the
00:00:30 "controller_clustering_ask_txt_master_stability_reregister_singleton_constant"
00:00:30 reference somewhere inside)
00:00:30 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_master_stability_reregister_singleton_constant&order=bug_status"
00:00:30
00:00:30 Variable '${unregistered_node}' not found.
00:00:30 ------------------------------------------------------------------------------
00:00:30 Verify_Stability_After_Reregistration :: Verify that the owner rem... | FAIL |
00:00:30 ... click for list of related bugs or create a new one if needed (with the
00:00:30 "controller_clustering_ask_txt_master_stability_verify_stability_after_reregistration"
00:00:30 reference somewhere inside)
00:00:30 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_master_stability_verify_stability_after_reregistration&order=bug_status"
00:00:30
00:00:30 Variable '${cs_owner}' not found.
00:00:30 ------------------------------------------------------------------------------
00:00:30 Unregister_Singleton_Constant_On_Each_Node :: Unregister the appli... | FAIL |
00:00:31 ... click for list of related bugs or create a new one if needed (with the
00:00:31 "controller_clustering_ask_txt_master_stability_unregister_singleton_constant_on_each_node"
00:00:31 reference somewhere inside)
00:00:31 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_master_stability_unregister_singleton_constant_on_each_node&order=bug_status"
00:00:31
00:00:31 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/operations/odl-mdsal-lowlevel-control:unregister-singleton-constant
00:00:31 ------------------------------------------------------------------------------
00:00:31 controller-clustering-ask.txt.Master Stability :: Cluster Singleto... | FAIL |
00:00:31 6 tests, 0 passed, 6 failed
00:00:31 ==============================================================================
00:00:31 controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te...
00:00:31 ==============================================================================
00:00:33 Stop_All_Members :: Stop every odl node. | FAIL |
00:06:38 Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
00:06:38 ------------------------------------------------------------------------------
00:06:38 Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS |
00:06:48 ------------------------------------------------------------------------------
00:06:48 Start_All_And_Sync :: Start each member and wait for sync. | FAIL |
00:12:54 Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore
00:12:54 ------------------------------------------------------------------------------
00:12:54 controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL |
00:12:54 3 tests, 1 passed, 2 failed
00:12:54 ==============================================================================
00:12:54 controller-clustering-ask.txt.Partition And Heal :: Cluster Singleton testi...
00:12:54 ==============================================================================
00:12:58 Register_Singleton_Constant_On_Each_Node :: Register a candidate a... | FAIL |
00:12:58 ... click for list of related bugs or create a new one if needed (with the
00:12:58 "controller_clustering_ask_txt_partition_and_heal_register_singleton_constant_on_each_node"
00:12:58 reference somewhere inside)
00:12:58 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_partition_and_heal_register_singleton_constant_on_each_node&order=bug_status"
00:12:58
00:12:58 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/operations/odl-mdsal-lowlevel-control:register-singleton-constant
00:12:58 ------------------------------------------------------------------------------
00:12:58 Verify_Singleton_Constant_On_Each_Node :: Store the owner and cand... | FAIL |
00:12:58 ... click for list of related bugs or create a new one if needed (with the
00:12:58 "controller_clustering_ask_txt_partition_and_heal_verify_singleton_constant_on_each_node"
00:12:58 reference somewhere inside)
00:12:58 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_partition_and_heal_verify_singleton_constant_on_each_node&order=bug_status"
00:12:58
00:12:58 Could not parse owner and candidates for device get-singleton-constant-service']
00:12:58 ------------------------------------------------------------------------------
00:12:58 Isolate_Owner_Node :: Isolate the cluster node which is the owner.... | FAIL |
00:12:59 ... click for list of related bugs or create a new one if needed (with the
00:12:59 "controller_clustering_ask_txt_partition_and_heal_isolate_owner_node"
00:12:59 reference somewhere inside)
00:12:59 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_partition_and_heal_isolate_owner_node&order=bug_status"
00:12:59
00:12:59 Variable '${cs_owner}' not found.
00:12:59 ------------------------------------------------------------------------------
00:12:59 Monitor_Stability_While_Isolated :: Monitor the stability of the s... | FAIL |
00:12:59 ... click for list of related bugs or create a new one if needed (with the
00:12:59 "controller_clustering_ask_txt_partition_and_heal_monitor_stability_while_isolated"
00:12:59 reference somewhere inside)
00:12:59 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_partition_and_heal_monitor_stability_while_isolated&order=bug_status"
00:12:59
00:12:59 Invalid IF condition: Evaluating expression '"${index}" == "${cs_isolated_index}"' failed: Variable '${cs_isolated_index}' not found.
00:12:59 ------------------------------------------------------------------------------
00:12:59 Rejoin_Isolated_node :: Rejoin isolated node. | FAIL |
00:12:59 ... click for list of related bugs or create a new one if needed (with the
00:12:59 "controller_clustering_ask_txt_partition_and_heal_rejoin_isolated_node"
00:12:59 reference somewhere inside)
00:12:59 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_partition_and_heal_rejoin_isolated_node&order=bug_status"
00:12:59
00:12:59 Variable '${cs_isolated_index}' not found.
00:12:59 ------------------------------------------------------------------------------
00:12:59 Unregister_Singleton_Constant_On_Each_Node :: Unregister the appli... | FAIL |
00:13:00 ... click for list of related bugs or create a new one if needed (with the
00:13:00 "controller_clustering_ask_txt_partition_and_heal_unregister_singleton_constant_on_each_node"
00:13:00 reference somewhere inside)
00:13:00 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_partition_and_heal_unregister_singleton_constant_on_each_node&order=bug_status"
00:13:00
00:13:00 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/operations/odl-mdsal-lowlevel-control:unregister-singleton-constant
00:13:00 ------------------------------------------------------------------------------
00:13:00 controller-clustering-ask.txt.Partition And Heal :: Cluster Single... | FAIL |
00:13:00 6 tests, 0 passed, 6 failed
00:13:00 ==============================================================================
00:13:00 controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te...
00:13:00 ==============================================================================
00:13:02 Stop_All_Members :: Stop every odl node. | FAIL |
00:19:06 Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
00:19:06 ------------------------------------------------------------------------------
00:19:06 Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS |
00:19:17 ------------------------------------------------------------------------------
00:19:17 Start_All_And_Sync :: Start each member and wait for sync. | FAIL |
00:25:22 Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore
00:25:22 ------------------------------------------------------------------------------
00:25:22 controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL |
00:25:22 3 tests, 1 passed, 2 failed
00:25:22 ==============================================================================
00:25:22 controller-clustering-ask.txt.Chasing The Leader :: Cluster Singleton testi...
00:25:22 ==============================================================================
00:25:26 Register_Candidates :: Register a candidate application on each no... | FAIL |
00:25:26 ... click for list of related bugs or create a new one if needed (with the
00:25:26 "controller_clustering_ask_txt_chasing_the_leader_register_candidates"
00:25:26 reference somewhere inside)
00:25:26 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_chasing_the_leader_register_candidates&order=bug_status"
00:25:26
00:25:26 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/operations/odl-mdsal-lowlevel-control:register-flapping-singleton
00:25:26 ------------------------------------------------------------------------------
00:25:26 Do_Nothing :: Do nothing for the time of the test duration, becaus... | PASS |
00:26:27 ------------------------------------------------------------------------------
00:26:27 Unregister_Candidates_And_Validate_Criteria :: Unregister the test... | FAIL |
00:26:27 ... click for list of related bugs or create a new one if needed (with the
00:26:27 "controller_clustering_ask_txt_chasing_the_leader_unregister_candidates_and_validate_criteria"
00:26:27 reference somewhere inside)
00:26:27 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_chasing_the_leader_unregister_candidates_and_validate_criteria&order=bug_status"
00:26:27
00:26:27 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/operations/odl-mdsal-lowlevel-control:unregister-flapping-singleton
00:26:27 ------------------------------------------------------------------------------
00:26:27 controller-clustering-ask.txt.Chasing The Leader :: Cluster Single... | FAIL |
00:26:27 3 tests, 1 passed, 2 failed
00:26:27 ==============================================================================
00:26:27 controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te...
00:26:27 ==============================================================================
00:26:30 Stop_All_Members :: Stop every odl node. | FAIL |
00:32:35 Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
00:32:35 ------------------------------------------------------------------------------
00:32:35 Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS |
00:32:45 ------------------------------------------------------------------------------
00:32:45 Start_All_And_Sync :: Start each member and wait for sync. | FAIL |
00:38:50 Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore
00:38:50 ------------------------------------------------------------------------------
00:38:50 controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL |
00:38:50 3 tests, 1 passed, 2 failed
00:38:50 ==============================================================================
00:38:50 controller-clustering-ask.txt.Global Rpc Kill :: Controller functional HA t...
00:38:50 ==============================================================================
00:38:55 Get_Basic_Rpc_Test_Owner :: Find a service owner and successors. | FAIL |
00:38:56 ... click for list of related bugs or create a new one if needed (with the
00:38:56 "controller_clustering_ask_txt_global_rpc_kill_get_basic_rpc_test_owner"
00:38:56 reference somewhere inside)
00:38:56 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_kill_get_basic_rpc_test_owner&order=bug_status"
00:38:56
00:38:56 Could not parse owner and candidates for device Basic-rpc-test']
00:38:56 ------------------------------------------------------------------------------
00:38:56 Rpc_Before_Stopping_On_Owner :: Run rpc on the service owner. | FAIL |
00:38:56 ... click for list of related bugs or create a new one if needed (with the
00:38:56 "controller_clustering_ask_txt_global_rpc_kill_rpc_before_stopping_on_owner"
00:38:56 reference somewhere inside)
00:38:56 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_kill_rpc_before_stopping_on_owner&order=bug_status"
00:38:56
00:38:56 Variable '${brt_owner}' not found.
00:38:56 ------------------------------------------------------------------------------
00:38:56 Rpc_Before_Stop_On_Successors :: Run rpc on non owher cluster nodes. | FAIL |
00:38:56 ... click for list of related bugs or create a new one if needed (with the
00:38:56 "controller_clustering_ask_txt_global_rpc_kill_rpc_before_stop_on_successors"
00:38:56 reference somewhere inside)
00:38:56 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_kill_rpc_before_stop_on_successors&order=bug_status"
00:38:56
00:38:56 Variable '@{brt_successors}' not found.
00:38:56 ------------------------------------------------------------------------------
00:38:56 Stop_Current_Owner_Member :: Stop cluster node which is the owner. | FAIL |
00:38:56 ... click for list of related bugs or create a new one if needed (with the
00:38:56 "controller_clustering_ask_txt_global_rpc_kill_stop_current_owner_member"
00:38:56 reference somewhere inside)
00:38:56 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_kill_stop_current_owner_member&order=bug_status"
00:38:56
00:38:56 Variable '${brt_owner}' not found.
00:38:56 ------------------------------------------------------------------------------
00:38:56 Verify_New_Basic_Rpc_Test_Owner_Elected :: Verify new owner of the... | FAIL |
00:38:57 ... click for list of related bugs or create a new one if needed (with the
00:38:57 "controller_clustering_ask_txt_global_rpc_kill_verify_new_basic_rpc_test_owner_elected"
00:38:57 reference somewhere inside)
00:38:57 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_kill_verify_new_basic_rpc_test_owner_elected&order=bug_status"
00:38:57
00:38:57 Variable '${old_brt_successors}' not found.
00:38:57 ------------------------------------------------------------------------------
00:38:57 Rpc_On_Remained_Cluster_Nodes :: Run rpc on remained cluster nodes. | FAIL |
00:38:57 ... click for list of related bugs or create a new one if needed (with the
00:38:57 "controller_clustering_ask_txt_global_rpc_kill_rpc_on_remained_cluster_nodes"
00:38:57 reference somewhere inside)
00:38:57 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_kill_rpc_on_remained_cluster_nodes&order=bug_status"
00:38:57
00:38:57 Variable '@{old_brt_successors}' not found.
00:38:57 ------------------------------------------------------------------------------
00:38:57 Restart_Stopped_Member :: Restart stopped node | FAIL |
00:38:57 ... click for list of related bugs or create a new one if needed (with the
00:38:57 "controller_clustering_ask_txt_global_rpc_kill_restart_stopped_member"
00:38:57 reference somewhere inside)
00:38:57 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_kill_restart_stopped_member&order=bug_status"
00:38:57
00:38:57 Variable '${old_brt_owner}' not found.
00:38:57 ------------------------------------------------------------------------------
00:38:57 Verify_New_Owner_Remained_After_Rejoin :: Verify no owner change h... | FAIL |
00:38:57 ... click for list of related bugs or create a new one if needed (with the
00:38:57 "controller_clustering_ask_txt_global_rpc_kill_verify_new_owner_remained_after_rejoin"
00:38:57 reference somewhere inside)
00:38:57 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_kill_verify_new_owner_remained_after_rejoin&order=bug_status"
00:38:57
00:38:57 Variable '${brt_owner}' not found.
00:38:57 ------------------------------------------------------------------------------
00:38:57 Rpc_After_Rejoin_On_New_Owner :: Run rpc on the new service owner ... | FAIL |
00:38:58 ... click for list of related bugs or create a new one if needed (with the
00:38:58 "controller_clustering_ask_txt_global_rpc_kill_rpc_after_rejoin_on_new_owner"
00:38:58 reference somewhere inside)
00:38:58 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_kill_rpc_after_rejoin_on_new_owner&order=bug_status"
00:38:58
00:38:58 Variable '${brt_owner}' not found.
00:38:58 ------------------------------------------------------------------------------
00:38:58 Rpc_After_Rejoin_On_Old_Owner :: Run rpc on rejoined cluster node. | FAIL |
00:38:58 ... click for list of related bugs or create a new one if needed (with the
00:38:58 "controller_clustering_ask_txt_global_rpc_kill_rpc_after_rejoin_on_old_owner"
00:38:58 reference somewhere inside)
00:38:58 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_kill_rpc_after_rejoin_on_old_owner&order=bug_status"
00:38:58
00:38:58 Variable '${old_brt_owner}' not found.
00:38:58 ------------------------------------------------------------------------------
00:38:58 Rpc_After_Rejoin_On_All :: Run rpc again on all nodes. | FAIL |
00:38:58 ... click for list of related bugs or create a new one if needed (with the
00:38:58 "controller_clustering_ask_txt_global_rpc_kill_rpc_after_rejoin_on_all"
00:38:58 reference somewhere inside)
00:38:58 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_kill_rpc_after_rejoin_on_all&order=bug_status"
00:38:58
00:38:58 Variable '${brt_owner}' not found.
00:38:58 ------------------------------------------------------------------------------
00:38:58 controller-clustering-ask.txt.Global Rpc Kill :: Controller functi... | FAIL |
00:38:58 11 tests, 0 passed, 11 failed
00:38:58 ==============================================================================
00:38:58 controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te...
00:38:58 ==============================================================================
00:39:00 Stop_All_Members :: Stop every odl node. | FAIL |
00:45:05 Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
00:45:05 ------------------------------------------------------------------------------
00:45:05 Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS |
00:45:16 ------------------------------------------------------------------------------
00:45:16 Start_All_And_Sync :: Start each member and wait for sync. | FAIL |
00:51:21 Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore
00:51:21 ------------------------------------------------------------------------------
00:51:21 controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL |
00:51:21 3 tests, 1 passed, 2 failed
00:51:21 ==============================================================================
00:51:21 controller-clustering-ask.txt.Global Rpc Freeze :: Controller functional HA...
00:51:21 ==============================================================================
00:51:25 Get_Basic_Rpc_Test_Owner :: Find a service owner and successors. | FAIL |
00:51:25 ... click for list of related bugs or create a new one if needed (with the
00:51:25 "controller_clustering_ask_txt_global_rpc_freeze_get_basic_rpc_test_owner"
00:51:25 reference somewhere inside)
00:51:25 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_freeze_get_basic_rpc_test_owner&order=bug_status"
00:51:25
00:51:25 Could not parse owner and candidates for device Basic-rpc-test']
00:51:25 ------------------------------------------------------------------------------
00:51:25 Rpc_Before_Freezing_On_Owner :: Run rpc on the service owner. | FAIL |
00:51:26 ... click for list of related bugs or create a new one if needed (with the
00:51:26 "controller_clustering_ask_txt_global_rpc_freeze_rpc_before_freezing_on_owner"
00:51:26 reference somewhere inside)
00:51:26 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_freeze_rpc_before_freezing_on_owner&order=bug_status"
00:51:26
00:51:26 Variable '${brt_owner}' not found.
00:51:26 ------------------------------------------------------------------------------
00:51:26 Rpc_Before_Freeze_On_Successors :: Run rpc on non owher cluster no... | FAIL |
00:51:26 ... click for list of related bugs or create a new one if needed (with the
00:51:26 "controller_clustering_ask_txt_global_rpc_freeze_rpc_before_freeze_on_successors"
00:51:26 reference somewhere inside)
00:51:26 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_freeze_rpc_before_freeze_on_successors&order=bug_status"
00:51:26
00:51:26 Variable '@{brt_successors}' not found.
00:51:26 ------------------------------------------------------------------------------
00:51:26 Freeze_Current_Owner_Member :: Stop cluster node which is the owner. | FAIL |
00:51:26 ... click for list of related bugs or create a new one if needed (with the
00:51:26 "controller_clustering_ask_txt_global_rpc_freeze_freeze_current_owner_member"
00:51:26 reference somewhere inside)
00:51:26 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_freeze_freeze_current_owner_member&order=bug_status"
00:51:26
00:51:26 Variable '${brt_owner}' not found.
00:51:26 ------------------------------------------------------------------------------
00:51:26 Verify_New_Basic_Rpc_Test_Owner_Elected :: Verify new owner of the... | FAIL |
00:51:26 ... click for list of related bugs or create a new one if needed (with the
00:51:26 "controller_clustering_ask_txt_global_rpc_freeze_verify_new_basic_rpc_test_owner_elected"
00:51:26 reference somewhere inside)
00:51:26 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_freeze_verify_new_basic_rpc_test_owner_elected&order=bug_status"
00:51:26
00:51:26 Variable '${old_brt_successors}' not found.
00:51:26 ------------------------------------------------------------------------------
00:51:26 Rpc_On_Remained_Cluster_Nodes :: Run rpc on remained cluster nodes. | FAIL |
00:51:27 ... click for list of related bugs or create a new one if needed (with the
00:51:27 "controller_clustering_ask_txt_global_rpc_freeze_rpc_on_remained_cluster_nodes"
00:51:27 reference somewhere inside)
00:51:27 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_freeze_rpc_on_remained_cluster_nodes&order=bug_status"
00:51:27
00:51:27 Variable '@{old_brt_successors}' not found.
00:51:27 ------------------------------------------------------------------------------
00:51:27 Unfreeze_Frozen_Member :: Restart frozen node | FAIL |
00:51:27 ... click for list of related bugs or create a new one if needed (with the
00:51:27 "controller_clustering_ask_txt_global_rpc_freeze_unfreeze_frozen_member"
00:51:27 reference somewhere inside)
00:51:27 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_freeze_unfreeze_frozen_member&order=bug_status"
00:51:27
00:51:27 Variable '${old_brt_owner}' not found.
00:51:27 ------------------------------------------------------------------------------
00:51:27 Rpc_After_Rejoin_On_New_Owner :: Run rpc on the new service owner ... | FAIL |
00:51:27 ... click for list of related bugs or create a new one if needed (with the
00:51:27 "controller_clustering_ask_txt_global_rpc_freeze_rpc_after_rejoin_on_new_owner"
00:51:27 reference somewhere inside)
00:51:27 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_freeze_rpc_after_rejoin_on_new_owner&order=bug_status"
00:51:27
00:51:27 Variable '${brt_owner}' not found.
00:51:27 ------------------------------------------------------------------------------
00:51:27 Rpc_After_Rejoin_On_Old_Owner :: Run rpc on rejoined cluster node. | FAIL |
00:51:28 ... click for list of related bugs or create a new one if needed (with the
00:51:28 "controller_clustering_ask_txt_global_rpc_freeze_rpc_after_rejoin_on_old_owner"
00:51:28 reference somewhere inside)
00:51:28 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_freeze_rpc_after_rejoin_on_old_owner&order=bug_status"
00:51:28
00:51:28 Variable '${old_brt_owner}' not found.
00:51:28 ------------------------------------------------------------------------------
00:51:28 Rpc_After_Rejoin_On_All :: Run rpc again on all nodes. | FAIL |
00:51:28 ... click for list of related bugs or create a new one if needed (with the
00:51:28 "controller_clustering_ask_txt_global_rpc_freeze_rpc_after_rejoin_on_all"
00:51:28 reference somewhere inside)
00:51:28 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_freeze_rpc_after_rejoin_on_all&order=bug_status"
00:51:28
00:51:28 Variable '${brt_owner}' not found.
00:51:28 ------------------------------------------------------------------------------
00:51:28 controller-clustering-ask.txt.Global Rpc Freeze :: Controller func... | FAIL |
00:51:28 10 tests, 0 passed, 10 failed
00:51:28 ==============================================================================
00:51:28 controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te...
00:51:28 ==============================================================================
00:51:30 Stop_All_Members :: Stop every odl node. | FAIL |
00:57:32 Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
00:57:32 ------------------------------------------------------------------------------
00:57:32 Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS |
00:57:43 ------------------------------------------------------------------------------
00:57:43 Start_All_And_Sync :: Start each member and wait for sync. | FAIL |
01:03:52 Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore
01:03:52 ------------------------------------------------------------------------------
01:03:52 controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL |
01:03:52 3 tests, 1 passed, 2 failed
01:03:52 ==============================================================================
01:03:52 controller-clustering-ask.txt.Global Rpc Isolate :: Controller functional H...
01:03:52 ==============================================================================
01:03:59 Get_Basic_Rpc_Test_Owner :: Find a service owner and successors. | FAIL |
01:03:59 ... click for list of related bugs or create a new one if needed (with the
01:03:59 "controller_clustering_ask_txt_global_rpc_isolate_get_basic_rpc_test_owner"
01:03:59 reference somewhere inside)
01:03:59 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_isolate_get_basic_rpc_test_owner&order=bug_status"
01:03:59
01:03:59 Could not parse owner and candidates for device Basic-rpc-test']
01:03:59 ------------------------------------------------------------------------------
01:03:59 Rpc_Before_Isolation_On_Owner :: Run rpc on the service owner. | FAIL |
01:04:00 ... click for list of related bugs or create a new one if needed (with the
01:04:00 "controller_clustering_ask_txt_global_rpc_isolate_rpc_before_isolation_on_owner"
01:04:00 reference somewhere inside)
01:04:00 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_isolate_rpc_before_isolation_on_owner&order=bug_status"
01:04:00
01:04:00 Variable '${brt_owner}' not found.
01:04:00 ------------------------------------------------------------------------------
01:04:00 Rpc_Before_Isolation_On_Successors :: Run rpc on non owher cluster... | FAIL |
01:04:00 ... click for list of related bugs or create a new one if needed (with the
01:04:00 "controller_clustering_ask_txt_global_rpc_isolate_rpc_before_isolation_on_successors"
01:04:00 reference somewhere inside)
01:04:00 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_isolate_rpc_before_isolation_on_successors&order=bug_status"
01:04:00
01:04:00 Variable '@{brt_successors}' not found.
01:04:00 ------------------------------------------------------------------------------
01:04:00 Isolate_Current_Owner_Member :: Isolating cluster node which is th... | FAIL |
01:04:00 ... click for list of related bugs or create a new one if needed (with the
01:04:00 "controller_clustering_ask_txt_global_rpc_isolate_isolate_current_owner_member"
01:04:00 reference somewhere inside)
01:04:00 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_isolate_isolate_current_owner_member&order=bug_status"
01:04:00
01:04:00 Variable '${brt_owner}' not found.
01:04:00 ------------------------------------------------------------------------------
01:04:00 Verify_New_Basic_Rpc_Test_Owner_Elected :: Verify new owner of the... | FAIL |
01:04:01 ... click for list of related bugs or create a new one if needed (with the
01:04:01 "controller_clustering_ask_txt_global_rpc_isolate_verify_new_basic_rpc_test_owner_elected"
01:04:01 reference somewhere inside)
01:04:01 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_isolate_verify_new_basic_rpc_test_owner_elected&order=bug_status"
01:04:01
01:04:01 Variable '${old_brt_successors}' not found.
01:04:01 ------------------------------------------------------------------------------
01:04:01 Rpc_On_Isolated_Node :: Run rpc on isolated cluster node. | FAIL |
01:04:01 ... click for list of related bugs or create a new one if needed (with the
01:04:01 "controller_clustering_ask_txt_global_rpc_isolate_rpc_on_isolated_node"
01:04:01 reference somewhere inside)
01:04:01 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_isolate_rpc_on_isolated_node&order=bug_status"
01:04:01
01:04:01 Variable '${old_brt_owner}' not found.
01:04:01 ------------------------------------------------------------------------------
01:04:01 Rpc_On_Non_Isolated_Cluster_Nodes :: Run rpc on remained cluster n... | FAIL |
01:04:01 ... click for list of related bugs or create a new one if needed (with the
01:04:01 "controller_clustering_ask_txt_global_rpc_isolate_rpc_on_non_isolated_cluster_nodes"
01:04:01 reference somewhere inside)
01:04:01 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_isolate_rpc_on_non_isolated_cluster_nodes&order=bug_status"
01:04:01
01:04:01 Variable '@{old_brt_successors}' not found.
01:04:01 ------------------------------------------------------------------------------
01:04:01 Rejoin_Isolated_Member :: Rejoin isolated node | FAIL |
01:04:01 ... click for list of related bugs or create a new one if needed (with the
01:04:01 "controller_clustering_ask_txt_global_rpc_isolate_rejoin_isolated_member"
01:04:01 reference somewhere inside)
01:04:01 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_isolate_rejoin_isolated_member&order=bug_status"
01:04:01
01:04:01 Variable '${old_brt_owner}' not found.
01:04:01 ------------------------------------------------------------------------------
01:04:01 Rpc_After_Rejoin_On_New_Owner :: Run rpc on the new service owner ... | FAIL |
01:04:02 ... click for list of related bugs or create a new one if needed (with the
01:04:02 "controller_clustering_ask_txt_global_rpc_isolate_rpc_after_rejoin_on_new_owner"
01:04:02 reference somewhere inside)
01:04:02 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_isolate_rpc_after_rejoin_on_new_owner&order=bug_status"
01:04:02
01:04:02 Variable '${brt_owner}' not found.
01:04:02 ------------------------------------------------------------------------------
01:04:02 Rpc_After_Rejoin_On_Old_Owner :: Run rpc on rejoined cluster node. | FAIL |
01:04:02 ... click for list of related bugs or create a new one if needed (with the
01:04:02 "controller_clustering_ask_txt_global_rpc_isolate_rpc_after_rejoin_on_old_owner"
01:04:02 reference somewhere inside)
01:04:02 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_isolate_rpc_after_rejoin_on_old_owner&order=bug_status"
01:04:02
01:04:02 Variable '${old_brt_owner}' not found.
01:04:02 ------------------------------------------------------------------------------
01:04:02 Rpc_After_Rejoin_On_All :: Run rpc again on all nodes. | FAIL |
01:04:02 ... click for list of related bugs or create a new one if needed (with the
01:04:02 "controller_clustering_ask_txt_global_rpc_isolate_rpc_after_rejoin_on_all"
01:04:02 reference somewhere inside)
01:04:02 "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_global_rpc_isolate_rpc_after_rejoin_on_all&order=bug_status"
01:04:02
01:04:02 Variable '${brt_owner}' not found.
01:04:02 ------------------------------------------------------------------------------
01:04:02 controller-clustering-ask.txt.Global Rpc Isolate :: Controller fun... | FAIL |
01:04:02 11 tests, 0 passed, 11 failed
01:04:02 ==============================================================================
01:04:02 controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te...
01:04:02 ==============================================================================
01:04:04 Stop_All_Members :: Stop every odl node. | FAIL |
01:10:07 Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
01:10:07 ------------------------------------------------------------------------------
01:10:07 Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS |
01:10:18 ------------------------------------------------------------------------------
01:10:18 Start_All_And_Sync :: Start each member and wait for sync. | FAIL |
01:16:23 Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore
01:16:23 ------------------------------------------------------------------------------
01:16:23 controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL |
01:16:23 3 tests, 1 passed, 2 failed
01:16:23 ==============================================================================
01:16:23 controller-clustering-ask.txt.Carpeople Crud :: Suite for performing basic ...
01:16:23 ==============================================================================
01:16:27 Add_Cars_To_Leader :: Add 30 cars to car Leader by one big PUT. | FAIL |
01:16:27 Parent suite setup failed:
01:16:27 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:27 ------------------------------------------------------------------------------
01:16:27 See_Added_Cars_On_Leader :: GET response from Leader should match ... | FAIL |
01:16:27 Parent suite setup failed:
01:16:27 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:27 ------------------------------------------------------------------------------
01:16:27 See_Added_Cars_On_Followers :: The same check on other members. | FAIL |
01:16:27 Parent suite setup failed:
01:16:27 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:27 ------------------------------------------------------------------------------
01:16:27 Add_People_To_First_Follower :: Add 30 people to people first Foll... | FAIL |
01:16:27 Parent suite setup failed:
01:16:27 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:27 ------------------------------------------------------------------------------
01:16:27 See_Added_People_On_Leader :: GET response from Leader should matc... | FAIL |
01:16:27 Parent suite setup failed:
01:16:27 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:27 ------------------------------------------------------------------------------
01:16:27 See_Added_People_On_Followers :: The same check on other members. | FAIL |
01:16:27 Parent suite setup failed:
01:16:27 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:27 ------------------------------------------------------------------------------
01:16:27 Buy_Cars_On_Leader :: Buy some cars on car-people Leader, loop of ... | FAIL |
01:16:27 Parent suite setup failed:
01:16:27 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:27 ------------------------------------------------------------------------------
01:16:27 Buy_Cars_On_Followers :: On each Follower buy corresponding ID seg... | FAIL |
01:16:27 Parent suite setup failed:
01:16:27 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:27 ------------------------------------------------------------------------------
01:16:27 See_Added_CarPeople_On_Leader :: GET car-person mappings from Lead... | FAIL |
01:16:27 Parent suite setup failed:
01:16:27 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:27 ------------------------------------------------------------------------------
01:16:27 See_Added_CarPeople_On_Followers :: The same check on other members. | FAIL |
01:16:27 Parent suite setup failed:
01:16:27 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:27 ------------------------------------------------------------------------------
01:16:27 Delete_All_CarPeople_On_Leader :: DELETE car-people container. No ... | FAIL |
01:16:27 Parent suite setup failed:
01:16:27 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:27 ------------------------------------------------------------------------------
01:16:27 Delete_All_People_On_Leader :: DELETE people container. No verific... | FAIL |
01:16:27 Parent suite setup failed:
01:16:27 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:27 ------------------------------------------------------------------------------
01:16:27 Delete_All_Cars_On_Leader :: DELETE cars container. No verificatio... | FAIL |
01:16:27 Parent suite setup failed:
01:16:27 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:27 ------------------------------------------------------------------------------
01:16:27 controller-clustering-ask.txt.Carpeople Crud :: Suite for performi... | FAIL |
01:16:27 Suite setup failed:
01:16:27 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:27
01:16:27 13 tests, 0 passed, 13 failed
01:16:27 ==============================================================================
01:16:27 controller-clustering-ask.txt.Car Failover Crud :: Suite mixing basic opera...
01:16:27 ==============================================================================
01:16:30 Add_Original_Cars_On_Old_Leader_And_Verify :: Add initial cars on ... | FAIL |
01:16:30 Parent suite setup failed:
01:16:30 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:30 ------------------------------------------------------------------------------
01:16:30 Stop_Original_Car_Leader :: Stop the car Leader to cause a new lea... | FAIL |
01:16:30 Parent suite setup failed:
01:16:30 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:30 ------------------------------------------------------------------------------
01:16:30 Wait_For_New_Leader :: Wait until new car Leader is elected. | FAIL |
01:16:30 Parent suite setup failed:
01:16:30 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:30 ------------------------------------------------------------------------------
01:16:30 See_Original_Cars_On_New_Leader :: GET cars from new Leader, shoul... | FAIL |
01:16:30 Parent suite setup failed:
01:16:30 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:30 ------------------------------------------------------------------------------
01:16:30 See_Original_Cars_On_New_Followers :: The same check on other exis... | FAIL |
01:16:30 Parent suite setup failed:
01:16:30 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:30 ------------------------------------------------------------------------------
01:16:30 Delete_Original_Cars_On_New_Leader :: Delete cars on the new Leader. | FAIL |
01:16:30 Parent suite setup failed:
01:16:30 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:30 ------------------------------------------------------------------------------
01:16:30 Add_Leader_Cars_On_New_Leader :: Add cars on the new Leader. | FAIL |
01:16:30 Parent suite setup failed:
01:16:30 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:30 ------------------------------------------------------------------------------
01:16:30 See_Leader_Cars_On_New_Leader :: GET cars from new Leader, should ... | FAIL |
01:16:30 Parent suite setup failed:
01:16:30 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:30 ------------------------------------------------------------------------------
01:16:30 See_Leader_Cars_On_New_Followers :: The same check on other existi... | FAIL |
01:16:30 Parent suite setup failed:
01:16:30 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:30 ------------------------------------------------------------------------------
01:16:30 Delete_Leader_Cars_On_New_First_Follower :: Delete cars in new fir... | FAIL |
01:16:30 Parent suite setup failed:
01:16:30 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:30 ------------------------------------------------------------------------------
01:16:30 Add_Follower_Cars_On_New_First_Follower :: Add cars on the new fir... | FAIL |
01:16:30 Parent suite setup failed:
01:16:30 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:30 ------------------------------------------------------------------------------
01:16:30 See_Folower_Cars_On_New_Leader :: Get cars from the new Leader, sh... | FAIL |
01:16:30 Parent suite setup failed:
01:16:30 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:30 ------------------------------------------------------------------------------
01:16:30 See_Follower_Cars_On_New_Followers :: The same check on other exis... | FAIL |
01:16:30 Parent suite setup failed:
01:16:30 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:30 ------------------------------------------------------------------------------
01:16:30 Start_Old_Car_Leader :: Start the stopped member without deleting ... | FAIL |
01:16:30 Parent suite setup failed:
01:16:30 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:30 ------------------------------------------------------------------------------
01:16:30 See_Folower_Cars_On_Old_Leader :: GET cars from the restarted memb... | FAIL |
01:16:30 Parent suite setup failed:
01:16:30 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:30 ------------------------------------------------------------------------------
01:16:30 Delete_Follower_Cars_On_New_Leader :: Delete cars on the last Leader. | FAIL |
01:16:30 Parent suite setup failed:
01:16:30 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:30 ------------------------------------------------------------------------------
01:16:30 controller-clustering-ask.txt.Car Failover Crud :: Suite mixing ba... | FAIL |
01:16:30 Suite setup failed:
01:16:30 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:30
01:16:30 16 tests, 0 passed, 16 failed
01:16:30 ==============================================================================
01:16:30 controller-clustering-ask.txt.Car Persistence Recovery :: This test restart...
01:16:30 ==============================================================================
01:16:32 Add_Cars_On_Leader_And_Verify :: Single big PUT to datastore to ad... | FAIL |
01:16:32 Parent suite setup failed:
01:16:32 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:32 ------------------------------------------------------------------------------
01:16:32 Stop_All_Members :: Stop all controllers. | FAIL |
01:16:32 Parent suite setup failed:
01:16:32 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:32 ------------------------------------------------------------------------------
01:16:32 Start_All_Members :: Start all controllers (should restore the per... | FAIL |
01:16:32 Parent suite setup failed:
01:16:32 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:32 ------------------------------------------------------------------------------
01:16:32 Memorize_Leader_And_Followers :: Locate current Leader of car Shard. | FAIL |
01:16:32 Parent suite setup failed:
01:16:32 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:32 ------------------------------------------------------------------------------
01:16:32 See_Cars_On_Leader :: GET cars from Leader, should match the PUT d... | FAIL |
01:16:32 Parent suite setup failed:
01:16:32 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:32 ------------------------------------------------------------------------------
01:16:32 See_Cars_On_Followers :: The same check on other members. | FAIL |
01:16:32 Parent suite setup failed:
01:16:32 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:32 ------------------------------------------------------------------------------
01:16:32 Delete_Cars_On_Leader :: Delete cars on the new Leader. | FAIL |
01:16:32 Parent suite setup failed:
01:16:32 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:32 ------------------------------------------------------------------------------
01:16:32 controller-clustering-ask.txt.Car Persistence Recovery :: This tes... | FAIL |
01:16:32 Suite setup failed:
01:16:32 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:32
01:16:32 7 tests, 0 passed, 7 failed
01:16:32 ==============================================================================
01:16:33 controller-clustering-ask.txt.Buycar Failover :: This test focuses on testi...
01:16:33 ==============================================================================
01:16:35 Add_Cars_To_Leader_And_Verify :: Add all needed cars to car Leader... | FAIL |
01:16:35 Parent suite setup failed:
01:16:35 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:35 ------------------------------------------------------------------------------
01:16:35 Add_People_To_First_Follower_And_Verify :: Add all needed people t... | FAIL |
01:16:35 Parent suite setup failed:
01:16:35 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:35 ------------------------------------------------------------------------------
01:16:35 Buy_Cars_On_Leader_And_Verify :: Buy some cars on the leader member. | FAIL |
01:16:35 Parent suite setup failed:
01:16:35 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:35 ------------------------------------------------------------------------------
01:16:35 Buy_Cars_On_Follower_And_Verify :: Buy some cars on the first foll... | FAIL |
01:16:35 Parent suite setup failed:
01:16:35 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:35 ------------------------------------------------------------------------------
01:16:35 Reboot_People_Leader :: Previous people Leader is rebooted. We sho... | FAIL |
01:16:35 Parent suite setup failed:
01:16:35 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:35 ------------------------------------------------------------------------------
01:16:35 Buy_Cars_On_Leader_After_Reboot_And_Verify :: Buy some cars on the... | FAIL |
01:16:35 Parent suite setup failed:
01:16:35 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:35 ------------------------------------------------------------------------------
01:16:35 Buy_Cars_On_Follower_After_Reboot_And_Verify :: Buy some cars on t... | FAIL |
01:16:35 Parent suite setup failed:
01:16:35 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:35 ------------------------------------------------------------------------------
01:16:35 Delete_All_CarPeople :: DELETE car-people container. No verificati... | FAIL |
01:16:35 Parent suite setup failed:
01:16:35 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:35 ------------------------------------------------------------------------------
01:16:35 Delete_All_People :: DELETE people container. No verification beyo... | FAIL |
01:16:35 Parent suite setup failed:
01:16:35 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:35 ------------------------------------------------------------------------------
01:16:35 Delete_All_Cars :: DELETE cars container. No verification beyond h... | FAIL |
01:16:35 Parent suite setup failed:
01:16:35 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:35 ------------------------------------------------------------------------------
01:16:35 controller-clustering-ask.txt.Buycar Failover :: This test focuses... | FAIL |
01:16:35 Suite setup failed:
01:16:35 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:35
01:16:35 10 tests, 0 passed, 10 failed
01:16:35 ==============================================================================
01:16:35 controller-clustering-ask.txt.Entity Isolate :: Suite for performing member...
01:16:35 ==============================================================================
01:16:37 Check All Shards Before Isolate :: Check all shards in controller. | FAIL |
01:16:37 Parent suite setup failed:
01:16:37 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:37 ------------------------------------------------------------------------------
01:16:37 Isolate Entity Leader :: Isolate the entity-ownership Leader to ca... | FAIL |
01:16:37 Parent suite setup failed:
01:16:37 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:37 ------------------------------------------------------------------------------
01:16:37 Check All Shards After Isolate :: Check all shards in controller. | FAIL |
01:16:37 Parent suite setup failed:
01:16:37 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:37 ------------------------------------------------------------------------------
01:16:37 Rejoin Entity Leader :: Rejoin the entity-ownership Leader. | FAIL |
01:16:37 Parent suite setup failed:
01:16:37 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:37 ------------------------------------------------------------------------------
01:16:37 Check All Shards After Rejoin :: Check all shards in controller. | FAIL |
01:16:37 Parent suite setup failed:
01:16:37 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:37 ------------------------------------------------------------------------------
01:16:39 controller-clustering-ask.txt.Entity Isolate :: Suite for performi... | FAIL |
01:16:39 Suite setup failed:
01:16:39 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:39
01:16:39 5 tests, 0 passed, 5 failed
01:16:39 ==============================================================================
01:16:39 controller-clustering-ask.txt.Buycar Failover Isolation :: This test focuse...
01:16:39 ==============================================================================
01:16:41 Add_Cars_To_Leader_And_Verify :: Add all needed cars to car Leader... | FAIL |
01:16:41 Parent suite setup failed:
01:16:41 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:41 ------------------------------------------------------------------------------
01:16:41 Add_People_To_First_Follower_And_Verify :: Add all needed people t... | FAIL |
01:16:41 Parent suite setup failed:
01:16:41 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:41 ------------------------------------------------------------------------------
01:16:41 Buy_Cars_On_Leader_And_Verify :: Buy some cars on the leader member. | FAIL |
01:16:41 Parent suite setup failed:
01:16:41 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:41 ------------------------------------------------------------------------------
01:16:41 Buy_Cars_On_Follower_And_Verify :: Buy some cars on the first foll... | FAIL |
01:16:41 Parent suite setup failed:
01:16:41 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:41 ------------------------------------------------------------------------------
01:16:41 Isolate_and_Rejoin_People_Leader :: Previous people Leader is isol... | FAIL |
01:16:41 Parent suite setup failed:
01:16:41 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:41 ------------------------------------------------------------------------------
01:16:41 Buy_Cars_On_Leader_After_Rejoin_And_Verify :: Buy some cars on the... | FAIL |
01:16:41 Parent suite setup failed:
01:16:41 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:41 ------------------------------------------------------------------------------
01:16:41 Buy_Cars_On_Follower_After_Rejoin_And_Verify :: Buy some cars on t... | FAIL |
01:16:41 Parent suite setup failed:
01:16:41 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:41 ------------------------------------------------------------------------------
01:16:41 Delete_All_CarPeople :: DELETE car-people container. No verificati... | FAIL |
01:16:41 Parent suite setup failed:
01:16:41 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:41 ------------------------------------------------------------------------------
01:16:41 Delete_All_People :: DELETE people container. No verification beyo... | FAIL |
01:16:41 Parent suite setup failed:
01:16:41 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:41 ------------------------------------------------------------------------------
01:16:41 Delete_All_Cars :: DELETE cars container. No verification beyond h... | FAIL |
01:16:41 Parent suite setup failed:
01:16:41 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:41 ------------------------------------------------------------------------------
01:16:41 controller-clustering-ask.txt.Buycar Failover Isolation :: This te... | FAIL |
01:16:41 Suite setup failed:
01:16:41 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:41
01:16:41 10 tests, 0 passed, 10 failed
01:16:41 ==============================================================================
01:16:41 controller-clustering-ask.txt.Car Failover Crud Isolation :: Suite mixing b...
01:16:41 ==============================================================================
01:16:43 Add_Original_Cars_On_Old_Leader_And_Verify :: Add initial cars on ... | FAIL |
01:16:43 Parent suite setup failed:
01:16:43 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:43 ------------------------------------------------------------------------------
01:16:43 Isolate_Original_Car_Leader :: Isolate the car Leader to cause a n... | FAIL |
01:16:43 Parent suite setup failed:
01:16:43 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:43 ------------------------------------------------------------------------------
01:16:43 Wait_For_New_Leader :: Wait until new car Leader is elected. | FAIL |
01:16:43 Parent suite setup failed:
01:16:43 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:43 ------------------------------------------------------------------------------
01:16:43 See_Original_Cars_On_New_Leader :: GET cars from new Leader, shoul... | FAIL |
01:16:43 Parent suite setup failed:
01:16:43 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:43 ------------------------------------------------------------------------------
01:16:43 See_Original_Cars_On_New_Followers :: The same check on other exis... | FAIL |
01:16:43 Parent suite setup failed:
01:16:43 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:43 ------------------------------------------------------------------------------
01:16:43 Delete_Original_Cars_On_New_Leader :: Delete cars on the new Leader. | FAIL |
01:16:43 Parent suite setup failed:
01:16:43 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:43 ------------------------------------------------------------------------------
01:16:43 Add_Leader_Cars_On_New_Leader :: Add cars on the new Leader. | FAIL |
01:16:43 Parent suite setup failed:
01:16:43 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:43 ------------------------------------------------------------------------------
01:16:43 See_Leader_Cars_On_New_Leader :: GET cars from new Leader, should ... | FAIL |
01:16:43 Parent suite setup failed:
01:16:43 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:43 ------------------------------------------------------------------------------
01:16:43 See_Leader_Cars_On_New_Followers :: The same check on other existi... | FAIL |
01:16:43 Parent suite setup failed:
01:16:43 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:43 ------------------------------------------------------------------------------
01:16:43 Delete_Leader_Cars_On_New_First_Follower :: Delete cars in new fir... | FAIL |
01:16:43 Parent suite setup failed:
01:16:43 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:43 ------------------------------------------------------------------------------
01:16:43 Add_Follower_Cars_On_New_First_Follower :: Add cars on the new fir... | FAIL |
01:16:43 Parent suite setup failed:
01:16:43 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:43 ------------------------------------------------------------------------------
01:16:43 See_Folower_Cars_On_New_Leader :: Get cars from the new Leader, sh... | FAIL |
01:16:43 Parent suite setup failed:
01:16:43 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:43 ------------------------------------------------------------------------------
01:16:43 See_Follower_Cars_On_New_Followers :: The same check on other exis... | FAIL |
01:16:43 Parent suite setup failed:
01:16:43 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:43 ------------------------------------------------------------------------------
01:16:43 Rejoin_Old_Car_Leader :: Rejoin the isolated member without deleti... | FAIL |
01:16:43 Parent suite setup failed:
01:16:43 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:43 ------------------------------------------------------------------------------
01:16:43 See_Folower_Cars_On_Old_Leader :: GET cars from the restarted memb... | FAIL |
01:16:43 Parent suite setup failed:
01:16:43 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:43 ------------------------------------------------------------------------------
01:16:43 Delete_Follower_Cars_On_New_Leader :: Delete cars on the last Leader. | FAIL |
01:16:43 Parent suite setup failed:
01:16:43 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:43 ------------------------------------------------------------------------------
01:16:43 controller-clustering-ask.txt.Car Failover Crud Isolation :: Suite... | FAIL |
01:16:43 Suite setup failed:
01:16:43 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:43
01:16:43 16 tests, 0 passed, 16 failed
01:16:43 ==============================================================================
01:16:44 controller-clustering-ask.txt.Car Outage Corners :: Cluster suite for testi...
01:16:44 ==============================================================================
01:16:46 Stop_Majority_Of_The_Followers :: Stop half plus one car Follower ... | FAIL |
01:16:46 Parent suite setup failed:
01:16:46 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:46 ------------------------------------------------------------------------------
01:16:46 Attempt_To_Add_Cars_To_Leader :: Adding cars should fail, as major... | FAIL |
01:16:46 Parent suite setup failed:
01:16:46 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:46 ------------------------------------------------------------------------------
01:16:46 Start_Tipping_Follower :: Start one Follower member without persis... | FAIL |
01:16:46 Parent suite setup failed:
01:16:46 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:46 ------------------------------------------------------------------------------
01:16:46 Add_Cars_On_Tipping_Follower :: Add cars on the tipping Follower. | FAIL |
01:16:46 Parent suite setup failed:
01:16:46 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:46 ------------------------------------------------------------------------------
01:16:46 See_Cars_On_Existing_Members :: On each up member: GET cars, shoul... | FAIL |
01:16:46 Parent suite setup failed:
01:16:46 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:46 ------------------------------------------------------------------------------
01:16:46 Start_Other_Followers :: Start other followers without persisted d... | FAIL |
01:16:46 Parent suite setup failed:
01:16:46 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:46 ------------------------------------------------------------------------------
01:16:46 See_Cars_On_New_Follower_Leader :: GET cars from a new follower to... | FAIL |
01:16:46 Parent suite setup failed:
01:16:46 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:46 ------------------------------------------------------------------------------
01:16:46 Delete_Cars_On_Leader :: Delete cars on Leader. | FAIL |
01:16:46 Parent suite setup failed:
01:16:46 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:46 ------------------------------------------------------------------------------
01:16:46 controller-clustering-ask.txt.Car Outage Corners :: Cluster suite ... | FAIL |
01:16:46 Suite setup failed:
01:16:46 HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.209:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
01:16:46
01:16:46 8 tests, 0 passed, 8 failed
01:16:46 ==============================================================================
01:16:46 controller-clustering-ask.txt | FAIL |
01:16:46 195 tests, 28 passed, 167 failed
01:16:46 ==============================================================================
01:16:46 Output: /w/workspace/controller-csit-3node-clustering-ask-all-titanium/output.xml
01:16:55 Log: /w/workspace/controller-csit-3node-clustering-ask-all-titanium/log.html
01:16:55 Report: /w/workspace/controller-csit-3node-clustering-ask-all-titanium/report.html
01:16:55 + true
01:16:55 + echo 'Examining the files in data/log and checking filesize'
01:16:55 Examining the files in data/log and checking filesize
01:16:55 + ssh 10.30.170.209 'ls -altr /tmp/karaf-0.22.0-SNAPSHOT/data/log/'
01:16:55 Warning: Permanently added '10.30.170.209' (ECDSA) to the list of known hosts.
01:16:55 total 5316
01:16:55 drwxrwxr-x 2 jenkins jenkins 4096 Mar 16 23:05 .
01:16:55 -rw-rw-r-- 1 jenkins jenkins 1720 Mar 16 23:05 karaf_console.log
01:16:55 drwxrwxr-x 4 jenkins jenkins 4096 Mar 17 01:10 ..
01:16:55 -rw-rw-r-- 1 jenkins jenkins 5431019 Mar 17 01:16 karaf.log
01:16:55 + ssh 10.30.170.209 'du -hs /tmp/karaf-0.22.0-SNAPSHOT/data/log/*'
01:16:55 Warning: Permanently added '10.30.170.209' (ECDSA) to the list of known hosts.
01:16:55 5.2M /tmp/karaf-0.22.0-SNAPSHOT/data/log/karaf.log
01:16:55 4.0K /tmp/karaf-0.22.0-SNAPSHOT/data/log/karaf_console.log
01:16:55 + ssh 10.30.170.81 'ls -altr /tmp/karaf-0.22.0-SNAPSHOT/data/log/'
01:16:55 Warning: Permanently added '10.30.170.81' (ECDSA) to the list of known hosts.
01:16:56 total 14120
01:16:56 drwxrwxr-x 2 jenkins jenkins 4096 Mar 16 23:05 .
01:16:56 -rw-rw-r-- 1 jenkins jenkins 1720 Mar 16 23:05 karaf_console.log
01:16:56 drwxrwxr-x 9 jenkins jenkins 4096 Mar 17 01:10 ..
01:16:56 -rw-rw-r-- 1 jenkins jenkins 14441238 Mar 17 01:16 karaf.log
01:16:56 + ssh 10.30.170.81 'du -hs /tmp/karaf-0.22.0-SNAPSHOT/data/log/*'
01:16:56 Warning: Permanently added '10.30.170.81' (ECDSA) to the list of known hosts.
01:16:56 14M /tmp/karaf-0.22.0-SNAPSHOT/data/log/karaf.log
01:16:56 4.0K /tmp/karaf-0.22.0-SNAPSHOT/data/log/karaf_console.log
01:16:56 + ssh 10.30.170.131 'ls -altr /tmp/karaf-0.22.0-SNAPSHOT/data/log/'
01:16:56 Warning: Permanently added '10.30.170.131' (ECDSA) to the list of known hosts.
01:16:56 total 15004
01:16:56 drwxrwxr-x 2 jenkins jenkins 4096 Mar 16 23:05 .
01:16:56 -rw-rw-r-- 1 jenkins jenkins 1720 Mar 16 23:05 karaf_console.log
01:16:56 drwxrwxr-x 9 jenkins jenkins 4096 Mar 17 01:10 ..
01:16:56 -rw-rw-r-- 1 jenkins jenkins 15346164 Mar 17 01:16 karaf.log
01:16:56 + ssh 10.30.170.131 'du -hs /tmp/karaf-0.22.0-SNAPSHOT/data/log/*'
01:16:56 Warning: Permanently added '10.30.170.131' (ECDSA) to the list of known hosts.
01:16:56 15M /tmp/karaf-0.22.0-SNAPSHOT/data/log/karaf.log
01:16:56 4.0K /tmp/karaf-0.22.0-SNAPSHOT/data/log/karaf_console.log
01:16:56 + set +e
01:16:56 ++ seq 1 3
01:16:56 + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
01:16:56 + CONTROLLERIP=ODL_SYSTEM_1_IP
01:16:56 + echo 'Let'\''s take the karaf thread dump again'
01:16:56 Let's take the karaf thread dump again
01:16:56 + ssh 10.30.170.209 'sudo ps aux'
01:16:56 Warning: Permanently added '10.30.170.209' (ECDSA) to the list of known hosts.
01:16:57 ++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-ask-all-titanium/ps_after.log
01:16:57 ++ grep -v grep
01:16:57 ++ tr -s ' '
01:16:57 ++ cut -f2 '-d '
01:16:57 + pid=34745
01:16:57 + echo 'karaf main: org.apache.karaf.main.Main, pid:34745'
01:16:57 karaf main: org.apache.karaf.main.Main, pid:34745
01:16:57 + ssh 10.30.170.209 '/usr/lib/jvm/java-21-openjdk-amd64/bin/jstack -l 34745'
01:16:57 Warning: Permanently added '10.30.170.209' (ECDSA) to the list of known hosts.
01:16:57 + echo 'killing karaf process...'
01:16:57 killing karaf process...
01:16:57 + ssh 10.30.170.209 bash -c 'ps axf | grep karaf | grep -v grep | awk '\''{print "kill -9 " $1}'\'' | sh'
01:16:57 Warning: Permanently added '10.30.170.209' (ECDSA) to the list of known hosts.
01:16:57 + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
01:16:57 + CONTROLLERIP=ODL_SYSTEM_2_IP
01:16:57 + echo 'Let'\''s take the karaf thread dump again'
01:16:57 Let's take the karaf thread dump again
01:16:57 + ssh 10.30.170.81 'sudo ps aux'
01:16:58 Warning: Permanently added '10.30.170.81' (ECDSA) to the list of known hosts.
01:16:58 ++ tr -s ' '
01:16:58 ++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-ask-all-titanium/ps_after.log
01:16:58 ++ grep -v grep
01:16:58 ++ cut -f2 '-d '
01:16:58 + pid=53660
01:16:58 + echo 'karaf main: org.apache.karaf.main.Main, pid:53660'
01:16:58 karaf main: org.apache.karaf.main.Main, pid:53660
01:16:58 + ssh 10.30.170.81 '/usr/lib/jvm/java-21-openjdk-amd64/bin/jstack -l 53660'
01:16:58 Warning: Permanently added '10.30.170.81' (ECDSA) to the list of known hosts.
01:16:58 + echo 'killing karaf process...'
01:16:58 killing karaf process...
01:16:58 + ssh 10.30.170.81 bash -c 'ps axf | grep karaf | grep -v grep | awk '\''{print "kill -9 " $1}'\'' | sh'
01:16:58 Warning: Permanently added '10.30.170.81' (ECDSA) to the list of known hosts.
01:16:59 + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
01:16:59 + CONTROLLERIP=ODL_SYSTEM_3_IP
01:16:59 + echo 'Let'\''s take the karaf thread dump again'
01:16:59 Let's take the karaf thread dump again
01:16:59 + ssh 10.30.170.131 'sudo ps aux'
01:16:59 Warning: Permanently added '10.30.170.131' (ECDSA) to the list of known hosts.
01:16:59 ++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-ask-all-titanium/ps_after.log
01:16:59 ++ grep -v grep
01:16:59 ++ tr -s ' '
01:16:59 ++ cut -f2 '-d '
01:16:59 + pid=69860
01:16:59 + echo 'karaf main: org.apache.karaf.main.Main, pid:69860'
01:16:59 karaf main: org.apache.karaf.main.Main, pid:69860
01:16:59 + ssh 10.30.170.131 '/usr/lib/jvm/java-21-openjdk-amd64/bin/jstack -l 69860'
01:16:59 Warning: Permanently added '10.30.170.131' (ECDSA) to the list of known hosts.
01:16:59 + echo 'killing karaf process...'
01:16:59 killing karaf process...
01:16:59 + ssh 10.30.170.131 bash -c 'ps axf | grep karaf | grep -v grep | awk '\''{print "kill -9 " $1}'\'' | sh'
01:16:59 Warning: Permanently added '10.30.170.131' (ECDSA) to the list of known hosts.
01:17:00 + sleep 5
01:17:05 ++ seq 1 3
01:17:05 + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
01:17:05 + CONTROLLERIP=ODL_SYSTEM_1_IP
01:17:05 + echo 'Compressing karaf.log 1'
01:17:05 Compressing karaf.log 1
01:17:05 + ssh 10.30.170.209 gzip --best /tmp/karaf-0.22.0-SNAPSHOT/data/log/karaf.log
01:17:05 Warning: Permanently added '10.30.170.209' (ECDSA) to the list of known hosts.
01:17:05 + echo 'Fetching compressed karaf.log 1'
01:17:05 Fetching compressed karaf.log 1
01:17:05 + scp 10.30.170.209:/tmp/karaf-0.22.0-SNAPSHOT/data/log/karaf.log.gz odl1_karaf.log.gz
01:17:05 Warning: Permanently added '10.30.170.209' (ECDSA) to the list of known hosts.
01:17:05 + ssh 10.30.170.209 rm -f /tmp/karaf-0.22.0-SNAPSHOT/data/log/karaf.log.gz
01:17:05 Warning: Permanently added '10.30.170.209' (ECDSA) to the list of known hosts.
01:17:05 + scp 10.30.170.209:/tmp/karaf-0.22.0-SNAPSHOT/data/log/karaf_console.log odl1_karaf_console.log
01:17:06 Warning: Permanently added '10.30.170.209' (ECDSA) to the list of known hosts.
01:17:06 + ssh 10.30.170.209 rm -f /tmp/karaf-0.22.0-SNAPSHOT/data/log/karaf_console.log
01:17:06 Warning: Permanently added '10.30.170.209' (ECDSA) to the list of known hosts.
01:17:06 + echo 'Fetch GC logs'
01:17:06 Fetch GC logs
01:17:06 + mkdir -p gclogs-1
01:17:06 + scp '10.30.170.209:/tmp/karaf-0.22.0-SNAPSHOT/data/log/*.log' gclogs-1/
01:17:06 Warning: Permanently added '10.30.170.209' (ECDSA) to the list of known hosts.
01:17:06 scp: /tmp/karaf-0.22.0-SNAPSHOT/data/log/*.log: No such file or directory
01:17:06 + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
01:17:06 + CONTROLLERIP=ODL_SYSTEM_2_IP
01:17:06 + echo 'Compressing karaf.log 2'
01:17:06 Compressing karaf.log 2
01:17:06 + ssh 10.30.170.81 gzip --best /tmp/karaf-0.22.0-SNAPSHOT/data/log/karaf.log
01:17:06 Warning: Permanently added '10.30.170.81' (ECDSA) to the list of known hosts.
01:17:07 + echo 'Fetching compressed karaf.log 2'
01:17:07 Fetching compressed karaf.log 2
01:17:07 + scp 10.30.170.81:/tmp/karaf-0.22.0-SNAPSHOT/data/log/karaf.log.gz odl2_karaf.log.gz
01:17:07 Warning: Permanently added '10.30.170.81' (ECDSA) to the list of known hosts.
01:17:07 + ssh 10.30.170.81 rm -f /tmp/karaf-0.22.0-SNAPSHOT/data/log/karaf.log.gz
01:17:07 Warning: Permanently added '10.30.170.81' (ECDSA) to the list of known hosts.
01:17:07 + scp 10.30.170.81:/tmp/karaf-0.22.0-SNAPSHOT/data/log/karaf_console.log odl2_karaf_console.log
01:17:07 Warning: Permanently added '10.30.170.81' (ECDSA) to the list of known hosts.
01:17:07 + ssh 10.30.170.81 rm -f /tmp/karaf-0.22.0-SNAPSHOT/data/log/karaf_console.log
01:17:07 Warning: Permanently added '10.30.170.81' (ECDSA) to the list of known hosts.
01:17:08 + echo 'Fetch GC logs'
01:17:08 Fetch GC logs
01:17:08 + mkdir -p gclogs-2
01:17:08 + scp '10.30.170.81:/tmp/karaf-0.22.0-SNAPSHOT/data/log/*.log' gclogs-2/
01:17:08 Warning: Permanently added '10.30.170.81' (ECDSA) to the list of known hosts.
01:17:08 scp: /tmp/karaf-0.22.0-SNAPSHOT/data/log/*.log: No such file or directory
01:17:08 + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
01:17:08 + CONTROLLERIP=ODL_SYSTEM_3_IP
01:17:08 + echo 'Compressing karaf.log 3'
01:17:08 Compressing karaf.log 3
01:17:08 + ssh 10.30.170.131 gzip --best /tmp/karaf-0.22.0-SNAPSHOT/data/log/karaf.log
01:17:08 Warning: Permanently added '10.30.170.131' (ECDSA) to the list of known hosts.
01:17:09 gzip: /tmp/karaf-0.22.0-SNAPSHOT/data/log/karaf.log: file size changed while zipping
01:17:09 + echo 'Fetching compressed karaf.log 3'
01:17:09 Fetching compressed karaf.log 3
01:17:09 + scp 10.30.170.131:/tmp/karaf-0.22.0-SNAPSHOT/data/log/karaf.log.gz odl3_karaf.log.gz
01:17:09 Warning: Permanently added '10.30.170.131' (ECDSA) to the list of known hosts.
01:17:09 + ssh 10.30.170.131 rm -f /tmp/karaf-0.22.0-SNAPSHOT/data/log/karaf.log.gz
01:17:09 Warning: Permanently added '10.30.170.131' (ECDSA) to the list of known hosts.
01:17:09 + scp 10.30.170.131:/tmp/karaf-0.22.0-SNAPSHOT/data/log/karaf_console.log odl3_karaf_console.log
01:17:09 Warning: Permanently added '10.30.170.131' (ECDSA) to the list of known hosts.
01:17:09 + ssh 10.30.170.131 rm -f /tmp/karaf-0.22.0-SNAPSHOT/data/log/karaf_console.log
01:17:09 Warning: Permanently added '10.30.170.131' (ECDSA) to the list of known hosts.
01:17:09 + echo 'Fetch GC logs'
01:17:09 Fetch GC logs
01:17:09 + mkdir -p gclogs-3
01:17:09 + scp '10.30.170.131:/tmp/karaf-0.22.0-SNAPSHOT/data/log/*.log' gclogs-3/
01:17:09 Warning: Permanently added '10.30.170.131' (ECDSA) to the list of known hosts.
01:17:10 scp: /tmp/karaf-0.22.0-SNAPSHOT/data/log/*.log: No such file or directory
01:17:10 + echo 'Examine copied files'
01:17:10 Examine copied files
01:17:10 + ls -lt
01:17:10 total 55852
01:17:10 drwxrwxr-x. 2 jenkins jenkins 6 Mar 17 01:17 gclogs-3
01:17:10 -rw-rw-r--. 1 jenkins jenkins 1720 Mar 17 01:17 odl3_karaf_console.log
01:17:10 -rw-rw-r--. 1 jenkins jenkins 954704 Mar 17 01:17 odl3_karaf.log.gz
01:17:10 drwxrwxr-x. 2 jenkins jenkins 6 Mar 17 01:17 gclogs-2
01:17:10 -rw-rw-r--. 1 jenkins jenkins 1720 Mar 17 01:17 odl2_karaf_console.log
01:17:10 -rw-rw-r--. 1 jenkins jenkins 925764 Mar 17 01:17 odl2_karaf.log.gz
01:17:10 drwxrwxr-x. 2 jenkins jenkins 6 Mar 17 01:17 gclogs-1
01:17:10 -rw-rw-r--. 1 jenkins jenkins 1720 Mar 17 01:17 odl1_karaf_console.log
01:17:10 -rw-rw-r--. 1 jenkins jenkins 234047 Mar 17 01:17 odl1_karaf.log.gz
01:17:10 -rw-rw-r--. 1 jenkins jenkins 113754 Mar 17 01:16 karaf_3_69860_threads_after.log
01:17:10 -rw-rw-r--. 1 jenkins jenkins 16322 Mar 17 01:16 ps_after.log
01:17:10 -rw-rw-r--. 1 jenkins jenkins 114060 Mar 17 01:16 karaf_2_53660_threads_after.log
01:17:10 -rw-rw-r--. 1 jenkins jenkins 168355 Mar 17 01:16 karaf_1_34745_threads_after.log
01:17:10 -rw-rw-r--. 1 jenkins jenkins 309479 Mar 17 01:16 report.html
01:17:10 -rw-rw-r--. 1 jenkins jenkins 3358961 Mar 17 01:16 log.html
01:17:10 -rw-rw-r--. 1 jenkins jenkins 50465655 Mar 17 01:16 output.xml
01:17:10 -rw-rw-r--. 1 jenkins jenkins 4120 Mar 16 23:08 testplan.txt
01:17:10 -rw-rw-r--. 1 jenkins jenkins 135216 Mar 16 23:08 karaf_3_28770_threads_before.log
01:17:10 -rw-rw-r--. 1 jenkins jenkins 16245 Mar 16 23:08 ps_before.log
01:17:10 -rw-rw-r--. 1 jenkins jenkins 135097 Mar 16 23:08 karaf_2_28775_threads_before.log
01:17:10 -rw-rw-r--. 1 jenkins jenkins 136189 Mar 16 23:08 karaf_1_28758_threads_before.log
01:17:10 -rw-rw-r--. 1 jenkins jenkins 3106 Mar 16 23:05 post-startup-script.sh
01:17:10 -rw-rw-r--. 1 jenkins jenkins 252 Mar 16 23:04 startup-script.sh
01:17:10 -rw-rw-r--. 1 jenkins jenkins 3463 Mar 16 23:04 configuration-script.sh
01:17:10 -rw-rw-r--. 1 jenkins jenkins 695 Mar 16 23:04 custom_shard_config.txt
01:17:10 -rw-rw-r--. 1 jenkins jenkins 135 Mar 16 23:04 scriptplan.txt
01:17:10 -rw-rw-r--. 1 jenkins jenkins 339 Mar 16 23:04 detect_variables.env
01:17:10 -rw-rw-r--. 1 jenkins jenkins 2619 Mar 16 23:04 pom.xml
01:17:10 -rw-rw-r--. 1 jenkins jenkins 92 Mar 16 23:04 set_variables.env
01:17:10 -rw-rw-r--. 1 jenkins jenkins 313 Mar 16 23:04 slave_addresses.txt
01:17:10 -rw-rw-r--. 1 jenkins jenkins 570 Mar 16 23:04 requirements.txt
01:17:10 -rw-rw-r--. 1 jenkins jenkins 26 Mar 16 23:04 env.properties
01:17:10 -rw-rw-r--. 1 jenkins jenkins 333 Mar 16 23:02 stack-parameters.yaml
01:17:10 drwxrwxr-x. 7 jenkins jenkins 4096 Mar 16 23:01 test
01:17:10 drwxrwxr-x. 2 jenkins jenkins 6 Mar 16 23:01 test@tmp
01:17:10 -rw-rw-r--. 1 jenkins jenkins 1416 Mar 16 17:54 maven-metadata.xml
01:17:10 + true
01:17:10 [controller-csit-3node-clustering-ask-all-titanium] $ /bin/sh /tmp/jenkins12203057853218706093.sh
01:17:10 Cleaning up Robot installation...
01:17:10 $ ssh-agent -k
01:17:10 unset SSH_AUTH_SOCK;
01:17:10 unset SSH_AGENT_PID;
01:17:10 echo Agent pid 5289 killed;
01:17:10 [ssh-agent] Stopped.
01:17:10 Recording plot data
01:17:10 Robot results publisher started...
01:17:10 INFO: Checking test criticality is deprecated and will be dropped in a future release!
01:17:10 -Parsing output xml:
01:17:12 Done!
01:17:12 -Copying log files to build dir:
01:17:14 Done!
01:17:14 -Assigning results to build:
01:17:14 Done!
01:17:14 -Checking thresholds:
01:17:14 Done!
01:17:14 Done publishing Robot results.
01:17:14 Build step 'Publish Robot Framework test results' changed build result to UNSTABLE
01:17:14 [PostBuildScript] - [INFO] Executing post build scripts.
01:17:14 [controller-csit-3node-clustering-ask-all-titanium] $ /bin/bash /tmp/jenkins3573395465147601032.sh
01:17:14 Archiving csit artifacts
01:17:14 mv: cannot stat '*_1.png': No such file or directory
01:17:14 mv: cannot stat '/tmp/odl1_*': No such file or directory
01:17:14 mv: cannot stat '*_2.png': No such file or directory
01:17:14 mv: cannot stat '/tmp/odl2_*': No such file or directory
01:17:14 mv: cannot stat '*_3.png': No such file or directory
01:17:14 mv: cannot stat '/tmp/odl3_*': No such file or directory
01:17:14 % Total % Received % Xferd Average Speed Time Time Time Current
01:17:14 Dload Upload Total Spent Left Speed
01:17:14
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 1512k 0 1512k 0 0 3886k 0 --:--:-- --:--:-- --:--:-- 3876k
100 4561k 0 4561k 0 0 4621k 0 --:--:-- --:--:-- --:--:-- 4617k
01:17:15 Archive: robot-plugin.zip
01:17:15 inflating: ./archives/robot-plugin/log.html
01:17:15 inflating: ./archives/robot-plugin/output.xml
01:17:15 inflating: ./archives/robot-plugin/report.html
01:17:15 mv: cannot stat '*.log.gz': No such file or directory
01:17:15 mv: cannot stat '*.csv': No such file or directory
01:17:15 mv: cannot stat '*.png': No such file or directory
01:17:15 [PostBuildScript] - [INFO] Executing post build scripts.
01:17:15 [controller-csit-3node-clustering-ask-all-titanium] $ /bin/bash /tmp/jenkins2574461886669578351.sh
01:17:15 [PostBuildScript] - [INFO] Executing post build scripts.
01:17:15 [EnvInject] - Injecting environment variables from a build step.
01:17:15 [EnvInject] - Injecting as environment variables the properties content
01:17:15 OS_CLOUD=vex
01:17:15 OS_STACK_NAME=releng-controller-csit-3node-clustering-ask-all-titanium-29
01:17:15
01:17:15 [EnvInject] - Variables injected successfully.
01:17:15 provisioning config files...
01:17:16 copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
01:17:16 [controller-csit-3node-clustering-ask-all-titanium] $ /bin/bash /tmp/jenkins2464707580237712185.sh
01:17:16 ---> openstack-stack-delete.sh
01:17:16 Setup pyenv:
01:17:16 system
01:17:16 3.8.13
01:17:16 3.9.13
01:17:16 3.10.13
01:17:16 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-titanium/.python-version)
01:17:16 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-Vbzb from file:/tmp/.os_lf_venv
01:17:17 lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes python-heatclient python-openstackclient
01:17:29 lf-activate-venv(): INFO: Adding /tmp/venv-Vbzb/bin to PATH
01:17:29 INFO: Retrieving stack cost for: releng-controller-csit-3node-clustering-ask-all-titanium-29
01:17:34 DEBUG: Successfully retrieved stack cost: total: 0.99
01:17:46 INFO: Deleting stack releng-controller-csit-3node-clustering-ask-all-titanium-29
01:17:46 Successfully deleted stack releng-controller-csit-3node-clustering-ask-all-titanium-29
01:17:46 [PostBuildScript] - [INFO] Executing post build scripts.
01:17:46 [controller-csit-3node-clustering-ask-all-titanium] $ /bin/bash /tmp/jenkins14558161198361590497.sh
01:17:46 ---> sysstat.sh
01:17:46 [controller-csit-3node-clustering-ask-all-titanium] $ /bin/bash /tmp/jenkins4153180425061576923.sh
01:17:46 ---> package-listing.sh
01:17:46 ++ facter osfamily
01:17:46 ++ tr '[:upper:]' '[:lower:]'
01:17:46 + OS_FAMILY=redhat
01:17:46 + workspace=/w/workspace/controller-csit-3node-clustering-ask-all-titanium
01:17:46 + START_PACKAGES=/tmp/packages_start.txt
01:17:46 + END_PACKAGES=/tmp/packages_end.txt
01:17:46 + DIFF_PACKAGES=/tmp/packages_diff.txt
01:17:46 + PACKAGES=/tmp/packages_start.txt
01:17:46 + '[' /w/workspace/controller-csit-3node-clustering-ask-all-titanium ']'
01:17:46 + PACKAGES=/tmp/packages_end.txt
01:17:46 + case "${OS_FAMILY}" in
01:17:46 + rpm -qa
01:17:46 + sort
01:17:47 + '[' -f /tmp/packages_start.txt ']'
01:17:47 + '[' -f /tmp/packages_end.txt ']'
01:17:47 + diff /tmp/packages_start.txt /tmp/packages_end.txt
01:17:47 + '[' /w/workspace/controller-csit-3node-clustering-ask-all-titanium ']'
01:17:47 + mkdir -p /w/workspace/controller-csit-3node-clustering-ask-all-titanium/archives/
01:17:47 + cp -f /tmp/packages_diff.txt /tmp/packages_end.txt /tmp/packages_start.txt /w/workspace/controller-csit-3node-clustering-ask-all-titanium/archives/
01:17:47 [controller-csit-3node-clustering-ask-all-titanium] $ /bin/bash /tmp/jenkins15485932933771858965.sh
01:17:47 ---> capture-instance-metadata.sh
01:17:47 Setup pyenv:
01:17:47 system
01:17:47 3.8.13
01:17:47 3.9.13
01:17:47 3.10.13
01:17:47 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-titanium/.python-version)
01:17:48 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-Vbzb from file:/tmp/.os_lf_venv
01:17:48 lf-activate-venv(): INFO: Installing: lftools
01:17:58 lf-activate-venv(): INFO: Adding /tmp/venv-Vbzb/bin to PATH
01:17:58 INFO: Running in OpenStack, capturing instance metadata
01:17:58 [controller-csit-3node-clustering-ask-all-titanium] $ /bin/bash /tmp/jenkins5497641996839642245.sh
01:17:58 provisioning config files...
01:17:58 Could not find credentials [logs] for controller-csit-3node-clustering-ask-all-titanium #29
01:17:58 copy managed file [jenkins-log-archives-settings] to file:/w/workspace/controller-csit-3node-clustering-ask-all-titanium@tmp/config6544022829043333124tmp
01:17:58 Regular expression run condition: Expression=[^.*logs-s3.*], Label=[odl-logs-s3-cloudfront-index]
01:17:58 Run condition [Regular expression match] enabling perform for step [Provide Configuration files]
01:17:58 provisioning config files...
01:17:58 copy managed file [jenkins-s3-log-ship] to file:/home/jenkins/.aws/credentials
01:17:58 [EnvInject] - Injecting environment variables from a build step.
01:17:58 [EnvInject] - Injecting as environment variables the properties content
01:17:58 SERVER_ID=logs
01:17:58
01:17:58 [EnvInject] - Variables injected successfully.
01:17:58 [controller-csit-3node-clustering-ask-all-titanium] $ /bin/bash /tmp/jenkins13384287880152491032.sh
01:17:58 ---> create-netrc.sh
01:17:58 WARN: Log server credential not found.
01:17:59 [controller-csit-3node-clustering-ask-all-titanium] $ /bin/bash /tmp/jenkins2429599780706265834.sh
01:17:59 ---> python-tools-install.sh
01:17:59 Setup pyenv:
01:17:59 system
01:17:59 3.8.13
01:17:59 3.9.13
01:17:59 3.10.13
01:17:59 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-titanium/.python-version)
01:17:59 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-Vbzb from file:/tmp/.os_lf_venv
01:18:00 lf-activate-venv(): INFO: Installing: lftools
01:18:07 lf-activate-venv(): INFO: Adding /tmp/venv-Vbzb/bin to PATH
01:18:07 [controller-csit-3node-clustering-ask-all-titanium] $ /bin/bash /tmp/jenkins5702466574121155987.sh
01:18:07 ---> sudo-logs.sh
01:18:07 Archiving 'sudo' log..
01:18:08 [controller-csit-3node-clustering-ask-all-titanium] $ /bin/bash /tmp/jenkins3145029482046957624.sh
01:18:08 ---> job-cost.sh
01:18:08 Setup pyenv:
01:18:08 system
01:18:08 3.8.13
01:18:08 3.9.13
01:18:08 3.10.13
01:18:08 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-titanium/.python-version)
01:18:08 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-Vbzb from file:/tmp/.os_lf_venv
01:18:09 lf-activate-venv(): INFO: Installing: zipp==1.1.0 python-openstackclient urllib3~=1.26.15
01:18:15 lf-activate-venv(): INFO: Adding /tmp/venv-Vbzb/bin to PATH
01:18:15 DEBUG: total: 0.99
01:18:15 INFO: Retrieving Stack Cost...
01:18:16 INFO: Retrieving Pricing Info for: v3-standard-2
01:18:16 INFO: Archiving Costs
01:18:16 [controller-csit-3node-clustering-ask-all-titanium] $ /bin/bash -l /tmp/jenkins8452718017764406602.sh
01:18:16 ---> logs-deploy.sh
01:18:16 Setup pyenv:
01:18:16 system
01:18:16 3.8.13
01:18:16 3.9.13
01:18:16 3.10.13
01:18:16 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-titanium/.python-version)
01:18:17 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-Vbzb from file:/tmp/.os_lf_venv
01:18:17 lf-activate-venv(): INFO: Installing: lftools
01:18:26 lf-activate-venv(): INFO: Adding /tmp/venv-Vbzb/bin to PATH
01:18:26 WARNING: Nexus logging server not set
01:18:26 INFO: S3 path logs/releng/vex-yul-odl-jenkins-1/controller-csit-3node-clustering-ask-all-titanium/29/
01:18:26 INFO: archiving logs to S3
01:18:27 ---> uname -a:
01:18:27 Linux prd-centos8-robot-2c-8g-8068.novalocal 4.18.0-553.5.1.el8.x86_64 #1 SMP Tue May 21 05:46:01 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
01:18:27
01:18:27
01:18:27 ---> lscpu:
01:18:27 Architecture: x86_64
01:18:27 CPU op-mode(s): 32-bit, 64-bit
01:18:27 Byte Order: Little Endian
01:18:27 CPU(s): 2
01:18:27 On-line CPU(s) list: 0,1
01:18:27 Thread(s) per core: 1
01:18:27 Core(s) per socket: 1
01:18:27 Socket(s): 2
01:18:27 NUMA node(s): 1
01:18:27 Vendor ID: AuthenticAMD
01:18:27 CPU family: 23
01:18:27 Model: 49
01:18:27 Model name: AMD EPYC-Rome Processor
01:18:27 Stepping: 0
01:18:27 CPU MHz: 2800.000
01:18:27 BogoMIPS: 5600.00
01:18:27 Virtualization: AMD-V
01:18:27 Hypervisor vendor: KVM
01:18:27 Virtualization type: full
01:18:27 L1d cache: 32K
01:18:27 L1i cache: 32K
01:18:27 L2 cache: 512K
01:18:27 L3 cache: 16384K
01:18:27 NUMA node0 CPU(s): 0,1
01:18:27 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr wbnoinvd arat npt nrip_save umip rdpid arch_capabilities
01:18:27
01:18:27
01:18:27 ---> nproc:
01:18:27 2
01:18:27
01:18:27
01:18:27 ---> df -h:
01:18:27 Filesystem Size Used Avail Use% Mounted on
01:18:27 devtmpfs 3.8G 0 3.8G 0% /dev
01:18:27 tmpfs 3.8G 0 3.8G 0% /dev/shm
01:18:27 tmpfs 3.8G 17M 3.8G 1% /run
01:18:27 tmpfs 3.8G 0 3.8G 0% /sys/fs/cgroup
01:18:27 /dev/vda1 40G 8.4G 32G 21% /
01:18:27 tmpfs 770M 0 770M 0% /run/user/1001
01:18:27
01:18:27
01:18:27 ---> free -m:
01:18:27 total used free shared buff/cache available
01:18:27 Mem: 7697 715 4706 19 2276 6685
01:18:27 Swap: 1023 0 1023
01:18:27
01:18:27
01:18:27 ---> ip addr:
01:18:27 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
01:18:27 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
01:18:27 inet 127.0.0.1/8 scope host lo
01:18:27 valid_lft forever preferred_lft forever
01:18:27 inet6 ::1/128 scope host
01:18:27 valid_lft forever preferred_lft forever
01:18:27 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1458 qdisc mq state UP group default qlen 1000
01:18:27 link/ether fa:16:3e:8c:fb:26 brd ff:ff:ff:ff:ff:ff
01:18:27 altname enp0s3
01:18:27 altname ens3
01:18:27 inet 10.30.171.43/23 brd 10.30.171.255 scope global dynamic noprefixroute eth0
01:18:27 valid_lft 78159sec preferred_lft 78159sec
01:18:27 inet6 fe80::f816:3eff:fe8c:fb26/64 scope link
01:18:27 valid_lft forever preferred_lft forever
01:18:27
01:18:27
01:18:27 ---> sar -b -r -n DEV:
01:18:27 Linux 4.18.0-553.5.1.el8.x86_64 (prd-centos8-robot-2c-8g-8068.novalocal) 03/17/2025 _x86_64_ (2 CPU)
01:18:27
01:18:27 12:00:01 AM tps rtps wtps bread/s bwrtn/s
01:18:27 12:01:01 AM 0.98 0.02 0.97 0.13 225.09
01:18:27 12:02:01 AM 1.00 0.07 0.93 1.73 26.80
01:18:27 12:03:02 AM 0.57 0.00 0.57 0.00 18.18
01:18:27 12:04:01 AM 0.51 0.00 0.51 0.00 17.88
01:18:27 12:05:01 AM 0.52 0.00 0.52 0.00 16.68
01:18:27 12:06:01 AM 0.53 0.00 0.53 0.00 17.68
01:18:27 12:07:01 AM 0.70 0.00 0.70 0.00 32.29
01:18:27 12:08:01 AM 0.77 0.07 0.70 7.60 16.96
01:18:27 12:09:01 AM 0.40 0.00 0.40 0.00 6.91
01:18:27 12:10:01 AM 0.40 0.00 0.40 0.00 6.92
01:18:27 12:11:01 AM 0.50 0.00 0.50 0.00 8.30
01:18:27 12:12:01 AM 0.45 0.00 0.45 0.00 7.50
01:18:27 12:13:01 AM 0.63 0.00 0.63 0.00 16.48
01:18:27 12:14:01 AM 0.25 0.00 0.25 0.00 26.83
01:18:27 12:15:01 AM 0.23 0.00 0.23 0.00 20.11
01:18:27 12:16:01 AM 1.13 0.32 0.82 6.93 27.43
01:18:27 12:17:01 AM 0.32 0.00 0.32 0.00 16.79
01:18:27 12:18:01 AM 0.27 0.00 0.27 0.00 17.88
01:18:27 12:19:01 AM 0.27 0.00 0.27 0.00 10.88
01:18:27 12:20:01 AM 0.23 0.00 0.23 0.00 25.95
01:18:27 12:21:01 AM 0.20 0.00 0.20 0.00 5.68
01:18:27 12:22:01 AM 0.28 0.00 0.28 0.00 6.10
01:18:27 12:23:01 AM 0.28 0.00 0.28 0.00 7.88
01:18:27 12:24:01 AM 0.12 0.00 0.12 0.00 2.95
01:18:27 12:25:01 AM 0.17 0.00 0.17 0.00 5.61
01:18:27 12:26:01 AM 0.22 0.00 0.22 0.00 14.55
01:18:27 12:27:01 AM 0.42 0.00 0.42 0.00 21.46
01:18:27 12:28:01 AM 0.17 0.00 0.17 0.00 8.55
01:18:27 12:29:01 AM 0.17 0.00 0.17 0.00 16.66
01:18:27 12:30:01 AM 0.23 0.00 0.23 0.00 15.33
01:18:27 12:31:01 AM 0.28 0.00 0.28 0.00 14.43
01:18:27 12:32:01 AM 0.37 0.00 0.37 0.00 14.23
01:18:27 12:33:01 AM 0.25 0.00 0.25 0.00 17.73
01:18:27 12:34:01 AM 0.20 0.00 0.20 0.00 18.43
01:18:27 12:35:01 AM 0.15 0.00 0.15 0.00 5.60
01:18:27 12:36:01 AM 0.28 0.00 0.28 0.00 6.28
01:18:27 12:37:01 AM 0.33 0.00 0.33 0.00 7.72
01:18:27 12:38:01 AM 0.15 0.00 0.15 0.00 5.06
01:18:27 12:39:01 AM 0.13 0.00 0.13 0.00 30.01
01:18:27 12:40:01 AM 0.17 0.00 0.17 0.00 17.88
01:18:27 12:41:01 AM 0.37 0.00 0.37 0.00 14.75
01:18:27 12:42:01 AM 0.83 0.00 0.83 0.00 22.61
01:18:27 12:43:01 AM 0.52 0.00 0.52 0.00 17.18
01:18:27 12:44:01 AM 0.52 0.00 0.52 0.00 17.08
01:18:27 12:45:01 AM 0.52 0.00 0.52 0.00 17.31
01:18:27 12:46:01 AM 0.22 0.00 0.22 0.00 19.64
01:18:27 12:47:01 AM 0.35 0.00 0.35 0.00 9.47
01:18:27 12:48:01 AM 0.27 0.00 0.27 0.00 4.83
01:18:27 12:49:01 AM 0.18 0.00 0.18 0.00 6.15
01:18:27 12:50:01 AM 0.13 0.00 0.13 0.00 3.07
01:18:27 12:51:01 AM 0.17 0.00 0.17 0.00 5.76
01:18:27 12:52:01 AM 0.37 0.00 0.37 0.00 36.67
01:18:27 12:53:01 AM 0.17 0.00 0.17 0.00 15.18
01:18:27 12:54:01 AM 0.13 0.00 0.13 0.00 14.95
01:18:27 12:55:01 AM 0.22 0.00 0.22 0.00 10.26
01:18:27 12:56:01 AM 0.35 0.00 0.35 0.00 18.68
01:18:27 12:57:01 AM 0.33 0.00 0.33 0.00 20.13
01:18:27 12:58:01 AM 0.15 0.00 0.15 0.00 25.60
01:18:27 12:59:01 AM 0.12 0.00 0.12 0.00 3.33
01:18:27 01:00:01 AM 0.22 0.00 0.22 0.00 5.81
01:18:27 01:01:01 AM 0.25 0.00 0.25 0.00 4.85
01:18:27 01:02:01 AM 0.45 0.00 0.45 0.00 9.11
01:18:27 01:03:01 AM 0.13 0.00 0.13 0.00 5.61
01:18:27 01:04:01 AM 0.18 0.00 0.18 0.00 16.16
01:18:27 01:05:02 AM 0.17 0.00 0.17 0.00 24.54
01:18:27 01:06:01 AM 0.32 0.00 0.32 0.00 20.06
01:18:27 01:07:01 AM 1.02 0.00 1.02 0.00 30.21
01:18:27 01:08:01 AM 0.30 0.00 0.30 0.00 14.95
01:18:27 01:09:01 AM 0.27 0.00 0.27 0.00 15.46
01:18:27 01:10:01 AM 0.52 0.00 0.52 0.00 17.58
01:18:27 01:11:01 AM 0.43 0.00 0.43 0.00 23.08
01:18:27 01:12:01 AM 0.33 0.00 0.33 0.00 9.16
01:18:27 01:13:01 AM 0.12 0.00 0.12 0.00 3.20
01:18:27 01:14:01 AM 0.15 0.00 0.15 0.00 5.20
01:18:27 01:15:01 AM 0.12 0.00 0.12 0.00 3.17
01:18:27 01:16:01 AM 0.28 0.00 0.28 0.00 7.28
01:18:27 01:17:01 AM 0.38 0.00 0.38 0.00 28.85
01:18:27 01:18:01 AM 26.96 0.52 26.44 55.45 3348.33
01:18:27 Average: 0.69 0.01 0.68 0.92 60.03
01:18:27
01:18:27 12:00:01 AM kbmemfree kbavail kbmemused %memused kbbuffers kbcached kbcommit %commit kbactive kbinact kbdirty
01:18:27 12:01:01 AM 4800356 6912612 3082076 39.10 2688 2281372 833204 9.33 267676 2439904 52
01:18:27 12:02:01 AM 4800396 6913084 3082036 39.10 2688 2281804 833204 9.33 267844 2439924 36
01:18:27 12:03:02 AM 4800496 6913544 3081936 39.10 2688 2282164 833204 9.33 267844 2440104 20
01:18:27 12:04:01 AM 4799620 6913036 3082812 39.11 2688 2282524 833204 9.33 267844 2440784 12
01:18:27 12:05:01 AM 4799660 6913440 3082772 39.11 2688 2282892 848952 9.51 267844 2440960 28
01:18:27 12:06:01 AM 4799064 6913212 3083368 39.12 2688 2283268 848952 9.51 267844 2441448 24
01:18:27 12:07:01 AM 4797880 6912712 3084552 39.13 2688 2283952 867756 9.72 267844 2442216 24
01:18:27 12:08:01 AM 4798048 6913332 3084384 39.13 2688 2284404 831716 9.31 267932 2442564 24
01:18:27 12:09:01 AM 4797548 6912916 3084884 39.14 2688 2284480 831716 9.31 267932 2442788 4
01:18:27 12:10:01 AM 4797020 6912480 3085412 39.14 2688 2284572 833988 9.34 267932 2443268 4
01:18:27 12:11:01 AM 4796572 6912128 3085860 39.15 2688 2284668 833988 9.34 267936 2443876 4
01:18:27 12:12:01 AM 4796424 6912072 3086008 39.15 2688 2284760 833988 9.34 267936 2444052 4
01:18:27 12:13:01 AM 4795368 6911692 3087064 39.16 2688 2285436 833988 9.34 267956 2445016 412
01:18:27 12:14:01 AM 4794928 6911748 3087504 39.17 2688 2285932 833988 9.34 267956 2445528 172
01:18:27 12:15:01 AM 4794588 6911768 3087844 39.17 2688 2286296 833988 9.34 267956 2445872 40
01:18:27 12:16:01 AM 4794032 6911792 3088400 39.18 2688 2286860 833988 9.34 268072 2446344 144
01:18:27 12:17:01 AM 4793456 6911584 3088976 39.19 2688 2287236 833988 9.34 268072 2446784 116
01:18:27 12:18:01 AM 4793292 6911780 3089140 39.19 2688 2287596 833988 9.34 268072 2447076 68
01:18:27 12:19:01 AM 4792640 6911492 3089792 39.20 2688 2287960 833988 9.34 268072 2447556 216
01:18:27 12:20:01 AM 4791824 6911236 3090608 39.21 2688 2288512 854636 9.57 268072 2448160 52
01:18:27 12:21:01 AM 4791980 6911492 3090452 39.21 2688 2288608 854636 9.57 268072 2448140 24
01:18:27 12:22:01 AM 4791904 6911500 3090528 39.21 2688 2288700 854636 9.57 268072 2448292 56
01:18:27 12:23:01 AM 4791908 6911596 3090524 39.21 2688 2288792 854636 9.57 268072 2448316 24
01:18:27 12:24:01 AM 4791472 6911260 3090960 39.21 2688 2288884 854636 9.57 268072 2448728 56
01:18:27 12:25:01 AM 4791304 6911176 3091128 39.22 2688 2288976 854636 9.57 268072 2448776 36
01:18:27 12:26:01 AM 4793296 6913512 3089136 39.19 2688 2289312 833296 9.33 268072 2446816 8
01:18:27 12:27:01 AM 4790932 6911596 3091500 39.22 2688 2289768 848704 9.50 268072 2449380 36
01:18:27 12:28:01 AM 4790080 6911112 3092352 39.23 2688 2290132 848704 9.50 268072 2449908 204
01:18:27 12:29:01 AM 4789860 6911252 3092572 39.23 2688 2290492 848704 9.50 268072 2450216 100
01:18:27 12:30:01 AM 4789420 6911172 3093012 39.24 2688 2290852 848704 9.50 268072 2450600 52
01:18:27 12:31:01 AM 4789304 6911420 3093128 39.24 2688 2291216 848704 9.50 268072 2450852 84
01:18:27 12:32:01 AM 4788708 6911200 3093724 39.25 2688 2291592 848704 9.50 268072 2451412 204
01:18:27 12:33:01 AM 4788080 6911256 3094352 39.26 2688 2292268 848704 9.50 268072 2452316 456
01:18:27 12:34:01 AM 4788148 6911416 3094284 39.26 2688 2292360 848704 9.50 268072 2452020 44
01:18:27 12:35:01 AM 4787476 6910836 3094956 39.26 2688 2292452 848704 9.50 268072 2452532 4
01:18:27 12:36:01 AM 4787940 6911384 3094492 39.26 2688 2292544 848704 9.50 268100 2452272 36
01:18:27 12:37:01 AM 4787900 6911432 3094532 39.26 2688 2292636 848704 9.50 268100 2452432 68
01:18:27 12:38:01 AM 4787568 6911192 3094864 39.26 2688 2292728 848704 9.50 268100 2452472 44
01:18:27 12:39:01 AM 4786080 6910620 3096352 39.28 2688 2293636 827224 9.26 268100 2453832 80
01:18:27 12:40:01 AM 4785516 6910472 3096916 39.29 2688 2294052 824320 9.23 268100 2454560 4
01:18:27 12:41:01 AM 4785636 6910956 3096796 39.29 2688 2294420 824320 9.23 268100 2454600 36
01:18:27 12:42:01 AM 4784760 6910448 3097672 39.30 2688 2294780 824320 9.23 268100 2455368 20
01:18:27 12:43:01 AM 4784716 6910756 3097716 39.30 2688 2295140 824320 9.23 268100 2455360 4
01:18:27 12:44:01 AM 4784140 6910560 3098292 39.31 2688 2295520 824320 9.23 268100 2456048 36
01:18:27 12:45:01 AM 4783868 6910664 3098564 39.31 2688 2295884 824320 9.23 268108 2456176 24
01:18:27 12:46:01 AM 4783568 6910892 3098864 39.31 2688 2296420 824320 9.23 268100 2456628 40
01:18:27 12:47:01 AM 4783412 6910832 3099020 39.32 2688 2296516 824320 9.23 268100 2456908 4
01:18:27 12:48:01 AM 4782876 6910396 3099556 39.32 2688 2296608 824320 9.23 268100 2457020 52
01:18:27 12:49:01 AM 4783044 6910652 3099388 39.32 2688 2296696 824320 9.23 268100 2457008 8
01:18:27 12:50:01 AM 4782604 6910308 3099828 39.33 2688 2296792 835160 9.35 268100 2457384 52
01:18:27 12:51:01 AM 4782952 6910752 3099480 39.32 2688 2296884 835160 9.35 268100 2457272 8
01:18:27 12:52:01 AM 4781520 6910364 3100912 39.34 2688 2297936 835936 9.36 268100 2458492 128
01:18:27 12:53:01 AM 4781564 6910780 3100868 39.34 2688 2298300 835936 9.36 268100 2458660 84
01:18:27 12:54:01 AM 4780664 6910236 3101768 39.35 2688 2298664 835936 9.36 268100 2459464 44
01:18:27 12:55:01 AM 4780300 6910252 3102132 39.36 2688 2299044 835936 9.36 268100 2459604 208
01:18:27 12:56:01 AM 4780496 6910808 3101936 39.35 2688 2299404 835936 9.36 268096 2459852 160
01:18:27 12:57:01 AM 4780456 6911136 3101976 39.35 2688 2299764 835936 9.36 268096 2459900 84
01:18:27 12:58:01 AM 4779812 6911156 3102620 39.36 2688 2300424 835936 9.36 268096 2460492 4
01:18:27 12:59:01 AM 4779548 6910976 3102884 39.36 2688 2300516 835936 9.36 268096 2460904 36
01:18:27 01:00:01 AM 4779436 6910956 3102996 39.37 2688 2300608 831272 9.31 268096 2460980 4
01:18:27 01:01:01 AM 4778388 6910004 3104044 39.38 2688 2300704 831272 9.31 268096 2461544 40
01:18:27 01:02:01 AM 4779188 6910896 3103244 39.37 2688 2300796 831272 9.31 268112 2461012 84
01:18:27 01:03:01 AM 4779296 6911096 3103136 39.37 2688 2300888 831272 9.31 268112 2461016 40
01:18:27 01:04:01 AM 4778472 6910908 3103960 39.38 2688 2301516 831272 9.31 268112 2461668 228
01:18:27 01:05:02 AM 4777420 6910512 3105012 39.39 2688 2302180 831272 9.31 268112 2462588 200
01:18:27 01:06:01 AM 4777340 6910792 3105092 39.39 2688 2302540 834660 9.35 268112 2462792 80
01:18:27 01:07:01 AM 4776980 6910804 3105452 39.40 2688 2302912 834656 9.35 277368 2453864 24
01:18:27 01:08:01 AM 4776888 6911072 3105544 39.40 2688 2303272 834656 9.35 277368 2454156 68
01:18:27 01:09:01 AM 4776452 6911000 3105980 39.40 2688 2303632 834656 9.35 277368 2454516 24
01:18:27 01:10:01 AM 4775512 6910432 3106920 39.42 2688 2303996 826532 9.25 277368 2455284 24
01:18:27 01:11:01 AM 4775440 6910908 3106992 39.42 2688 2304552 826532 9.25 277368 2455560 52
01:18:27 01:12:01 AM 4775092 6910656 3107340 39.42 2688 2304648 826532 9.25 277368 2455744 36
01:18:27 01:13:01 AM 4775044 6910700 3107388 39.42 2688 2304740 826532 9.25 277368 2455908 56
01:18:27 01:14:01 AM 4775260 6911000 3107172 39.42 2688 2304816 826532 9.25 277368 2455836 4
01:18:27 01:15:01 AM 4775036 6910872 3107396 39.42 2688 2304912 826532 9.25 277368 2455944 44
01:18:27 01:16:01 AM 4774860 6910784 3107572 39.42 2688 2305000 826532 9.25 277368 2455920 4
01:18:27 01:17:01 AM 4831400 6972900 3051032 38.71 2688 2310560 737988 8.26 277700 2399736 4908
01:18:27 01:18:01 AM 4870192 6892480 3012240 38.21 2688 2196308 784368 8.78 519032 2138732 512
01:18:27 Average: 4788407 6911875 3094025 39.25 2688 2292628 834743 9.35 272580 2447539 135
01:18:27
01:18:27 12:00:01 AM IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s %ifutil
01:18:27 12:01:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:01:01 AM eth0 67.87 63.90 11.17 5.70 0.00 0.00 0.00 0.00
01:18:27 12:02:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:02:01 AM eth0 13.69 9.30 2.79 1.83 0.00 0.00 0.00 0.00
01:18:27 12:03:02 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:03:02 AM eth0 13.01 9.31 2.66 1.77 0.00 0.00 0.00 0.00
01:18:27 12:04:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:04:01 AM eth0 14.42 9.46 3.02 1.98 0.00 0.00 0.00 0.00
01:18:27 12:05:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:05:01 AM eth0 13.50 9.40 2.78 1.78 0.00 0.00 0.00 0.00
01:18:27 12:06:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:06:01 AM eth0 13.68 9.48 2.79 1.83 0.00 0.00 0.00 0.00
01:18:27 12:07:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:07:01 AM eth0 24.36 17.24 5.16 3.23 0.00 0.00 0.00 0.00
01:18:27 12:08:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:08:01 AM eth0 0.80 0.30 0.16 0.05 0.00 0.00 0.00 0.00
01:18:27 12:09:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:09:01 AM eth0 0.55 0.20 0.16 0.04 0.00 0.00 0.00 0.00
01:18:27 12:10:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:10:01 AM eth0 0.73 0.17 0.09 0.04 0.00 0.00 0.00 0.00
01:18:27 12:11:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:11:01 AM eth0 0.27 0.20 0.08 0.04 0.00 0.00 0.00 0.00
01:18:27 12:12:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:12:01 AM eth0 0.85 0.30 0.17 0.11 0.00 0.00 0.00 0.00
01:18:27 12:13:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:13:01 AM eth0 48.92 47.38 7.94 3.84 0.00 0.00 0.00 0.00
01:18:27 12:14:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:14:01 AM eth0 25.85 20.90 4.82 2.81 0.00 0.00 0.00 0.00
01:18:27 12:15:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:15:01 AM eth0 12.90 9.11 2.65 1.76 0.00 0.00 0.00 0.00
01:18:27 12:16:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:16:01 AM eth0 13.88 9.57 2.81 1.84 0.00 0.00 0.00 0.00
01:18:27 12:17:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:17:01 AM eth0 13.56 9.38 2.86 1.83 0.00 0.00 0.00 0.00
01:18:27 12:18:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:18:01 AM eth0 13.70 9.23 2.73 1.77 0.00 0.00 0.00 0.00
01:18:27 12:19:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:19:01 AM eth0 13.58 9.50 2.81 1.83 0.00 0.00 0.00 0.00
01:18:27 12:20:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:20:01 AM eth0 16.95 12.11 3.58 2.23 0.00 0.00 0.00 0.00
01:18:27 12:21:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:21:01 AM eth0 0.32 0.20 0.08 0.04 0.00 0.00 0.00 0.00
01:18:27 12:22:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:22:01 AM eth0 0.83 0.32 0.22 0.11 0.00 0.00 0.00 0.00
01:18:27 12:23:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:23:01 AM eth0 0.35 0.25 0.08 0.05 0.00 0.00 0.00 0.00
01:18:27 12:24:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:24:01 AM eth0 1.47 0.42 0.40 0.24 0.00 0.00 0.00 0.00
01:18:27 12:25:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:25:01 AM eth0 1.07 0.40 0.44 0.23 0.00 0.00 0.00 0.00
01:18:27 12:26:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:26:01 AM eth0 22.76 21.46 3.75 1.89 0.00 0.00 0.00 0.00
01:18:27 12:27:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:27:01 AM eth0 27.07 23.96 4.69 2.68 0.00 0.00 0.00 0.00
01:18:27 12:28:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:28:01 AM eth0 14.56 9.50 3.26 2.00 0.00 0.00 0.00 0.00
01:18:27 12:29:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:29:01 AM eth0 13.43 9.12 2.78 1.76 0.00 0.00 0.00 0.00
01:18:27 12:30:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:30:01 AM eth0 13.96 9.73 2.82 1.82 0.00 0.00 0.00 0.00
01:18:27 12:31:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:31:01 AM eth0 13.40 9.50 2.74 1.81 0.00 0.00 0.00 0.00
01:18:27 12:32:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:32:01 AM eth0 14.80 9.42 3.00 1.84 0.00 0.00 0.00 0.00
01:18:27 12:33:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:33:01 AM eth0 23.78 16.91 5.47 3.47 0.00 0.00 0.00 0.00
01:18:27 12:34:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:34:01 AM eth0 0.92 0.22 0.14 0.04 0.00 0.00 0.00 0.00
01:18:27 12:35:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:35:01 AM eth0 0.67 0.42 0.34 0.24 0.00 0.00 0.00 0.00
01:18:27 12:36:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:36:01 AM eth0 0.80 0.20 0.13 0.04 0.00 0.00 0.00 0.00
01:18:27 12:37:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:37:01 AM eth0 0.58 0.30 0.16 0.11 0.00 0.00 0.00 0.00
01:18:27 12:38:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:38:01 AM eth0 0.60 0.20 0.09 0.04 0.00 0.00 0.00 0.00
01:18:27 12:39:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:39:01 AM eth0 86.45 84.15 14.00 6.66 0.00 0.00 0.00 0.00
01:18:27 12:40:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:40:01 AM eth0 15.11 10.46 3.28 2.13 0.00 0.00 0.00 0.00
01:18:27 12:41:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:41:01 AM eth0 12.76 9.28 2.64 1.77 0.00 0.00 0.00 0.00
01:18:27 12:42:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:42:01 AM eth0 14.78 9.77 3.26 2.08 0.00 0.00 0.00 0.00
01:18:27 12:43:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:43:01 AM eth0 13.53 9.41 2.77 1.78 0.00 0.00 0.00 0.00
01:18:27 12:44:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:44:01 AM eth0 13.50 9.36 3.00 1.99 0.00 0.00 0.00 0.00
01:18:27 12:45:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:45:01 AM eth0 13.58 9.35 2.79 1.77 0.00 0.00 0.00 0.00
01:18:27 12:46:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:46:01 AM eth0 16.93 12.15 3.63 2.23 0.00 0.00 0.00 0.00
01:18:27 12:47:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:47:01 AM eth0 0.60 0.33 0.22 0.11 0.00 0.00 0.00 0.00
01:18:27 12:48:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:48:01 AM eth0 1.25 0.38 0.39 0.24 0.00 0.00 0.00 0.00
01:18:27 12:49:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:49:01 AM eth0 0.32 0.22 0.08 0.04 0.00 0.00 0.00 0.00
01:18:27 12:50:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:50:01 AM eth0 0.85 0.35 0.34 0.23 0.00 0.00 0.00 0.00
01:18:27 12:51:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:51:01 AM eth0 0.33 0.25 0.08 0.05 0.00 0.00 0.00 0.00
01:18:27 12:52:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:52:01 AM eth0 87.80 83.54 15.05 7.53 0.00 0.00 0.00 0.00
01:18:27 12:53:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:53:01 AM eth0 13.30 9.22 2.70 1.77 0.00 0.00 0.00 0.00
01:18:27 12:54:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:54:01 AM eth0 19.74 9.65 4.15 2.33 0.00 0.00 0.00 0.00
01:18:27 12:55:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:55:01 AM eth0 15.11 10.11 3.64 2.43 0.00 0.00 0.00 0.00
01:18:27 12:56:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:56:01 AM eth0 16.01 9.45 3.32 1.99 0.00 0.00 0.00 0.00
01:18:27 12:57:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:57:01 AM eth0 14.51 9.68 2.96 1.85 0.00 0.00 0.00 0.00
01:18:27 12:58:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:58:01 AM eth0 23.61 16.35 4.83 3.02 0.00 0.00 0.00 0.00
01:18:27 12:59:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 12:59:01 AM eth0 0.92 0.40 0.39 0.27 0.00 0.00 0.00 0.00
01:18:27 01:00:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 01:00:01 AM eth0 0.92 0.20 0.11 0.04 0.00 0.00 0.00 0.00
01:18:27 01:01:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 01:01:01 AM eth0 1.62 0.27 0.20 0.07 0.00 0.00 0.00 0.00
01:18:27 01:02:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 01:02:01 AM eth0 4.61 0.45 0.83 0.30 0.00 0.00 0.00 0.00
01:18:27 01:03:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 01:03:01 AM eth0 1.10 0.20 0.14 0.04 0.00 0.00 0.00 0.00
01:18:27 01:04:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 01:04:01 AM eth0 66.57 61.27 10.53 4.78 0.00 0.00 0.00 0.00
01:18:27 01:05:02 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 01:05:02 AM eth0 39.69 34.49 7.17 4.01 0.00 0.00 0.00 0.00
01:18:27 01:06:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 01:06:01 AM eth0 13.79 9.15 2.76 1.78 0.00 0.00 0.00 0.00
01:18:27 01:07:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 01:07:01 AM eth0 13.80 9.45 2.89 1.84 0.00 0.00 0.00 0.00
01:18:27 01:08:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 01:08:01 AM eth0 13.61 9.18 2.67 1.76 0.00 0.00 0.00 0.00
01:18:27 01:09:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 01:09:01 AM eth0 13.96 9.28 2.75 1.77 0.00 0.00 0.00 0.00
01:18:27 01:10:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 01:10:01 AM eth0 16.99 9.76 3.36 2.21 0.00 0.00 0.00 0.00
01:18:27 01:11:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 01:11:01 AM eth0 21.81 12.71 3.94 2.31 0.00 0.00 0.00 0.00
01:18:27 01:12:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 01:12:01 AM eth0 2.78 0.45 0.65 0.30 0.00 0.00 0.00 0.00
01:18:27 01:13:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 01:13:01 AM eth0 0.60 0.32 0.33 0.23 0.00 0.00 0.00 0.00
01:18:27 01:14:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 01:14:01 AM eth0 2.98 0.23 0.40 0.05 0.00 0.00 0.00 0.00
01:18:27 01:15:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 01:15:01 AM eth0 0.50 0.22 0.08 0.04 0.00 0.00 0.00 0.00
01:18:27 01:16:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 01:16:01 AM eth0 1.03 0.20 0.14 0.04 0.00 0.00 0.00 0.00
01:18:27 01:17:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 01:17:01 AM eth0 120.88 93.32 27.97 10.79 0.00 0.00 0.00 0.00
01:18:27 01:18:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 01:18:01 AM eth0 92.05 66.09 193.78 225.24 0.00 0.00 0.00 0.00
01:18:27 Average: lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
01:18:27 Average: eth0 16.47 12.69 5.46 4.55 0.00 0.00 0.00 0.00
01:18:27
01:18:27
01:18:27 ---> sar -P ALL:
01:18:27 Linux 4.18.0-553.5.1.el8.x86_64 (prd-centos8-robot-2c-8g-8068.novalocal) 03/17/2025 _x86_64_ (2 CPU)
01:18:27
01:18:27 12:00:01 AM CPU %user %nice %system %iowait %steal %idle
01:18:27 12:01:01 AM all 5.80 0.02 0.73 0.02 0.05 93.39
01:18:27 12:01:01 AM 0 1.86 0.00 0.44 0.03 0.05 97.62
01:18:27 12:01:01 AM 1 9.76 0.03 1.03 0.00 0.05 89.13
01:18:27 12:02:01 AM all 4.23 0.00 0.33 0.00 0.06 95.39
01:18:27 12:02:01 AM 0 2.22 0.00 0.18 0.00 0.05 97.54
01:18:27 12:02:01 AM 1 6.23 0.00 0.47 0.00 0.07 93.23
01:18:27 12:03:02 AM all 4.26 0.00 0.29 0.00 0.04 95.41
01:18:27 12:03:02 AM 0 3.11 0.00 0.23 0.00 0.03 96.62
01:18:27 12:03:02 AM 1 5.40 0.00 0.35 0.00 0.05 94.20
01:18:27 12:04:01 AM all 4.40 0.00 0.33 0.00 0.06 95.20
01:18:27 12:04:01 AM 0 2.01 0.00 0.24 0.00 0.05 97.70
01:18:27 12:04:01 AM 1 6.80 0.00 0.43 0.00 0.07 92.71
01:18:27 12:05:01 AM all 4.20 0.00 0.31 0.00 0.04 95.44
01:18:27 12:05:01 AM 0 4.73 0.00 0.35 0.00 0.05 94.87
01:18:27 12:05:01 AM 1 3.68 0.00 0.27 0.00 0.03 96.02
01:18:27 12:06:01 AM all 4.37 0.00 0.36 0.00 0.06 95.21
01:18:27 12:06:01 AM 0 5.12 0.00 0.37 0.00 0.05 94.46
01:18:27 12:06:01 AM 1 3.62 0.00 0.35 0.00 0.07 95.96
01:18:27 12:07:01 AM all 6.67 0.00 0.38 0.00 0.05 92.89
01:18:27 12:07:01 AM 0 5.42 0.00 0.32 0.00 0.05 94.21
01:18:27 12:07:01 AM 1 7.92 0.00 0.45 0.00 0.05 91.58
01:18:27 12:08:01 AM all 0.62 0.00 0.13 0.01 0.04 99.20
01:18:27 12:08:01 AM 0 0.49 0.00 0.15 0.02 0.05 99.30
01:18:27 12:08:01 AM 1 0.75 0.00 0.12 0.00 0.03 99.10
01:18:27 12:09:01 AM all 0.61 0.00 0.09 0.00 0.05 99.25
01:18:27 12:09:01 AM 0 0.57 0.00 0.15 0.00 0.07 99.21
01:18:27 12:09:01 AM 1 0.65 0.00 0.03 0.00 0.03 99.28
01:18:27 12:10:01 AM all 0.55 0.00 0.11 0.00 0.05 99.29
01:18:27 12:10:01 AM 0 0.45 0.00 0.10 0.00 0.05 99.40
01:18:27 12:10:01 AM 1 0.65 0.00 0.12 0.00 0.05 99.18
01:18:27 12:11:01 AM all 0.47 0.00 0.12 0.00 0.03 99.38
01:18:27 12:11:01 AM 0 0.49 0.00 0.12 0.00 0.05 99.35
01:18:27 12:11:01 AM 1 0.45 0.00 0.12 0.00 0.02 99.41
01:18:27
01:18:27 12:11:01 AM CPU %user %nice %system %iowait %steal %idle
01:18:27 12:12:01 AM all 0.45 0.00 0.11 0.01 0.05 99.38
01:18:27 12:12:01 AM 0 0.37 0.00 0.12 0.00 0.05 99.46
01:18:27 12:12:01 AM 1 0.54 0.00 0.10 0.02 0.05 99.30
01:18:27 12:13:01 AM all 3.39 0.00 0.44 0.00 0.04 96.13
01:18:27 12:13:01 AM 0 3.12 0.00 0.34 0.00 0.05 96.50
01:18:27 12:13:01 AM 1 3.67 0.00 0.54 0.00 0.03 95.76
01:18:27 12:14:01 AM all 5.07 0.00 0.37 0.00 0.07 94.50
01:18:27 12:14:01 AM 0 4.06 0.00 0.37 0.00 0.07 95.50
01:18:27 12:14:01 AM 1 6.07 0.00 0.37 0.00 0.07 93.49
01:18:27 12:15:01 AM all 4.29 0.00 0.33 0.00 0.05 95.34
01:18:27 12:15:01 AM 0 2.02 0.00 0.18 0.00 0.05 97.74
01:18:27 12:15:01 AM 1 6.55 0.00 0.47 0.00 0.05 92.93
01:18:27 12:16:01 AM all 4.36 0.01 0.35 0.00 0.06 95.22
01:18:27 12:16:01 AM 0 2.51 0.00 0.20 0.00 0.05 97.24
01:18:27 12:16:01 AM 1 6.22 0.02 0.50 0.00 0.07 93.19
01:18:27 12:17:01 AM all 4.16 0.00 0.33 0.00 0.06 95.45
01:18:27 12:17:01 AM 0 1.87 0.00 0.18 0.00 0.05 97.89
01:18:27 12:17:01 AM 1 6.45 0.00 0.47 0.00 0.07 93.02
01:18:27 12:18:01 AM all 4.27 0.00 0.32 0.00 0.05 95.36
01:18:27 12:18:01 AM 0 2.24 0.00 0.18 0.00 0.03 97.54
01:18:27 12:18:01 AM 1 6.30 0.00 0.45 0.00 0.07 93.18
01:18:27 12:19:01 AM all 4.39 0.00 0.33 0.00 0.04 95.24
01:18:27 12:19:01 AM 0 4.73 0.00 0.38 0.00 0.05 94.83
01:18:27 12:19:01 AM 1 4.04 0.00 0.27 0.00 0.03 95.66
01:18:27 12:20:01 AM all 4.71 0.00 0.25 0.00 0.05 94.98
01:18:27 12:20:01 AM 0 1.94 0.00 0.20 0.00 0.05 97.81
01:18:27 12:20:01 AM 1 7.49 0.00 0.30 0.00 0.05 92.16
01:18:27 12:21:01 AM all 0.57 0.00 0.11 0.00 0.05 99.27
01:18:27 12:21:01 AM 0 0.65 0.00 0.12 0.00 0.03 99.20
01:18:27 12:21:01 AM 1 0.48 0.00 0.10 0.00 0.07 99.35
01:18:27 12:22:01 AM all 0.47 0.00 0.09 0.00 0.04 99.40
01:18:27 12:22:01 AM 0 0.59 0.00 0.10 0.00 0.05 99.26
01:18:27 12:22:01 AM 1 0.35 0.00 0.08 0.00 0.03 99.53
01:18:27
01:18:27 12:22:01 AM CPU %user %nice %system %iowait %steal %idle
01:18:27 12:23:01 AM all 0.58 0.00 0.10 0.00 0.04 99.28
01:18:27 12:23:01 AM 0 0.58 0.00 0.07 0.00 0.03 99.31
01:18:27 12:23:01 AM 1 0.57 0.00 0.13 0.00 0.05 99.25
01:18:27 12:24:01 AM all 0.59 0.00 0.11 0.00 0.04 99.26
01:18:27 12:24:01 AM 0 0.60 0.00 0.07 0.00 0.02 99.32
01:18:27 12:24:01 AM 1 0.58 0.00 0.15 0.00 0.07 99.20
01:18:27 12:25:01 AM all 0.60 0.00 0.09 0.00 0.04 99.26
01:18:27 12:25:01 AM 0 0.75 0.00 0.08 0.00 0.05 99.11
01:18:27 12:25:01 AM 1 0.45 0.00 0.10 0.00 0.03 99.42
01:18:27 12:26:01 AM all 2.11 0.00 0.33 0.00 0.05 97.51
01:18:27 12:26:01 AM 0 0.80 0.00 0.20 0.00 0.05 98.95
01:18:27 12:26:01 AM 1 3.42 0.00 0.47 0.00 0.05 96.06
01:18:27 12:27:01 AM all 3.88 0.00 0.27 0.00 0.05 95.80
01:18:27 12:27:01 AM 0 3.28 0.00 0.18 0.00 0.05 96.48
01:18:27 12:27:01 AM 1 4.49 0.00 0.35 0.00 0.05 95.11
01:18:27 12:28:01 AM all 4.26 0.00 0.35 0.00 0.05 95.33
01:18:27 12:28:01 AM 0 3.20 0.00 0.27 0.00 0.05 96.49
01:18:27 12:28:01 AM 1 5.33 0.00 0.43 0.00 0.05 94.18
01:18:27 12:29:01 AM all 4.24 0.00 0.33 0.00 0.05 95.39
01:18:27 12:29:01 AM 0 3.18 0.00 0.28 0.00 0.05 96.49
01:18:27 12:29:01 AM 1 5.30 0.00 0.37 0.00 0.05 94.28
01:18:27 12:30:01 AM all 4.34 0.00 0.34 0.00 0.05 95.27
01:18:27 12:30:01 AM 0 3.66 0.00 0.35 0.00 0.05 95.94
01:18:27 12:30:01 AM 1 5.01 0.00 0.33 0.00 0.05 94.60
01:18:27 12:31:01 AM all 4.32 0.00 0.31 0.00 0.07 95.30
01:18:27 12:31:01 AM 0 1.79 0.00 0.18 0.00 0.05 97.98
01:18:27 12:31:01 AM 1 6.85 0.00 0.43 0.00 0.08 92.63
01:18:27 12:32:01 AM all 4.20 0.00 0.35 0.00 0.05 95.40
01:18:27 12:32:01 AM 0 3.63 0.00 0.33 0.00 0.05 95.99
01:18:27 12:32:01 AM 1 4.76 0.00 0.37 0.00 0.05 94.82
01:18:27 12:33:01 AM all 6.45 0.00 0.33 0.00 0.05 93.17
01:18:27 12:33:01 AM 0 9.97 0.00 0.47 0.00 0.07 89.50
01:18:27 12:33:01 AM 1 2.94 0.00 0.20 0.00 0.03 96.83
01:18:27
01:18:27 12:33:01 AM CPU %user %nice %system %iowait %steal %idle
01:18:27 12:34:01 AM all 0.58 0.00 0.10 0.00 0.04 99.27
01:18:27 12:34:01 AM 0 0.25 0.00 0.10 0.00 0.03 99.62
01:18:27 12:34:01 AM 1 0.92 0.00 0.10 0.00 0.05 98.93
01:18:27 12:35:01 AM all 0.65 0.00 0.09 0.00 0.05 99.21
01:18:27 12:35:01 AM 0 0.38 0.00 0.05 0.00 0.05 99.52
01:18:27 12:35:01 AM 1 0.92 0.00 0.13 0.00 0.05 98.90
01:18:27 12:36:01 AM all 0.50 0.01 0.11 0.00 0.04 99.34
01:18:27 12:36:01 AM 0 0.75 0.02 0.12 0.00 0.05 99.06
01:18:27 12:36:01 AM 1 0.25 0.00 0.10 0.00 0.03 99.62
01:18:27 12:37:01 AM all 0.55 0.00 0.11 0.00 0.05 99.29
01:18:27 12:37:01 AM 0 0.64 0.00 0.08 0.00 0.03 99.25
01:18:27 12:37:01 AM 1 0.47 0.00 0.13 0.00 0.07 99.33
01:18:27 12:38:01 AM all 0.60 0.00 0.08 0.00 0.04 99.27
01:18:27 12:38:01 AM 0 0.67 0.00 0.05 0.00 0.03 99.25
01:18:27 12:38:01 AM 1 0.53 0.00 0.12 0.00 0.05 99.30
01:18:27 12:39:01 AM all 4.45 0.00 0.52 0.00 0.05 94.98
01:18:27 12:39:01 AM 0 5.06 0.00 0.55 0.00 0.03 94.35
01:18:27 12:39:01 AM 1 3.84 0.00 0.49 0.00 0.07 95.60
01:18:27 12:40:01 AM all 4.57 0.00 0.34 0.00 0.05 95.04
01:18:27 12:40:01 AM 0 4.92 0.00 0.37 0.00 0.05 94.66
01:18:27 12:40:01 AM 1 4.21 0.00 0.32 0.00 0.05 95.42
01:18:27 12:41:01 AM all 4.24 0.00 0.34 0.01 0.05 95.36
01:18:27 12:41:01 AM 0 3.11 0.00 0.25 0.00 0.05 96.59
01:18:27 12:41:01 AM 1 5.36 0.00 0.43 0.02 0.05 94.14
01:18:27 12:42:01 AM all 4.34 0.00 0.32 0.01 0.05 95.28
01:18:27 12:42:01 AM 0 5.04 0.00 0.37 0.02 0.05 94.53
01:18:27 12:42:01 AM 1 3.64 0.00 0.27 0.00 0.05 96.04
01:18:27 12:43:01 AM all 4.40 0.00 0.38 0.00 0.05 95.18
01:18:27 12:43:01 AM 0 4.37 0.00 0.37 0.00 0.05 95.22
01:18:27 12:43:01 AM 1 4.43 0.00 0.38 0.00 0.05 95.14
01:18:27 12:44:01 AM all 4.45 0.00 0.33 0.00 0.05 95.18
01:18:27 12:44:01 AM 0 3.24 0.00 0.25 0.00 0.03 96.47
01:18:27 12:44:01 AM 1 5.65 0.00 0.40 0.00 0.07 93.88
01:18:27
01:18:27 12:44:01 AM CPU %user %nice %system %iowait %steal %idle
01:18:27 12:45:01 AM all 4.40 0.00 0.33 0.00 0.06 95.22
01:18:27 12:45:01 AM 0 2.24 0.00 0.20 0.00 0.05 97.51
01:18:27 12:45:01 AM 1 6.55 0.00 0.45 0.00 0.07 92.93
01:18:27 12:46:01 AM all 4.81 0.00 0.26 0.00 0.05 94.88
01:18:27 12:46:01 AM 0 3.12 0.00 0.23 0.00 0.05 96.59
01:18:27 12:46:01 AM 1 6.49 0.00 0.28 0.00 0.05 93.18
01:18:27 12:47:01 AM all 0.59 0.00 0.09 0.00 0.04 99.27
01:18:27 12:47:01 AM 0 0.64 0.00 0.08 0.00 0.03 99.25
01:18:27 12:47:01 AM 1 0.55 0.00 0.10 0.00 0.05 99.30
01:18:27 12:48:01 AM all 0.61 0.00 0.09 0.00 0.05 99.25
01:18:27 12:48:01 AM 0 0.62 0.00 0.12 0.00 0.05 99.21
01:18:27 12:48:01 AM 1 0.60 0.00 0.07 0.00 0.05 99.28
01:18:27 12:49:01 AM all 0.58 0.00 0.08 0.00 0.04 99.29
01:18:27 12:49:01 AM 0 0.54 0.00 0.08 0.00 0.05 99.33
01:18:27 12:49:01 AM 1 0.63 0.00 0.08 0.00 0.03 99.25
01:18:27 12:50:01 AM all 0.63 0.00 0.09 0.00 0.04 99.24
01:18:27 12:50:01 AM 0 0.52 0.00 0.12 0.00 0.05 99.31
01:18:27 12:50:01 AM 1 0.73 0.00 0.07 0.00 0.03 99.17
01:18:27 12:51:01 AM all 0.58 0.00 0.11 0.00 0.03 99.27
01:18:27 12:51:01 AM 0 0.64 0.00 0.17 0.00 0.05 99.15
01:18:27 12:51:01 AM 1 0.53 0.00 0.05 0.00 0.02 99.40
01:18:27 12:52:01 AM all 6.61 0.00 0.69 0.00 0.05 92.65
01:18:27 12:52:01 AM 0 6.02 0.00 0.74 0.00 0.05 93.19
01:18:27 12:52:01 AM 1 7.20 0.00 0.64 0.00 0.05 92.11
01:18:27 12:53:01 AM all 4.51 0.00 0.35 0.00 0.06 95.08
01:18:27 12:53:01 AM 0 3.26 0.00 0.27 0.00 0.05 96.42
01:18:27 12:53:01 AM 1 5.76 0.00 0.43 0.00 0.07 93.73
01:18:27 12:54:01 AM all 4.38 0.00 0.34 0.00 0.06 95.22
01:18:27 12:54:01 AM 0 1.87 0.00 0.22 0.00 0.05 97.86
01:18:27 12:54:01 AM 1 6.89 0.00 0.47 0.00 0.07 92.58
01:18:27 12:55:01 AM all 4.45 0.00 0.37 0.00 0.05 95.13
01:18:27 12:55:01 AM 0 3.29 0.00 0.27 0.00 0.05 96.39
01:18:27 12:55:01 AM 1 5.60 0.00 0.47 0.00 0.05 93.88
01:18:27
01:18:27 12:55:01 AM CPU %user %nice %system %iowait %steal %idle
01:18:27 12:56:01 AM all 4.38 0.00 0.33 0.00 0.06 95.23
01:18:27 12:56:01 AM 0 2.81 0.00 0.23 0.00 0.05 96.91
01:18:27 12:56:01 AM 1 5.95 0.00 0.42 0.00 0.07 93.56
01:18:27 12:57:01 AM all 4.34 0.00 0.32 0.00 0.05 95.29
01:18:27 12:57:01 AM 0 2.29 0.00 0.20 0.00 0.03 97.47
01:18:27 12:57:01 AM 1 6.38 0.00 0.43 0.00 0.07 93.11
01:18:27 12:58:01 AM all 6.37 0.00 0.36 0.00 0.06 93.21
01:18:27 12:58:01 AM 0 3.46 0.00 0.30 0.00 0.07 96.17
01:18:27 12:58:01 AM 1 9.28 0.00 0.42 0.00 0.05 90.25
01:18:27 12:59:01 AM all 0.63 0.00 0.10 0.00 0.04 99.22
01:18:27 12:59:01 AM 0 0.60 0.00 0.13 0.00 0.05 99.21
01:18:27 12:59:01 AM 1 0.67 0.00 0.07 0.00 0.03 99.23
01:18:27 01:00:01 AM all 0.69 0.00 0.12 0.00 0.06 99.13
01:18:27 01:00:01 AM 0 0.54 0.00 0.13 0.00 0.07 99.26
01:18:27 01:00:01 AM 1 0.85 0.00 0.10 0.00 0.05 99.00
01:18:27 01:01:01 AM all 0.68 0.00 0.14 0.00 0.09 99.10
01:18:27 01:01:01 AM 0 0.70 0.00 0.14 0.00 0.09 99.08
01:18:27 01:01:01 AM 1 0.66 0.00 0.14 0.00 0.09 99.12
01:18:27 01:02:01 AM all 0.77 0.00 0.17 0.00 0.03 99.04
01:18:27 01:02:01 AM 0 0.47 0.00 0.20 0.00 0.03 99.30
01:18:27 01:02:01 AM 1 1.07 0.00 0.13 0.00 0.02 98.78
01:18:27 01:03:01 AM all 0.56 0.00 0.13 0.00 0.03 99.28
01:18:27 01:03:01 AM 0 0.65 0.00 0.13 0.00 0.03 99.18
01:18:27 01:03:01 AM 1 0.47 0.00 0.12 0.00 0.03 99.38
01:18:27 01:04:01 AM all 3.41 0.00 0.45 0.00 0.06 96.08
01:18:27 01:04:01 AM 0 2.46 0.00 0.27 0.00 0.05 97.22
01:18:27 01:04:01 AM 1 4.37 0.00 0.64 0.00 0.07 94.92
01:18:27 01:05:02 AM all 5.61 0.00 0.40 0.00 0.06 93.93
01:18:27 01:05:02 AM 0 2.88 0.00 0.30 0.00 0.07 96.75
01:18:27 01:05:02 AM 1 8.34 0.00 0.50 0.00 0.05 91.11
01:18:27 01:06:01 AM all 4.20 0.00 0.37 0.00 0.05 95.39
01:18:27 01:06:01 AM 0 2.75 0.00 0.27 0.00 0.05 96.92
01:18:27 01:06:01 AM 1 5.64 0.00 0.46 0.00 0.05 93.85
01:18:27
01:18:27 01:06:01 AM CPU %user %nice %system %iowait %steal %idle
01:18:27 01:07:01 AM all 4.30 0.16 0.38 0.00 0.04 95.13
01:18:27 01:07:01 AM 0 2.01 0.00 0.23 0.00 0.03 97.72
01:18:27 01:07:01 AM 1 6.58 0.32 0.52 0.00 0.05 92.53
01:18:27 01:08:01 AM all 4.32 0.00 0.33 0.00 0.05 95.30
01:18:27 01:08:01 AM 0 2.50 0.00 0.23 0.00 0.05 97.22
01:18:27 01:08:01 AM 1 6.14 0.00 0.42 0.00 0.05 93.40
01:18:27 01:09:01 AM all 4.37 0.00 0.32 0.00 0.06 95.25
01:18:27 01:09:01 AM 0 2.21 0.00 0.20 0.00 0.05 97.54
01:18:27 01:09:01 AM 1 6.54 0.00 0.43 0.00 0.07 92.96
01:18:27 01:10:01 AM all 4.26 0.00 0.38 0.00 0.05 95.30
01:18:27 01:10:01 AM 0 2.73 0.00 0.27 0.00 0.05 96.96
01:18:27 01:10:01 AM 1 5.80 0.00 0.50 0.00 0.05 93.65
01:18:27 01:11:01 AM all 4.95 0.00 0.28 0.00 0.06 94.72
01:18:27 01:11:01 AM 0 3.85 0.00 0.25 0.00 0.07 95.83
01:18:27 01:11:01 AM 1 6.05 0.00 0.30 0.00 0.05 93.60
01:18:27 01:12:01 AM all 0.71 0.00 0.11 0.00 0.04 99.14
01:18:27 01:12:01 AM 0 0.67 0.00 0.15 0.00 0.05 99.13
01:18:27 01:12:01 AM 1 0.75 0.00 0.07 0.00 0.03 99.15
01:18:27 01:13:01 AM all 0.64 0.00 0.11 0.00 0.04 99.21
01:18:27 01:13:01 AM 0 0.45 0.00 0.12 0.00 0.05 99.38
01:18:27 01:13:01 AM 1 0.83 0.00 0.10 0.00 0.03 99.03
01:18:27 01:14:01 AM all 0.65 0.00 0.12 0.00 0.06 99.17
01:18:27 01:14:01 AM 0 0.59 0.00 0.15 0.00 0.07 99.20
01:18:27 01:14:01 AM 1 0.72 0.00 0.08 0.00 0.05 99.15
01:18:27 01:15:01 AM all 0.46 0.00 0.11 0.00 0.03 99.40
01:18:27 01:15:01 AM 0 0.48 0.00 0.13 0.00 0.03 99.35
01:18:27 01:15:01 AM 1 0.43 0.00 0.08 0.00 0.03 99.45
01:18:27 01:16:01 AM all 0.66 0.00 0.11 0.00 0.05 99.18
01:18:27 01:16:01 AM 0 0.62 0.00 0.13 0.00 0.05 99.20
01:18:27 01:16:01 AM 1 0.70 0.00 0.08 0.00 0.05 99.17
01:18:27 01:17:01 AM all 16.23 0.00 2.25 0.01 0.07 81.45
01:18:27 01:17:01 AM 0 22.05 0.00 2.30 0.02 0.07 75.56
01:18:27 01:17:01 AM 1 10.41 0.00 2.20 0.00 0.07 87.32
01:18:27
01:18:27 01:17:01 AM CPU %user %nice %system %iowait %steal %idle
01:18:27 01:18:01 AM all 25.02 0.00 3.49 0.16 0.08 71.26
01:18:27 01:18:01 AM 0 19.35 0.00 3.56 0.08 0.07 76.94
01:18:27 01:18:01 AM 1 30.67 0.00 3.42 0.23 0.08 65.59
01:18:27 Average: all 3.43 0.00 0.33 0.00 0.05 96.19
01:18:27 Average: 0 2.70 0.00 0.29 0.00 0.05 96.96
01:18:27 Average: 1 4.16 0.00 0.36 0.00 0.05 95.41
01:18:27
01:18:27
01:18:27