04:45:05  Started by upstream project "integration-distribution-test-calcium" build number 659
04:45:05  originally caused by:
04:45:05   Started by upstream project "autorelease-release-calcium-mvn38-openjdk17" build number 560
04:45:05   originally caused by:
04:45:05    Started by timer
04:45:05  Running as SYSTEM
04:45:05  [EnvInject] - Loading node environment variables.
04:45:05  Building remotely on prd-centos8-robot-2c-8g-5694 (centos8-robot-2c-8g) in workspace /w/workspace/openflowplugin-csit-3node-clustering-only-calcium
04:45:06  [ssh-agent] Looking for ssh-agent implementation...
04:45:06  [ssh-agent]   Exec ssh-agent (binary ssh-agent on a remote machine)
04:45:06  $ ssh-agent
04:45:06  SSH_AUTH_SOCK=/tmp/ssh-9rUHCfrgY9hU/agent.5281
04:45:06  SSH_AGENT_PID=5282
04:45:06  [ssh-agent] Started.
04:45:06  Running ssh-add (command line suppressed)
04:45:06  Identity added: /w/workspace/openflowplugin-csit-3node-clustering-only-calcium@tmp/private_key_3935194982984691047.key (/w/workspace/openflowplugin-csit-3node-clustering-only-calcium@tmp/private_key_3935194982984691047.key)
04:45:06  [ssh-agent] Using credentials jenkins (Release Engineering Jenkins Key)
04:45:06  The recommended git tool is: NONE
04:45:09  using credential opendaylight-jenkins-ssh
04:45:09  Wiping out workspace first.
04:45:09  Cloning the remote Git repository
04:45:09  Cloning repository git://devvexx.opendaylight.org/mirror/integration/test
04:45:09   > git init /w/workspace/openflowplugin-csit-3node-clustering-only-calcium/test # timeout=10
04:45:09  Fetching upstream changes from git://devvexx.opendaylight.org/mirror/integration/test
04:45:09   > git --version # timeout=10
04:45:09   > git --version # 'git version 2.43.0'
04:45:09  using GIT_SSH to set credentials Release Engineering Jenkins Key
04:45:09  [INFO] Currently running in a labeled security context
04:45:09  [INFO] Currently SELinux is 'enforcing' on the host
04:45:09   > /usr/bin/chcon --type=ssh_home_t /w/workspace/openflowplugin-csit-3node-clustering-only-calcium/test@tmp/jenkins-gitclient-ssh4142936272825172176.key
04:45:09  Verifying host key using known hosts file
04:45:09  You're using 'Known hosts file' strategy to verify ssh host keys, but your known_hosts file does not exist, please go to 'Manage Jenkins' -> 'Security' -> 'Git Host Key Verification Configuration' and configure host key verification.
04:45:09   > git fetch --tags --force --progress -- git://devvexx.opendaylight.org/mirror/integration/test +refs/heads/*:refs/remotes/origin/* # timeout=10
04:45:11   > git config remote.origin.url git://devvexx.opendaylight.org/mirror/integration/test # timeout=10
04:45:11   > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
04:45:12   > git config remote.origin.url git://devvexx.opendaylight.org/mirror/integration/test # timeout=10
04:45:12  Fetching upstream changes from git://devvexx.opendaylight.org/mirror/integration/test
04:45:12  using GIT_SSH to set credentials Release Engineering Jenkins Key
04:45:12  [INFO] Currently running in a labeled security context
04:45:12  [INFO] Currently SELinux is 'enforcing' on the host
04:45:12   > /usr/bin/chcon --type=ssh_home_t /w/workspace/openflowplugin-csit-3node-clustering-only-calcium/test@tmp/jenkins-gitclient-ssh230939110950413396.key
04:45:12  Verifying host key using known hosts file
04:45:12  You're using 'Known hosts file' strategy to verify ssh host keys, but your known_hosts file does not exist, please go to 'Manage Jenkins' -> 'Security' -> 'Git Host Key Verification Configuration' and configure host key verification.
04:45:12   > git fetch --tags --force --progress -- git://devvexx.opendaylight.org/mirror/integration/test master # timeout=10
04:45:12   > git rev-parse FETCH_HEAD^{commit} # timeout=10
04:45:12  Checking out Revision 1d2a1a30bcc48e3c8d571eb0d9b6891fe29dfed4 (origin/master)
04:45:12   > git config core.sparsecheckout # timeout=10
04:45:12   > git checkout -f 1d2a1a30bcc48e3c8d571eb0d9b6891fe29dfed4 # timeout=10
04:45:12  Commit message: "Add bgpcep variables for titanium"
04:45:12   > git rev-parse FETCH_HEAD^{commit} # timeout=10
04:45:12   > git rev-list --no-walk d42b2c24fe4fcc7fe3ae0d352311e768f07a15b7 # timeout=10
04:45:13  No emails were triggered.
04:45:13  provisioning config files...
04:45:13  copy managed file [npmrc] to file:/home/jenkins/.npmrc
04:45:13  copy managed file [pipconf] to file:/home/jenkins/.config/pip/pip.conf
04:45:13  copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
04:45:13  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins16814928985855028552.sh
04:45:13  ---> python-tools-install.sh
04:45:13  Setup pyenv:
04:45:13    system
04:45:13  * 3.8.13 (set by /opt/pyenv/version)
04:45:13  * 3.9.13 (set by /opt/pyenv/version)
04:45:13  * 3.10.13 (set by /opt/pyenv/version)
04:45:13  * 3.11.7 (set by /opt/pyenv/version)
04:45:18  lf-activate-venv(): INFO: Creating python3 venv at /tmp/venv-S6z6
04:45:18  lf-activate-venv(): INFO: Save venv in file: /tmp/.os_lf_venv
04:45:21  lf-activate-venv(): INFO: Installing: lftools 
04:45:44  lf-activate-venv(): INFO: Adding /tmp/venv-S6z6/bin to PATH
04:45:44  Generating Requirements File
04:46:06  Python 3.11.7
04:46:06  pip 25.0.1 from /tmp/venv-S6z6/lib/python3.11/site-packages/pip (python 3.11)
04:46:06  appdirs==1.4.4
04:46:06  argcomplete==3.6.0
04:46:06  aspy.yaml==1.3.0
04:46:06  attrs==25.1.0
04:46:06  autopage==0.5.2
04:46:06  beautifulsoup4==4.13.3
04:46:06  boto3==1.37.9
04:46:06  botocore==1.37.9
04:46:06  bs4==0.0.2
04:46:06  cachetools==5.5.2
04:46:06  certifi==2025.1.31
04:46:06  cffi==1.17.1
04:46:06  cfgv==3.4.0
04:46:06  chardet==5.2.0
04:46:06  charset-normalizer==3.4.1
04:46:06  click==8.1.8
04:46:06  cliff==4.9.1
04:46:06  cmd2==2.5.11
04:46:06  cryptography==3.3.2
04:46:06  debtcollector==3.0.0
04:46:06  decorator==5.2.1
04:46:06  defusedxml==0.7.1
04:46:06  Deprecated==1.2.18
04:46:06  distlib==0.3.9
04:46:06  dnspython==2.7.0
04:46:06  docker==4.2.2
04:46:06  dogpile.cache==1.3.4
04:46:06  durationpy==0.9
04:46:06  email_validator==2.2.0
04:46:06  filelock==3.17.0
04:46:06  future==1.0.0
04:46:06  gitdb==4.0.12
04:46:06  GitPython==3.1.44
04:46:06  google-auth==2.38.0
04:46:06  httplib2==0.22.0
04:46:06  identify==2.6.8
04:46:06  idna==3.10
04:46:06  importlib-resources==1.5.0
04:46:06  iso8601==2.1.0
04:46:06  Jinja2==3.1.6
04:46:06  jmespath==1.0.1
04:46:06  jsonpatch==1.33
04:46:06  jsonpointer==3.0.0
04:46:06  jsonschema==4.23.0
04:46:06  jsonschema-specifications==2024.10.1
04:46:06  keystoneauth1==5.10.0
04:46:06  kubernetes==32.0.1
04:46:06  lftools==0.37.10
04:46:06  lxml==5.3.1
04:46:06  MarkupSafe==3.0.2
04:46:06  msgpack==1.1.0
04:46:06  multi_key_dict==2.0.3
04:46:06  munch==4.0.0
04:46:06  netaddr==1.3.0
04:46:06  niet==1.4.2
04:46:06  nodeenv==1.9.1
04:46:06  oauth2client==4.1.3
04:46:06  oauthlib==3.2.2
04:46:06  openstacksdk==4.4.0
04:46:06  os-client-config==2.1.0
04:46:06  os-service-types==1.7.0
04:46:06  osc-lib==3.2.0
04:46:06  oslo.config==9.7.1
04:46:06  oslo.context==5.7.1
04:46:06  oslo.i18n==6.5.1
04:46:06  oslo.log==7.1.0
04:46:06  oslo.serialization==5.7.0
04:46:06  oslo.utils==8.2.0
04:46:06  packaging==24.2
04:46:06  pbr==6.1.1
04:46:06  platformdirs==4.3.6
04:46:06  prettytable==3.15.1
04:46:06  psutil==7.0.0
04:46:06  pyasn1==0.6.1
04:46:06  pyasn1_modules==0.4.1
04:46:06  pycparser==2.22
04:46:06  pygerrit2==2.0.15
04:46:06  PyGithub==2.6.1
04:46:06  PyJWT==2.10.1
04:46:06  PyNaCl==1.5.0
04:46:06  pyparsing==2.4.7
04:46:06  pyperclip==1.9.0
04:46:06  pyrsistent==0.20.0
04:46:06  python-cinderclient==9.7.0
04:46:06  python-dateutil==2.9.0.post0
04:46:06  python-heatclient==4.1.0
04:46:06  python-jenkins==1.8.2
04:46:06  python-keystoneclient==5.6.0
04:46:06  python-magnumclient==4.8.1
04:46:06  python-openstackclient==7.4.0
04:46:06  python-swiftclient==4.7.0
04:46:06  PyYAML==6.0.2
04:46:06  referencing==0.36.2
04:46:06  requests==2.32.3
04:46:06  requests-oauthlib==2.0.0
04:46:06  requestsexceptions==1.4.0
04:46:06  rfc3986==2.0.0
04:46:06  rpds-py==0.23.1
04:46:06  rsa==4.9
04:46:06  ruamel.yaml==0.18.10
04:46:06  ruamel.yaml.clib==0.2.12
04:46:06  s3transfer==0.11.4
04:46:06  simplejson==3.20.1
04:46:06  six==1.17.0
04:46:06  smmap==5.0.2
04:46:06  soupsieve==2.6
04:46:06  stevedore==5.4.1
04:46:06  tabulate==0.9.0
04:46:06  toml==0.10.2
04:46:06  tomlkit==0.13.2
04:46:06  tqdm==4.67.1
04:46:06  typing_extensions==4.12.2
04:46:06  tzdata==2025.1
04:46:06  urllib3==1.26.20
04:46:06  virtualenv==20.29.3
04:46:06  wcwidth==0.2.13
04:46:06  websocket-client==1.8.0
04:46:06  wrapt==1.17.2
04:46:06  xdg==6.0.0
04:46:06  xmltodict==0.14.2
04:46:06  yq==3.4.3
04:46:06  [EnvInject] - Injecting environment variables from a build step.
04:46:06  [EnvInject] - Injecting as environment variables the properties content 
04:46:06  OS_STACK_TEMPLATE=csit-2-instance-type.yaml
04:46:06  OS_CLOUD=vex
04:46:06  OS_STACK_NAME=releng-openflowplugin-csit-3node-clustering-only-calcium-658
04:46:06  OS_STACK_TEMPLATE_DIR=openstack-hot
04:46:06  
04:46:06  [EnvInject] - Variables injected successfully.
04:46:06  provisioning config files...
04:46:07  copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
04:46:07  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins11214546315013381777.sh
04:46:07  ---> Create parameters file for OpenStack HOT
04:46:07  OpenStack Heat parameters generated
04:46:07  -----------------------------------
04:46:07  parameters:
04:46:07      vm_0_count: '3'
04:46:07      vm_0_flavor: 'v3-standard-4'
04:46:07      vm_0_image: 'ZZCI - Ubuntu 20.04 - builder - x86_64 - 20250101-160150.998'
04:46:07      vm_1_count: '1'
04:46:07      vm_1_flavor: 'v3-standard-2'
04:46:07      vm_1_image: 'ZZCI - Ubuntu 22.04 - mininet-ovs-217 - x86_64 - 20250201-060151.911'
04:46:07      
04:46:07      job_name: '57946-658'
04:46:07      silo: 'releng'
04:46:07  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash -l /tmp/jenkins6923692915941698412.sh
04:46:07  ---> Create HEAT stack
04:46:07  + source /home/jenkins/lf-env.sh
04:46:07  + lf-activate-venv --python python3 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
04:46:07  ++ mktemp -d /tmp/venv-XXXX
04:46:07  + lf_venv=/tmp/venv-l2cF
04:46:07  + local venv_file=/tmp/.os_lf_venv
04:46:07  + local python=python3
04:46:07  + local options
04:46:07  + local set_path=true
04:46:07  + local install_args=
04:46:07  ++ getopt -o np:v: -l no-path,system-site-packages,python:,venv-file: -n lf-activate-venv -- --python python3 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
04:46:07  + options=' --python '\''python3'\'' -- '\''lftools[openstack]'\'' '\''kubernetes'\'' '\''niet'\'' '\''python-heatclient'\'' '\''python-openstackclient'\'' '\''python-magnumclient'\'' '\''yq'\'''
04:46:07  + eval set -- ' --python '\''python3'\'' -- '\''lftools[openstack]'\'' '\''kubernetes'\'' '\''niet'\'' '\''python-heatclient'\'' '\''python-openstackclient'\'' '\''python-magnumclient'\'' '\''yq'\'''
04:46:07  ++ set -- --python python3 -- 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
04:46:07  + true
04:46:07  + case $1 in
04:46:07  + python=python3
04:46:07  + shift 2
04:46:07  + true
04:46:07  + case $1 in
04:46:07  + shift
04:46:07  + break
04:46:07  + case $python in
04:46:07  + local pkg_list=
04:46:07  + [[ -d /opt/pyenv ]]
04:46:07  + echo 'Setup pyenv:'
04:46:07  Setup pyenv:
04:46:07  + export PYENV_ROOT=/opt/pyenv
04:46:07  + PYENV_ROOT=/opt/pyenv
04:46:07  + export PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
04:46:07  + PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
04:46:07  + pyenv versions
04:46:07    system
04:46:07    3.8.13
04:46:07    3.9.13
04:46:07    3.10.13
04:46:07  * 3.11.7 (set by /w/workspace/openflowplugin-csit-3node-clustering-only-calcium/.python-version)
04:46:07  + command -v pyenv
04:46:07  ++ pyenv init - --no-rehash
04:46:07  + eval 'PATH="$(bash --norc -ec '\''IFS=:; paths=($PATH); 
04:46:07  for i in ${!paths[@]}; do 
04:46:07  if [[ ${paths[i]} == "'\'''\''/opt/pyenv/shims'\'''\''" ]]; then unset '\''\'\'''\''paths[i]'\''\'\'''\''; 
04:46:07  fi; done; 
04:46:07  echo "${paths[*]}"'\'')"
04:46:07  export PATH="/opt/pyenv/shims:${PATH}"
04:46:07  export PYENV_SHELL=bash
04:46:07  source '\''/opt/pyenv/libexec/../completions/pyenv.bash'\''
04:46:07  pyenv() {
04:46:07    local command
04:46:07    command="${1:-}"
04:46:07    if [ "$#" -gt 0 ]; then
04:46:07      shift
04:46:07    fi
04:46:07  
04:46:07    case "$command" in
04:46:07    rehash|shell)
04:46:07      eval "$(pyenv "sh-$command" "$@")"
04:46:07      ;;
04:46:07    *)
04:46:07      command pyenv "$command" "$@"
04:46:07      ;;
04:46:07    esac
04:46:07  }'
04:46:07  +++ bash --norc -ec 'IFS=:; paths=($PATH); 
04:46:07  for i in ${!paths[@]}; do 
04:46:07  if [[ ${paths[i]} == "/opt/pyenv/shims" ]]; then unset '\''paths[i]'\''; 
04:46:07  fi; done; 
04:46:07  echo "${paths[*]}"'
04:46:07  ++ PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
04:46:07  ++ export PATH=/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
04:46:07  ++ PATH=/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
04:46:07  ++ export PYENV_SHELL=bash
04:46:07  ++ PYENV_SHELL=bash
04:46:07  ++ source /opt/pyenv/libexec/../completions/pyenv.bash
04:46:07  +++ complete -F _pyenv pyenv
04:46:07  ++ lf-pyver python3
04:46:07  ++ local py_version_xy=python3
04:46:07  ++ local py_version_xyz=
04:46:07  ++ pyenv versions
04:46:07  ++ local command
04:46:07  ++ command=versions
04:46:07  ++ '[' 1 -gt 0 ']'
04:46:07  ++ shift
04:46:07  ++ case "$command" in
04:46:07  ++ command pyenv versions
04:46:07  ++ pyenv versions
04:46:07  ++ awk '{ print $1 }'
04:46:07  ++ sed 's/^[ *]* //'
04:46:07  ++ grep -E '^[0-9.]*[0-9]$'
04:46:07  ++ [[ ! -s /tmp/.pyenv_versions ]]
04:46:07  +++ grep '^3' /tmp/.pyenv_versions
04:46:07  +++ tail -n 1
04:46:07  +++ sort -V
04:46:07  ++ py_version_xyz=3.11.7
04:46:07  ++ [[ -z 3.11.7 ]]
04:46:07  ++ echo 3.11.7
04:46:07  ++ return 0
04:46:07  + pyenv local 3.11.7
04:46:07  + local command
04:46:07  + command=local
04:46:07  + '[' 2 -gt 0 ']'
04:46:07  + shift
04:46:07  + case "$command" in
04:46:07  + command pyenv local 3.11.7
04:46:07  + pyenv local 3.11.7
04:46:07  + for arg in "$@"
04:46:07  + case $arg in
04:46:07  + pkg_list+='lftools[openstack] '
04:46:07  + for arg in "$@"
04:46:07  + case $arg in
04:46:07  + pkg_list+='kubernetes '
04:46:07  + for arg in "$@"
04:46:07  + case $arg in
04:46:07  + pkg_list+='niet '
04:46:07  + for arg in "$@"
04:46:07  + case $arg in
04:46:07  + pkg_list+='python-heatclient '
04:46:07  + for arg in "$@"
04:46:07  + case $arg in
04:46:07  + pkg_list+='python-openstackclient '
04:46:07  + for arg in "$@"
04:46:07  + case $arg in
04:46:07  + pkg_list+='python-magnumclient '
04:46:07  + for arg in "$@"
04:46:07  + case $arg in
04:46:07  + pkg_list+='yq '
04:46:07  + [[ -f /tmp/.os_lf_venv ]]
04:46:07  ++ cat /tmp/.os_lf_venv
04:46:07  + lf_venv=/tmp/venv-S6z6
04:46:07  + echo 'lf-activate-venv(): INFO: Reuse venv:/tmp/venv-S6z6 from' file:/tmp/.os_lf_venv
04:46:07  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-S6z6 from file:/tmp/.os_lf_venv
04:46:07  + /tmp/venv-S6z6/bin/python3 -m pip install --upgrade --quiet pip virtualenv
04:46:08  + [[ -z lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq  ]]
04:46:08  + echo 'lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq '
04:46:08  lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq 
04:46:08  + /tmp/venv-S6z6/bin/python3 -m pip install --upgrade --quiet --upgrade-strategy eager 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
04:46:23  + type python3
04:46:23  + true
04:46:23  + echo 'lf-activate-venv(): INFO: Adding /tmp/venv-S6z6/bin to PATH'
04:46:23  lf-activate-venv(): INFO: Adding /tmp/venv-S6z6/bin to PATH
04:46:23  + PATH=/tmp/venv-S6z6/bin:/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
04:46:23  + return 0
04:46:23  + openstack --os-cloud vex limits show --absolute
04:46:25  +--------------------------+---------+
04:46:25  | Name                     |   Value |
04:46:25  +--------------------------+---------+
04:46:25  | maxTotalInstances        |      -1 |
04:46:25  | maxTotalCores            |     450 |
04:46:25  | maxTotalRAMSize          | 1000000 |
04:46:25  | maxServerMeta            |     128 |
04:46:25  | maxImageMeta             |     128 |
04:46:25  | maxPersonality           |       5 |
04:46:25  | maxPersonalitySize       |   10240 |
04:46:25  | maxTotalKeypairs         |     100 |
04:46:25  | maxServerGroups          |      10 |
04:46:25  | maxServerGroupMembers    |      10 |
04:46:25  | maxTotalFloatingIps      |      -1 |
04:46:25  | maxSecurityGroups        |      -1 |
04:46:25  | maxSecurityGroupRules    |      -1 |
04:46:25  | totalRAMUsed             |  851968 |
04:46:25  | totalCoresUsed           |     208 |
04:46:25  | totalInstancesUsed       |      69 |
04:46:25  | totalFloatingIpsUsed     |       0 |
04:46:25  | totalSecurityGroupsUsed  |       0 |
04:46:25  | totalServerGroupsUsed    |       0 |
04:46:25  | maxTotalVolumes          |      -1 |
04:46:25  | maxTotalSnapshots        |      10 |
04:46:25  | maxTotalVolumeGigabytes  |    4096 |
04:46:25  | maxTotalBackups          |      10 |
04:46:25  | maxTotalBackupGigabytes  |    1000 |
04:46:25  | totalVolumesUsed         |       3 |
04:46:25  | totalGigabytesUsed       |      60 |
04:46:25  | totalSnapshotsUsed       |       0 |
04:46:25  | totalBackupsUsed         |       0 |
04:46:25  | totalBackupGigabytesUsed |       0 |
04:46:25  +--------------------------+---------+
04:46:25  + pushd /opt/ciman/openstack-hot
04:46:25  /opt/ciman/openstack-hot /w/workspace/openflowplugin-csit-3node-clustering-only-calcium
04:46:25  + lftools openstack --os-cloud vex stack create releng-openflowplugin-csit-3node-clustering-only-calcium-658 csit-2-instance-type.yaml /w/workspace/openflowplugin-csit-3node-clustering-only-calcium/stack-parameters.yaml
04:46:40  Creating stack releng-openflowplugin-csit-3node-clustering-only-calcium-658
04:46:40  WARN: Failed to initialize stack. Reason: Resource CREATE failed: Forbidden: resources.vm_0_group.resources[0].resources.instance: Quota exceeded for ram: Requested 16384, but already used 999424 of 1000000 ram (HTTP 403) (Request-ID: req-910dfddf-b1c5-407b-b8bc-87091b92fd18)
04:46:40  TypeError: delete() missing 1 required positional argument: 'force'
04:46:40  Build step 'Execute shell' marked build as failure
04:46:40  $ ssh-agent -k
04:46:40  unset SSH_AUTH_SOCK;
04:46:40  unset SSH_AGENT_PID;
04:46:40  echo Agent pid 5282 killed;
04:46:40  [ssh-agent] Stopped.
04:46:41  Recording plot data
04:46:41  Robot results publisher started...
04:46:41  INFO: Checking test criticality is deprecated and will be dropped in a future release!
04:46:41  -Parsing output xml:
04:46:41  Failed!
04:46:41  hudson.AbortException: No files found in path /w/workspace/openflowplugin-csit-3node-clustering-only-calcium with configured filemask: output.xml
04:46:41  	at hudson.plugins.robot.RobotParser$RobotParserCallable.invoke(RobotParser.java:81)
04:46:41  	at hudson.plugins.robot.RobotParser$RobotParserCallable.invoke(RobotParser.java:52)
04:46:41  	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3616)
04:46:41  	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
04:46:41  	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
04:46:41  	at hudson.remoting.Request$2.run(Request.java:377)
04:46:41  	at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
04:46:41  	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
04:46:41  	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
04:46:41  	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
04:46:41  	at java.base/java.lang.Thread.run(Thread.java:829)
04:46:41  	Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to prd-centos8-robot-2c-8g-5694
04:46:41  		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1787)
04:46:41  		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
04:46:41  		at hudson.remoting.Channel.call(Channel.java:1003)
04:46:41  		at hudson.FilePath.act(FilePath.java:1230)
04:46:41  		at hudson.FilePath.act(FilePath.java:1219)
04:46:41  		at hudson.plugins.robot.RobotParser.parse(RobotParser.java:48)
04:46:41  		at hudson.plugins.robot.RobotPublisher.parse(RobotPublisher.java:246)
04:46:41  		at hudson.plugins.robot.RobotPublisher.perform(RobotPublisher.java:268)
04:46:41  		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:80)
04:46:41  		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
04:46:41  		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:818)
04:46:41  		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:767)
04:46:41  		at hudson.model.Build$BuildExecution.post2(Build.java:179)
04:46:41  		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:711)
04:46:41  		at hudson.model.Run.execute(Run.java:1918)
04:46:41  		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:44)
04:46:41  		at hudson.model.ResourceController.execute(ResourceController.java:101)
04:46:41  		at hudson.model.Executor.run(Executor.java:442)
04:46:41  [PostBuildScript] - [INFO] Executing post build scripts.
04:46:41  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins14083009072765001469.sh
04:46:41  Archiving csit artifacts
04:46:41  seq: invalid floating point argument: ‘’
04:46:41  Try 'seq --help' for more information.
04:46:41    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
04:46:41                                   Dload  Upload   Total   Spent    Left  Speed
04:46:41  
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   237    0   237    0     0   1559      0 --:--:-- --:--:-- --:--:--  1559
04:46:41  Archive:  robot-plugin.zip
04:46:41    End-of-central-directory signature not found.  Either this file is not
04:46:41    a zipfile, or it constitutes one disk of a multi-part archive.  In the
04:46:41    latter case the central directory and zipfile comment will be found on
04:46:41    the last disk(s) of this archive.
04:46:41  unzip:  cannot find zipfile directory in one of robot-plugin.zip or
04:46:41          robot-plugin.zip.zip, and cannot find robot-plugin.zip.ZIP, period.
04:46:41  mv: cannot stat '*.log': No such file or directory
04:46:41  mv: cannot stat '*.log.gz': No such file or directory
04:46:41  mv: cannot stat '*.csv': No such file or directory
04:46:41  mv: cannot stat '*.png': No such file or directory
04:46:41  [PostBuildScript] - [INFO] Executing post build scripts.
04:46:41  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins16223937236247036125.sh
04:46:41  [PostBuildScript] - [INFO] Executing post build scripts.
04:46:41  [EnvInject] - Injecting environment variables from a build step.
04:46:41  [EnvInject] - Injecting as environment variables the properties content 
04:46:41  OS_CLOUD=vex
04:46:41  OS_STACK_NAME=releng-openflowplugin-csit-3node-clustering-only-calcium-658
04:46:41  
04:46:41  [EnvInject] - Variables injected successfully.
04:46:41  provisioning config files...
04:46:41  copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
04:46:41  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins9969593394819329871.sh
04:46:41  ---> openstack-stack-delete.sh
04:46:41  Setup pyenv:
04:46:42    system
04:46:42    3.8.13
04:46:42    3.9.13
04:46:42    3.10.13
04:46:42  * 3.11.7 (set by /w/workspace/openflowplugin-csit-3node-clustering-only-calcium/.python-version)
04:46:42  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-S6z6 from file:/tmp/.os_lf_venv
04:46:43  lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes python-heatclient python-openstackclient 
04:46:55  lf-activate-venv(): INFO: Adding /tmp/venv-S6z6/bin to PATH
04:46:55  INFO: Retrieving stack cost for: releng-openflowplugin-csit-3node-clustering-only-calcium-658
04:47:00  openstack.exceptions.BadRequestException: BadRequestException: 400: Client Error for url: https://compute.public.mtl1.vexxhost.net/v2.1/12c36e260d8e4bb2913965203b1b491f/servers/detail?name=, Invalid input for query parameters name. Value: . '' is not a 'regex'
04:47:00  WARNING: Unable to get stack costs, continuing anyway
04:47:12  INFO: Deleting stack releng-openflowplugin-csit-3node-clustering-only-calcium-658
04:47:12  Successfully deleted stack releng-openflowplugin-csit-3node-clustering-only-calcium-658
04:47:12  [PostBuildScript] - [INFO] Executing post build scripts.
04:47:12  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins10236242501766634313.sh
04:47:12  ---> sysstat.sh
04:47:12  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins14307028820936501049.sh
04:47:12  ---> package-listing.sh
04:47:12  ++ tr '[:upper:]' '[:lower:]'
04:47:12  ++ facter osfamily
04:47:12  + OS_FAMILY=redhat
04:47:12  + workspace=/w/workspace/openflowplugin-csit-3node-clustering-only-calcium
04:47:12  + START_PACKAGES=/tmp/packages_start.txt
04:47:12  + END_PACKAGES=/tmp/packages_end.txt
04:47:12  + DIFF_PACKAGES=/tmp/packages_diff.txt
04:47:12  + PACKAGES=/tmp/packages_start.txt
04:47:12  + '[' /w/workspace/openflowplugin-csit-3node-clustering-only-calcium ']'
04:47:12  + PACKAGES=/tmp/packages_end.txt
04:47:12  + case "${OS_FAMILY}" in
04:47:12  + rpm -qa
04:47:12  + sort
04:47:13  + '[' -f /tmp/packages_start.txt ']'
04:47:13  + '[' -f /tmp/packages_end.txt ']'
04:47:13  + diff /tmp/packages_start.txt /tmp/packages_end.txt
04:47:13  + '[' /w/workspace/openflowplugin-csit-3node-clustering-only-calcium ']'
04:47:13  + mkdir -p /w/workspace/openflowplugin-csit-3node-clustering-only-calcium/archives/
04:47:13  + cp -f /tmp/packages_diff.txt /tmp/packages_end.txt /tmp/packages_start.txt /w/workspace/openflowplugin-csit-3node-clustering-only-calcium/archives/
04:47:13  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins5187667674667507208.sh
04:47:13  ---> capture-instance-metadata.sh
04:47:13  Setup pyenv:
04:47:13    system
04:47:13    3.8.13
04:47:13    3.9.13
04:47:13    3.10.13
04:47:13  * 3.11.7 (set by /w/workspace/openflowplugin-csit-3node-clustering-only-calcium/.python-version)
04:47:14  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-S6z6 from file:/tmp/.os_lf_venv
04:47:14  lf-activate-venv(): INFO: Installing: lftools 
04:47:22  lf-activate-venv(): INFO: Adding /tmp/venv-S6z6/bin to PATH
04:47:22  INFO: Running in OpenStack, capturing instance metadata
04:47:23  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins14963058697303808994.sh
04:47:23  provisioning config files...
04:47:23  Could not find credentials [logs] for openflowplugin-csit-3node-clustering-only-calcium #658
04:47:23  copy managed file [jenkins-log-archives-settings] to file:/w/workspace/openflowplugin-csit-3node-clustering-only-calcium@tmp/config13863176476265945364tmp
04:47:23  Regular expression run condition: Expression=[^.*logs-s3.*], Label=[odl-logs-s3-cloudfront-index]
04:47:23  Run condition [Regular expression match] enabling perform for step [Provide Configuration files]
04:47:23  provisioning config files...
04:47:23  copy managed file [jenkins-s3-log-ship] to file:/home/jenkins/.aws/credentials
04:47:23  [EnvInject] - Injecting environment variables from a build step.
04:47:23  [EnvInject] - Injecting as environment variables the properties content 
04:47:23  SERVER_ID=logs
04:47:23  
04:47:23  [EnvInject] - Variables injected successfully.
04:47:23  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins11257675843484414785.sh
04:47:23  ---> create-netrc.sh
04:47:23  WARN: Log server credential not found.
04:47:23  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins1762866645294379361.sh
04:47:23  ---> python-tools-install.sh
04:47:23  Setup pyenv:
04:47:23    system
04:47:23    3.8.13
04:47:23    3.9.13
04:47:23    3.10.13
04:47:23  * 3.11.7 (set by /w/workspace/openflowplugin-csit-3node-clustering-only-calcium/.python-version)
04:47:24  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-S6z6 from file:/tmp/.os_lf_venv
04:47:25  lf-activate-venv(): INFO: Installing: lftools 
04:47:33  lf-activate-venv(): INFO: Adding /tmp/venv-S6z6/bin to PATH
04:47:33  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins15341763176868111662.sh
04:47:33  ---> sudo-logs.sh
04:47:33  Archiving 'sudo' log..
04:47:34  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins14573496682250541566.sh
04:47:34  ---> job-cost.sh
04:47:34  Setup pyenv:
04:47:34    system
04:47:34    3.8.13
04:47:34    3.9.13
04:47:34    3.10.13
04:47:34  * 3.11.7 (set by /w/workspace/openflowplugin-csit-3node-clustering-only-calcium/.python-version)
04:47:34  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-S6z6 from file:/tmp/.os_lf_venv
04:47:35  lf-activate-venv(): INFO: Installing: zipp==1.1.0 python-openstackclient urllib3~=1.26.15 
04:47:40  lf-activate-venv(): INFO: Adding /tmp/venv-S6z6/bin to PATH
04:47:40  DEBUG: total: 0
04:47:40  INFO: Retrieving Stack Cost...
04:47:40  INFO: Retrieving Pricing Info for: v3-standard-2
04:47:41  INFO: Archiving Costs
04:47:41  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash -l /tmp/jenkins18381587019332463265.sh
04:47:41  ---> logs-deploy.sh
04:47:41  Setup pyenv:
04:47:41    system
04:47:41    3.8.13
04:47:41    3.9.13
04:47:41    3.10.13
04:47:41  * 3.11.7 (set by /w/workspace/openflowplugin-csit-3node-clustering-only-calcium/.python-version)
04:47:41  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-S6z6 from file:/tmp/.os_lf_venv
04:47:42  lf-activate-venv(): INFO: Installing: lftools 
04:47:50  lf-activate-venv(): INFO: Adding /tmp/venv-S6z6/bin to PATH
04:47:50  WARNING: Nexus logging server not set
04:47:50  INFO: S3 path logs/releng/vex-yul-odl-jenkins-1/openflowplugin-csit-3node-clustering-only-calcium/658/
04:47:50  INFO: archiving logs to S3
04:47:51  ---> uname -a:
04:47:51  Linux prd-centos8-robot-2c-8g-5694.novalocal 4.18.0-553.5.1.el8.x86_64 #1 SMP Tue May 21 05:46:01 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
04:47:51  
04:47:51  
04:47:51  ---> lscpu:
04:47:51  Architecture:        x86_64
04:47:51  CPU op-mode(s):      32-bit, 64-bit
04:47:51  Byte Order:          Little Endian
04:47:51  CPU(s):              2
04:47:51  On-line CPU(s) list: 0,1
04:47:51  Thread(s) per core:  1
04:47:51  Core(s) per socket:  1
04:47:51  Socket(s):           2
04:47:51  NUMA node(s):        1
04:47:51  Vendor ID:           AuthenticAMD
04:47:51  CPU family:          23
04:47:51  Model:               49
04:47:51  Model name:          AMD EPYC-Rome Processor
04:47:51  Stepping:            0
04:47:51  CPU MHz:             2800.000
04:47:51  BogoMIPS:            5600.00
04:47:51  Virtualization:      AMD-V
04:47:51  Hypervisor vendor:   KVM
04:47:51  Virtualization type: full
04:47:51  L1d cache:           32K
04:47:51  L1i cache:           32K
04:47:51  L2 cache:            512K
04:47:51  L3 cache:            16384K
04:47:51  NUMA node0 CPU(s):   0,1
04:47:51  Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr wbnoinvd arat npt nrip_save umip rdpid arch_capabilities
04:47:51  
04:47:51  
04:47:51  ---> nproc:
04:47:51  2
04:47:51  
04:47:51  
04:47:51  ---> df -h:
04:47:51  Filesystem      Size  Used Avail Use% Mounted on
04:47:51  devtmpfs        3.8G     0  3.8G   0% /dev
04:47:51  tmpfs           3.8G     0  3.8G   0% /dev/shm
04:47:51  tmpfs           3.8G   17M  3.8G   1% /run
04:47:51  tmpfs           3.8G     0  3.8G   0% /sys/fs/cgroup
04:47:51  /dev/vda1        40G  8.3G   32G  21% /
04:47:51  tmpfs           770M     0  770M   0% /run/user/1001
04:47:51  
04:47:51  
04:47:51  ---> free -m:
04:47:51                total        used        free      shared  buff/cache   available
04:47:51  Mem:           7697         588        5041          19        2068        6814
04:47:51  Swap:          1023           0        1023
04:47:51  
04:47:51  
04:47:51  ---> ip addr:
04:47:51  1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
04:47:51      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
04:47:51      inet 127.0.0.1/8 scope host lo
04:47:51         valid_lft forever preferred_lft forever
04:47:51      inet6 ::1/128 scope host 
04:47:51         valid_lft forever preferred_lft forever
04:47:51  2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1458 qdisc mq state UP group default qlen 1000
04:47:51      link/ether fa:16:3e:04:51:eb brd ff:ff:ff:ff:ff:ff
04:47:51      altname enp0s3
04:47:51      altname ens3
04:47:51      inet 10.30.171.233/23 brd 10.30.171.255 scope global dynamic noprefixroute eth0
04:47:51         valid_lft 86201sec preferred_lft 86201sec
04:47:51      inet6 fe80::f816:3eff:fe04:51eb/64 scope link 
04:47:51         valid_lft forever preferred_lft forever
04:47:51  
04:47:51  
04:47:51  ---> sar -b -r -n DEV:
04:47:51  Linux 4.18.0-553.5.1.el8.x86_64 (centos-stream-8-robot-7d7a37eb-bc14-4dd6-9530-dc22c5eae738.noval) 	03/08/2025 	_x86_64_	(2 CPU)
04:47:51  
04:47:51  04:44:28     LINUX RESTART	(2 CPU)
04:47:51  
04:47:51  04:45:01 AM       tps      rtps      wtps   bread/s   bwrtn/s
04:47:51  04:46:01 AM     98.98     15.38     83.61   4354.15   7300.85
04:47:51  04:47:01 AM     81.72      0.95     80.77     88.77  10419.44
04:47:51  Average:        90.35      8.16     82.19   2221.46   8860.15
04:47:51  
04:47:51  04:45:01 AM kbmemfree   kbavail kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit  kbactive   kbinact   kbdirty
04:47:51  04:46:01 AM   5257140   6998688   2625292     33.31      2688   1922688    755916      8.46    193796   2105900    209580
04:47:51  04:47:01 AM   5183700   6992116   2698732     34.24      2688   1985756    739676      8.28    224644   2129052       172
04:47:51  Average:      5220420   6995402   2662012     33.77      2688   1954222    747796      8.37    209220   2117476    104876
04:47:51  
04:47:51  04:45:01 AM     IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
04:47:51  04:46:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
04:47:51  04:46:01 AM      eth0    314.33    207.46   1809.37     64.54      0.00      0.00      0.00      0.00
04:47:51  04:47:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
04:47:51  04:47:01 AM      eth0     74.56     49.55    527.98     11.74      0.00      0.00      0.00      0.00
04:47:51  Average:           lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
04:47:51  Average:         eth0    194.44    128.51   1168.67     38.14      0.00      0.00      0.00      0.00
04:47:51  
04:47:51  
04:47:51  ---> sar -P ALL:
04:47:51  Linux 4.18.0-553.5.1.el8.x86_64 (centos-stream-8-robot-7d7a37eb-bc14-4dd6-9530-dc22c5eae738.noval) 	03/08/2025 	_x86_64_	(2 CPU)
04:47:51  
04:47:51  04:44:28     LINUX RESTART	(2 CPU)
04:47:51  
04:47:51  04:45:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
04:47:51  04:46:01 AM     all     43.15      0.00      7.07      1.00      0.13     48.64
04:47:51  04:46:01 AM       0     42.56      0.00      6.97      0.83      0.13     49.50
04:47:51  04:46:01 AM       1     43.74      0.00      7.18      1.17      0.13     47.79
04:47:51  04:47:01 AM     all     25.28      0.00      3.50      1.54      0.08     69.59
04:47:51  04:47:01 AM       0     14.88      0.00      2.97      1.20      0.08     80.86
04:47:51  04:47:01 AM       1     35.66      0.00      4.02      1.89      0.08     58.34
04:47:51  Average:        all     34.22      0.00      5.29      1.27      0.11     59.11
04:47:51  Average:          0     28.73      0.00      4.97      1.02      0.11     65.16
04:47:51  Average:          1     39.71      0.00      5.60      1.53      0.11     53.06
04:47:51  
04:47:51  
04:47:51