04:49:48  Started by upstream project "integration-distribution-test-calcium" build number 652
04:49:48  originally caused by:
04:49:48   Started by upstream project "autorelease-release-calcium-mvn38-openjdk17" build number 553
04:49:48   originally caused by:
04:49:48    Started by timer
04:49:48  Running as SYSTEM
04:49:48  [EnvInject] - Loading node environment variables.
04:49:48  Building remotely on prd-centos8-robot-2c-8g-3701 (centos8-robot-2c-8g) in workspace /w/workspace/openflowplugin-csit-3node-clustering-only-calcium
04:49:48  [ssh-agent] Looking for ssh-agent implementation...
04:49:48  [ssh-agent]   Exec ssh-agent (binary ssh-agent on a remote machine)
04:49:48  $ ssh-agent
04:49:48  SSH_AUTH_SOCK=/tmp/ssh-a68ij1ngwuD2/agent.5284
04:49:48  SSH_AGENT_PID=5285
04:49:48  [ssh-agent] Started.
04:49:48  Running ssh-add (command line suppressed)
04:49:48  Identity added: /w/workspace/openflowplugin-csit-3node-clustering-only-calcium@tmp/private_key_15990794016256741486.key (/w/workspace/openflowplugin-csit-3node-clustering-only-calcium@tmp/private_key_15990794016256741486.key)
04:49:48  [ssh-agent] Using credentials jenkins (Release Engineering Jenkins Key)
04:49:48  The recommended git tool is: NONE
04:49:51  using credential opendaylight-jenkins-ssh
04:49:51  Wiping out workspace first.
04:49:51  Cloning the remote Git repository
04:49:51  Cloning repository git://devvexx.opendaylight.org/mirror/integration/test
04:49:51   > git init /w/workspace/openflowplugin-csit-3node-clustering-only-calcium/test # timeout=10
04:49:51  Fetching upstream changes from git://devvexx.opendaylight.org/mirror/integration/test
04:49:51   > git --version # timeout=10
04:49:51   > git --version # 'git version 2.43.0'
04:49:51  using GIT_SSH to set credentials Release Engineering Jenkins Key
04:49:51  [INFO] Currently running in a labeled security context
04:49:51  [INFO] Currently SELinux is 'enforcing' on the host
04:49:51   > /usr/bin/chcon --type=ssh_home_t /w/workspace/openflowplugin-csit-3node-clustering-only-calcium/test@tmp/jenkins-gitclient-ssh15556979303439230902.key
04:49:51  Verifying host key using known hosts file
04:49:51  You're using 'Known hosts file' strategy to verify ssh host keys, but your known_hosts file does not exist, please go to 'Manage Jenkins' -> 'Security' -> 'Git Host Key Verification Configuration' and configure host key verification.
04:49:51   > git fetch --tags --force --progress -- git://devvexx.opendaylight.org/mirror/integration/test +refs/heads/*:refs/remotes/origin/* # timeout=10
04:49:54   > git config remote.origin.url git://devvexx.opendaylight.org/mirror/integration/test # timeout=10
04:49:54   > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
04:49:55   > git config remote.origin.url git://devvexx.opendaylight.org/mirror/integration/test # timeout=10
04:49:55  Fetching upstream changes from git://devvexx.opendaylight.org/mirror/integration/test
04:49:55  using GIT_SSH to set credentials Release Engineering Jenkins Key
04:49:55  [INFO] Currently running in a labeled security context
04:49:55  [INFO] Currently SELinux is 'enforcing' on the host
04:49:55   > /usr/bin/chcon --type=ssh_home_t /w/workspace/openflowplugin-csit-3node-clustering-only-calcium/test@tmp/jenkins-gitclient-ssh979481899790097830.key
04:49:55  Verifying host key using known hosts file
04:49:55  You're using 'Known hosts file' strategy to verify ssh host keys, but your known_hosts file does not exist, please go to 'Manage Jenkins' -> 'Security' -> 'Git Host Key Verification Configuration' and configure host key verification.
04:49:55   > git fetch --tags --force --progress -- git://devvexx.opendaylight.org/mirror/integration/test master # timeout=10
04:49:55   > git rev-parse FETCH_HEAD^{commit} # timeout=10
04:49:55  Checking out Revision 1d2a1a30bcc48e3c8d571eb0d9b6891fe29dfed4 (origin/master)
04:49:55   > git config core.sparsecheckout # timeout=10
04:49:55   > git checkout -f 1d2a1a30bcc48e3c8d571eb0d9b6891fe29dfed4 # timeout=10
04:49:55  Commit message: "Add bgpcep variables for titanium"
04:49:55   > git rev-parse FETCH_HEAD^{commit} # timeout=10
04:49:55   > git rev-list --no-walk d42b2c24fe4fcc7fe3ae0d352311e768f07a15b7 # timeout=10
04:49:56  No emails were triggered.
04:49:56  provisioning config files...
04:49:56  copy managed file [npmrc] to file:/home/jenkins/.npmrc
04:49:56  copy managed file [pipconf] to file:/home/jenkins/.config/pip/pip.conf
04:49:56  copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
04:49:56  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins14724089788352901300.sh
04:49:56  ---> python-tools-install.sh
04:49:56  Setup pyenv:
04:49:56    system
04:49:56  * 3.8.13 (set by /opt/pyenv/version)
04:49:56  * 3.9.13 (set by /opt/pyenv/version)
04:49:56  * 3.10.13 (set by /opt/pyenv/version)
04:49:56  * 3.11.7 (set by /opt/pyenv/version)
04:50:01  lf-activate-venv(): INFO: Creating python3 venv at /tmp/venv-wryl
04:50:01  lf-activate-venv(): INFO: Save venv in file: /tmp/.os_lf_venv
04:50:04  lf-activate-venv(): INFO: Installing: lftools 
04:50:30  lf-activate-venv(): INFO: Adding /tmp/venv-wryl/bin to PATH
04:50:30  Generating Requirements File
04:50:50  Python 3.11.7
04:50:51  pip 25.0.1 from /tmp/venv-wryl/lib/python3.11/site-packages/pip (python 3.11)
04:50:51  appdirs==1.4.4
04:50:51  argcomplete==3.5.3
04:50:51  aspy.yaml==1.3.0
04:50:51  attrs==25.1.0
04:50:51  autopage==0.5.2
04:50:51  beautifulsoup4==4.13.3
04:50:51  boto3==1.37.4
04:50:51  botocore==1.37.4
04:50:51  bs4==0.0.2
04:50:51  cachetools==5.5.2
04:50:51  certifi==2025.1.31
04:50:51  cffi==1.17.1
04:50:51  cfgv==3.4.0
04:50:51  chardet==5.2.0
04:50:51  charset-normalizer==3.4.1
04:50:51  click==8.1.8
04:50:51  cliff==4.9.1
04:50:51  cmd2==2.5.11
04:50:51  cryptography==3.3.2
04:50:51  debtcollector==3.0.0
04:50:51  decorator==5.2.1
04:50:51  defusedxml==0.7.1
04:50:51  Deprecated==1.2.18
04:50:51  distlib==0.3.9
04:50:51  dnspython==2.7.0
04:50:51  docker==4.2.2
04:50:51  dogpile.cache==1.3.4
04:50:51  durationpy==0.9
04:50:51  email_validator==2.2.0
04:50:51  filelock==3.17.0
04:50:51  future==1.0.0
04:50:51  gitdb==4.0.12
04:50:51  GitPython==3.1.44
04:50:51  google-auth==2.38.0
04:50:51  httplib2==0.22.0
04:50:51  identify==2.6.8
04:50:51  idna==3.10
04:50:51  importlib-resources==1.5.0
04:50:51  iso8601==2.1.0
04:50:51  Jinja2==3.1.5
04:50:51  jmespath==1.0.1
04:50:51  jsonpatch==1.33
04:50:51  jsonpointer==3.0.0
04:50:51  jsonschema==4.23.0
04:50:51  jsonschema-specifications==2024.10.1
04:50:51  keystoneauth1==5.10.0
04:50:51  kubernetes==32.0.1
04:50:51  lftools==0.37.10
04:50:51  lxml==5.3.1
04:50:51  MarkupSafe==3.0.2
04:50:51  msgpack==1.1.0
04:50:51  multi_key_dict==2.0.3
04:50:51  munch==4.0.0
04:50:51  netaddr==1.3.0
04:50:51  niet==1.4.2
04:50:51  nodeenv==1.9.1
04:50:51  oauth2client==4.1.3
04:50:51  oauthlib==3.2.2
04:50:51  openstacksdk==4.4.0
04:50:51  os-client-config==2.1.0
04:50:51  os-service-types==1.7.0
04:50:51  osc-lib==3.2.0
04:50:51  oslo.config==9.7.1
04:50:51  oslo.context==5.7.1
04:50:51  oslo.i18n==6.5.1
04:50:51  oslo.log==7.1.0
04:50:51  oslo.serialization==5.7.0
04:50:51  oslo.utils==8.2.0
04:50:51  packaging==24.2
04:50:51  pbr==6.1.1
04:50:51  platformdirs==4.3.6
04:50:51  prettytable==3.15.1
04:50:51  psutil==7.0.0
04:50:51  pyasn1==0.6.1
04:50:51  pyasn1_modules==0.4.1
04:50:51  pycparser==2.22
04:50:51  pygerrit2==2.0.15
04:50:51  PyGithub==2.6.1
04:50:51  PyJWT==2.10.1
04:50:51  PyNaCl==1.5.0
04:50:51  pyparsing==2.4.7
04:50:51  pyperclip==1.9.0
04:50:51  pyrsistent==0.20.0
04:50:51  python-cinderclient==9.6.0
04:50:51  python-dateutil==2.9.0.post0
04:50:51  python-heatclient==4.1.0
04:50:51  python-jenkins==1.8.2
04:50:51  python-keystoneclient==5.6.0
04:50:51  python-magnumclient==4.8.0
04:50:51  python-openstackclient==7.4.0
04:50:51  python-swiftclient==4.7.0
04:50:51  PyYAML==6.0.2
04:50:51  referencing==0.36.2
04:50:51  requests==2.32.3
04:50:51  requests-oauthlib==2.0.0
04:50:51  requestsexceptions==1.4.0
04:50:51  rfc3986==2.0.0
04:50:51  rpds-py==0.23.1
04:50:51  rsa==4.9
04:50:51  ruamel.yaml==0.18.10
04:50:51  ruamel.yaml.clib==0.2.12
04:50:51  s3transfer==0.11.3
04:50:51  simplejson==3.20.1
04:50:51  six==1.17.0
04:50:51  smmap==5.0.2
04:50:51  soupsieve==2.6
04:50:51  stevedore==5.4.1
04:50:51  tabulate==0.9.0
04:50:51  toml==0.10.2
04:50:51  tomlkit==0.13.2
04:50:51  tqdm==4.67.1
04:50:51  typing_extensions==4.12.2
04:50:51  tzdata==2025.1
04:50:51  urllib3==1.26.20
04:50:51  virtualenv==20.29.2
04:50:51  wcwidth==0.2.13
04:50:51  websocket-client==1.8.0
04:50:51  wrapt==1.17.2
04:50:51  xdg==6.0.0
04:50:51  xmltodict==0.14.2
04:50:51  yq==3.4.3
04:50:51  [EnvInject] - Injecting environment variables from a build step.
04:50:51  [EnvInject] - Injecting as environment variables the properties content 
04:50:51  OS_STACK_TEMPLATE=csit-2-instance-type.yaml
04:50:51  OS_CLOUD=vex
04:50:51  OS_STACK_NAME=releng-openflowplugin-csit-3node-clustering-only-calcium-651
04:50:51  OS_STACK_TEMPLATE_DIR=openstack-hot
04:50:51  
04:50:51  [EnvInject] - Variables injected successfully.
04:50:51  provisioning config files...
04:50:51  copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
04:50:51  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins6418215092302673630.sh
04:50:51  ---> Create parameters file for OpenStack HOT
04:50:51  OpenStack Heat parameters generated
04:50:51  -----------------------------------
04:50:51  parameters:
04:50:51      vm_0_count: '3'
04:50:51      vm_0_flavor: 'v3-standard-4'
04:50:51      vm_0_image: 'ZZCI - Ubuntu 20.04 - builder - x86_64 - 20250101-160150.998'
04:50:51      vm_1_count: '1'
04:50:51      vm_1_flavor: 'v3-standard-2'
04:50:51      vm_1_image: 'ZZCI - Ubuntu 22.04 - mininet-ovs-217 - x86_64 - 20250201-060151.911'
04:50:51      
04:50:51      job_name: '57946-651'
04:50:51      silo: 'releng'
04:50:51  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash -l /tmp/jenkins12804230207333157325.sh
04:50:51  ---> Create HEAT stack
04:50:51  + source /home/jenkins/lf-env.sh
04:50:51  + lf-activate-venv --python python3 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
04:50:51  ++ mktemp -d /tmp/venv-XXXX
04:50:51  + lf_venv=/tmp/venv-otDg
04:50:51  + local venv_file=/tmp/.os_lf_venv
04:50:51  + local python=python3
04:50:51  + local options
04:50:51  + local set_path=true
04:50:51  + local install_args=
04:50:51  ++ getopt -o np:v: -l no-path,system-site-packages,python:,venv-file: -n lf-activate-venv -- --python python3 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
04:50:51  + options=' --python '\''python3'\'' -- '\''lftools[openstack]'\'' '\''kubernetes'\'' '\''niet'\'' '\''python-heatclient'\'' '\''python-openstackclient'\'' '\''python-magnumclient'\'' '\''yq'\'''
04:50:51  + eval set -- ' --python '\''python3'\'' -- '\''lftools[openstack]'\'' '\''kubernetes'\'' '\''niet'\'' '\''python-heatclient'\'' '\''python-openstackclient'\'' '\''python-magnumclient'\'' '\''yq'\'''
04:50:51  ++ set -- --python python3 -- 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
04:50:51  + true
04:50:51  + case $1 in
04:50:51  + python=python3
04:50:51  + shift 2
04:50:51  + true
04:50:51  + case $1 in
04:50:51  + shift
04:50:51  + break
04:50:51  + case $python in
04:50:51  + local pkg_list=
04:50:51  + [[ -d /opt/pyenv ]]
04:50:51  + echo 'Setup pyenv:'
04:50:51  Setup pyenv:
04:50:51  + export PYENV_ROOT=/opt/pyenv
04:50:51  + PYENV_ROOT=/opt/pyenv
04:50:51  + export PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
04:50:51  + PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
04:50:51  + pyenv versions
04:50:51    system
04:50:51    3.8.13
04:50:51    3.9.13
04:50:51    3.10.13
04:50:52  * 3.11.7 (set by /w/workspace/openflowplugin-csit-3node-clustering-only-calcium/.python-version)
04:50:52  + command -v pyenv
04:50:52  ++ pyenv init - --no-rehash
04:50:52  + eval 'PATH="$(bash --norc -ec '\''IFS=:; paths=($PATH); 
04:50:52  for i in ${!paths[@]}; do 
04:50:52  if [[ ${paths[i]} == "'\'''\''/opt/pyenv/shims'\'''\''" ]]; then unset '\''\'\'''\''paths[i]'\''\'\'''\''; 
04:50:52  fi; done; 
04:50:52  echo "${paths[*]}"'\'')"
04:50:52  export PATH="/opt/pyenv/shims:${PATH}"
04:50:52  export PYENV_SHELL=bash
04:50:52  source '\''/opt/pyenv/libexec/../completions/pyenv.bash'\''
04:50:52  pyenv() {
04:50:52    local command
04:50:52    command="${1:-}"
04:50:52    if [ "$#" -gt 0 ]; then
04:50:52      shift
04:50:52    fi
04:50:52  
04:50:52    case "$command" in
04:50:52    rehash|shell)
04:50:52      eval "$(pyenv "sh-$command" "$@")"
04:50:52      ;;
04:50:52    *)
04:50:52      command pyenv "$command" "$@"
04:50:52      ;;
04:50:52    esac
04:50:52  }'
04:50:52  +++ bash --norc -ec 'IFS=:; paths=($PATH); 
04:50:52  for i in ${!paths[@]}; do 
04:50:52  if [[ ${paths[i]} == "/opt/pyenv/shims" ]]; then unset '\''paths[i]'\''; 
04:50:52  fi; done; 
04:50:52  echo "${paths[*]}"'
04:50:52  ++ PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
04:50:52  ++ export PATH=/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
04:50:52  ++ PATH=/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
04:50:52  ++ export PYENV_SHELL=bash
04:50:52  ++ PYENV_SHELL=bash
04:50:52  ++ source /opt/pyenv/libexec/../completions/pyenv.bash
04:50:52  +++ complete -F _pyenv pyenv
04:50:52  ++ lf-pyver python3
04:50:52  ++ local py_version_xy=python3
04:50:52  ++ local py_version_xyz=
04:50:52  ++ awk '{ print $1 }'
04:50:52  ++ pyenv versions
04:50:52  ++ local command
04:50:52  ++ command=versions
04:50:52  ++ '[' 1 -gt 0 ']'
04:50:52  ++ shift
04:50:52  ++ case "$command" in
04:50:52  ++ command pyenv versions
04:50:52  ++ pyenv versions
04:50:52  ++ sed 's/^[ *]* //'
04:50:52  ++ grep -E '^[0-9.]*[0-9]$'
04:50:52  ++ [[ ! -s /tmp/.pyenv_versions ]]
04:50:52  +++ grep '^3' /tmp/.pyenv_versions
04:50:52  +++ sort -V
04:50:52  +++ tail -n 1
04:50:52  ++ py_version_xyz=3.11.7
04:50:52  ++ [[ -z 3.11.7 ]]
04:50:52  ++ echo 3.11.7
04:50:52  ++ return 0
04:50:52  + pyenv local 3.11.7
04:50:52  + local command
04:50:52  + command=local
04:50:52  + '[' 2 -gt 0 ']'
04:50:52  + shift
04:50:52  + case "$command" in
04:50:52  + command pyenv local 3.11.7
04:50:52  + pyenv local 3.11.7
04:50:52  + for arg in "$@"
04:50:52  + case $arg in
04:50:52  + pkg_list+='lftools[openstack] '
04:50:52  + for arg in "$@"
04:50:52  + case $arg in
04:50:52  + pkg_list+='kubernetes '
04:50:52  + for arg in "$@"
04:50:52  + case $arg in
04:50:52  + pkg_list+='niet '
04:50:52  + for arg in "$@"
04:50:52  + case $arg in
04:50:52  + pkg_list+='python-heatclient '
04:50:52  + for arg in "$@"
04:50:52  + case $arg in
04:50:52  + pkg_list+='python-openstackclient '
04:50:52  + for arg in "$@"
04:50:52  + case $arg in
04:50:52  + pkg_list+='python-magnumclient '
04:50:52  + for arg in "$@"
04:50:52  + case $arg in
04:50:52  + pkg_list+='yq '
04:50:52  + [[ -f /tmp/.os_lf_venv ]]
04:50:52  ++ cat /tmp/.os_lf_venv
04:50:52  + lf_venv=/tmp/venv-wryl
04:50:52  + echo 'lf-activate-venv(): INFO: Reuse venv:/tmp/venv-wryl from' file:/tmp/.os_lf_venv
04:50:52  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-wryl from file:/tmp/.os_lf_venv
04:50:52  + /tmp/venv-wryl/bin/python3 -m pip install --upgrade --quiet pip virtualenv
04:50:53  + [[ -z lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq  ]]
04:50:53  + echo 'lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq '
04:50:53  lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq 
04:50:53  + /tmp/venv-wryl/bin/python3 -m pip install --upgrade --quiet --upgrade-strategy eager 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
04:51:08  + type python3
04:51:08  + true
04:51:08  + echo 'lf-activate-venv(): INFO: Adding /tmp/venv-wryl/bin to PATH'
04:51:08  lf-activate-venv(): INFO: Adding /tmp/venv-wryl/bin to PATH
04:51:08  + PATH=/tmp/venv-wryl/bin:/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
04:51:08  + return 0
04:51:08  + openstack --os-cloud vex limits show --absolute
04:51:10  +--------------------------+---------+
04:51:10  | Name                     |   Value |
04:51:10  +--------------------------+---------+
04:51:10  | maxTotalInstances        |      -1 |
04:51:10  | maxTotalCores            |     450 |
04:51:10  | maxTotalRAMSize          | 1000000 |
04:51:10  | maxServerMeta            |     128 |
04:51:10  | maxImageMeta             |     128 |
04:51:10  | maxPersonality           |       5 |
04:51:10  | maxPersonalitySize       |   10240 |
04:51:10  | maxTotalKeypairs         |     100 |
04:51:10  | maxServerGroups          |      10 |
04:51:10  | maxServerGroupMembers    |      10 |
04:51:10  | maxTotalFloatingIps      |      -1 |
04:51:10  | maxSecurityGroups        |      -1 |
04:51:10  | maxSecurityGroupRules    |      -1 |
04:51:10  | totalRAMUsed             |  901120 |
04:51:10  | totalCoresUsed           |     220 |
04:51:10  | totalInstancesUsed       |      72 |
04:51:10  | totalFloatingIpsUsed     |       0 |
04:51:10  | totalSecurityGroupsUsed  |       0 |
04:51:10  | totalServerGroupsUsed    |       0 |
04:51:10  | maxTotalVolumes          |      -1 |
04:51:10  | maxTotalSnapshots        |      10 |
04:51:10  | maxTotalVolumeGigabytes  |    4096 |
04:51:10  | maxTotalBackups          |      10 |
04:51:10  | maxTotalBackupGigabytes  |    1000 |
04:51:10  | totalVolumesUsed         |       3 |
04:51:10  | totalGigabytesUsed       |      60 |
04:51:10  | totalSnapshotsUsed       |       0 |
04:51:10  | totalBackupsUsed         |       0 |
04:51:10  | totalBackupGigabytesUsed |       0 |
04:51:10  +--------------------------+---------+
04:51:10  + pushd /opt/ciman/openstack-hot
04:51:10  /opt/ciman/openstack-hot /w/workspace/openflowplugin-csit-3node-clustering-only-calcium
04:51:10  + lftools openstack --os-cloud vex stack create releng-openflowplugin-csit-3node-clustering-only-calcium-651 csit-2-instance-type.yaml /w/workspace/openflowplugin-csit-3node-clustering-only-calcium/stack-parameters.yaml
04:51:24  Creating stack releng-openflowplugin-csit-3node-clustering-only-calcium-651
04:51:24  WARN: Failed to initialize stack. Reason: Resource CREATE failed: Forbidden: resources.vm_0_group.resources[1].resources.instance: Quota exceeded for ram: Requested 16384, but already used 1015808 of 1000000 ram (HTTP 403) (Request-ID: req-cda115db-5615-46ab-aac7-2fadc21f16df)
04:51:24  TypeError: delete() missing 1 required positional argument: 'force'
04:51:24  Build step 'Execute shell' marked build as failure
04:51:24  $ ssh-agent -k
04:51:24  unset SSH_AUTH_SOCK;
04:51:24  unset SSH_AGENT_PID;
04:51:24  echo Agent pid 5285 killed;
04:51:24  [ssh-agent] Stopped.
04:51:24  Recording plot data
04:51:25  Robot results publisher started...
04:51:25  INFO: Checking test criticality is deprecated and will be dropped in a future release!
04:51:25  -Parsing output xml:
04:51:25  Failed!
04:51:25  hudson.AbortException: No files found in path /w/workspace/openflowplugin-csit-3node-clustering-only-calcium with configured filemask: output.xml
04:51:25  	at hudson.plugins.robot.RobotParser$RobotParserCallable.invoke(RobotParser.java:81)
04:51:25  	at hudson.plugins.robot.RobotParser$RobotParserCallable.invoke(RobotParser.java:52)
04:51:25  	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3616)
04:51:25  	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
04:51:25  	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
04:51:25  	at hudson.remoting.Request$2.run(Request.java:377)
04:51:25  	at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
04:51:25  	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
04:51:25  	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
04:51:25  	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
04:51:25  	at java.base/java.lang.Thread.run(Thread.java:829)
04:51:25  	Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to prd-centos8-robot-2c-8g-3701
04:51:25  		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1787)
04:51:25  		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
04:51:25  		at hudson.remoting.Channel.call(Channel.java:1003)
04:51:25  		at hudson.FilePath.act(FilePath.java:1230)
04:51:25  		at hudson.FilePath.act(FilePath.java:1219)
04:51:25  		at hudson.plugins.robot.RobotParser.parse(RobotParser.java:48)
04:51:25  		at hudson.plugins.robot.RobotPublisher.parse(RobotPublisher.java:246)
04:51:25  		at hudson.plugins.robot.RobotPublisher.perform(RobotPublisher.java:268)
04:51:25  		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:80)
04:51:25  		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
04:51:25  		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:818)
04:51:25  		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:767)
04:51:25  		at hudson.model.Build$BuildExecution.post2(Build.java:179)
04:51:25  		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:711)
04:51:25  		at hudson.model.Run.execute(Run.java:1918)
04:51:25  		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:44)
04:51:25  		at hudson.model.ResourceController.execute(ResourceController.java:101)
04:51:25  		at hudson.model.Executor.run(Executor.java:442)
04:51:25  [PostBuildScript] - [INFO] Executing post build scripts.
04:51:25  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins2821504125318162137.sh
04:51:25  Archiving csit artifacts
04:51:25  seq: invalid floating point argument: ‘’
04:51:25  Try 'seq --help' for more information.
04:51:25    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
04:51:25                                   Dload  Upload   Total   Spent    Left  Speed
04:51:25  
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   237    0   237    0     0   9480      0 --:--:-- --:--:-- --:--:--  9480
04:51:25  Archive:  robot-plugin.zip
04:51:25    End-of-central-directory signature not found.  Either this file is not
04:51:25    a zipfile, or it constitutes one disk of a multi-part archive.  In the
04:51:25    latter case the central directory and zipfile comment will be found on
04:51:25    the last disk(s) of this archive.
04:51:25  unzip:  cannot find zipfile directory in one of robot-plugin.zip or
04:51:25          robot-plugin.zip.zip, and cannot find robot-plugin.zip.ZIP, period.
04:51:25  mv: cannot stat '*.log': No such file or directory
04:51:25  mv: cannot stat '*.log.gz': No such file or directory
04:51:25  mv: cannot stat '*.csv': No such file or directory
04:51:25  mv: cannot stat '*.png': No such file or directory
04:51:25  [PostBuildScript] - [INFO] Executing post build scripts.
04:51:25  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins8034236401259975396.sh
04:51:25  [PostBuildScript] - [INFO] Executing post build scripts.
04:51:25  [EnvInject] - Injecting environment variables from a build step.
04:51:25  [EnvInject] - Injecting as environment variables the properties content 
04:51:25  OS_CLOUD=vex
04:51:25  OS_STACK_NAME=releng-openflowplugin-csit-3node-clustering-only-calcium-651
04:51:25  
04:51:25  [EnvInject] - Variables injected successfully.
04:51:25  provisioning config files...
04:51:25  copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
04:51:25  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins9325769557238102094.sh
04:51:25  ---> openstack-stack-delete.sh
04:51:25  Setup pyenv:
04:51:25    system
04:51:25    3.8.13
04:51:25    3.9.13
04:51:25    3.10.13
04:51:25  * 3.11.7 (set by /w/workspace/openflowplugin-csit-3node-clustering-only-calcium/.python-version)
04:51:26  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-wryl from file:/tmp/.os_lf_venv
04:51:26  lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes python-heatclient python-openstackclient 
04:51:38  lf-activate-venv(): INFO: Adding /tmp/venv-wryl/bin to PATH
04:51:38  INFO: Retrieving stack cost for: releng-openflowplugin-csit-3node-clustering-only-calcium-651
04:51:42  openstack.exceptions.BadRequestException: BadRequestException: 400: Client Error for url: https://compute.public.mtl1.vexxhost.net/v2.1/12c36e260d8e4bb2913965203b1b491f/servers/detail?name=, Invalid input for query parameters name. Value: . '' is not a 'regex'
04:51:42  WARNING: Unable to get stack costs, continuing anyway
04:51:54  INFO: Deleting stack releng-openflowplugin-csit-3node-clustering-only-calcium-651
04:51:54  Successfully deleted stack releng-openflowplugin-csit-3node-clustering-only-calcium-651
04:51:54  [PostBuildScript] - [INFO] Executing post build scripts.
04:51:54  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins5087442083577108275.sh
04:51:54  ---> sysstat.sh
04:51:54  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins18398156762169595736.sh
04:51:54  ---> package-listing.sh
04:51:54  ++ tr '[:upper:]' '[:lower:]'
04:51:54  ++ facter osfamily
04:51:55  + OS_FAMILY=redhat
04:51:55  + workspace=/w/workspace/openflowplugin-csit-3node-clustering-only-calcium
04:51:55  + START_PACKAGES=/tmp/packages_start.txt
04:51:55  + END_PACKAGES=/tmp/packages_end.txt
04:51:55  + DIFF_PACKAGES=/tmp/packages_diff.txt
04:51:55  + PACKAGES=/tmp/packages_start.txt
04:51:55  + '[' /w/workspace/openflowplugin-csit-3node-clustering-only-calcium ']'
04:51:55  + PACKAGES=/tmp/packages_end.txt
04:51:55  + case "${OS_FAMILY}" in
04:51:55  + rpm -qa
04:51:55  + sort
04:51:55  + '[' -f /tmp/packages_start.txt ']'
04:51:55  + '[' -f /tmp/packages_end.txt ']'
04:51:55  + diff /tmp/packages_start.txt /tmp/packages_end.txt
04:51:55  + '[' /w/workspace/openflowplugin-csit-3node-clustering-only-calcium ']'
04:51:55  + mkdir -p /w/workspace/openflowplugin-csit-3node-clustering-only-calcium/archives/
04:51:55  + cp -f /tmp/packages_diff.txt /tmp/packages_end.txt /tmp/packages_start.txt /w/workspace/openflowplugin-csit-3node-clustering-only-calcium/archives/
04:51:55  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins17996928060686303784.sh
04:51:55  ---> capture-instance-metadata.sh
04:51:55  Setup pyenv:
04:51:55    system
04:51:55    3.8.13
04:51:55    3.9.13
04:51:55    3.10.13
04:51:55  * 3.11.7 (set by /w/workspace/openflowplugin-csit-3node-clustering-only-calcium/.python-version)
04:51:56  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-wryl from file:/tmp/.os_lf_venv
04:51:56  lf-activate-venv(): INFO: Installing: lftools 
04:52:04  lf-activate-venv(): INFO: Adding /tmp/venv-wryl/bin to PATH
04:52:04  INFO: Running in OpenStack, capturing instance metadata
04:52:04  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins9517829644418205729.sh
04:52:04  provisioning config files...
04:52:04  Could not find credentials [logs] for openflowplugin-csit-3node-clustering-only-calcium #651
04:52:04  copy managed file [jenkins-log-archives-settings] to file:/w/workspace/openflowplugin-csit-3node-clustering-only-calcium@tmp/config12893026844828708559tmp
04:52:04  Regular expression run condition: Expression=[^.*logs-s3.*], Label=[odl-logs-s3-cloudfront-index]
04:52:04  Run condition [Regular expression match] enabling perform for step [Provide Configuration files]
04:52:04  provisioning config files...
04:52:05  copy managed file [jenkins-s3-log-ship] to file:/home/jenkins/.aws/credentials
04:52:05  [EnvInject] - Injecting environment variables from a build step.
04:52:05  [EnvInject] - Injecting as environment variables the properties content 
04:52:05  SERVER_ID=logs
04:52:05  
04:52:05  [EnvInject] - Variables injected successfully.
04:52:05  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins3304924179826934938.sh
04:52:05  ---> create-netrc.sh
04:52:05  WARN: Log server credential not found.
04:52:05  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins8819302957766228073.sh
04:52:05  ---> python-tools-install.sh
04:52:05  Setup pyenv:
04:52:05    system
04:52:05    3.8.13
04:52:05    3.9.13
04:52:05    3.10.13
04:52:05  * 3.11.7 (set by /w/workspace/openflowplugin-csit-3node-clustering-only-calcium/.python-version)
04:52:05  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-wryl from file:/tmp/.os_lf_venv
04:52:06  lf-activate-venv(): INFO: Installing: lftools 
04:52:13  lf-activate-venv(): INFO: Adding /tmp/venv-wryl/bin to PATH
04:52:13  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins438262431612532333.sh
04:52:13  ---> sudo-logs.sh
04:52:13  Archiving 'sudo' log..
04:52:14  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash /tmp/jenkins11350267274994352049.sh
04:52:14  ---> job-cost.sh
04:52:14  Setup pyenv:
04:52:14    system
04:52:14    3.8.13
04:52:14    3.9.13
04:52:14    3.10.13
04:52:14  * 3.11.7 (set by /w/workspace/openflowplugin-csit-3node-clustering-only-calcium/.python-version)
04:52:14  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-wryl from file:/tmp/.os_lf_venv
04:52:15  lf-activate-venv(): INFO: Installing: zipp==1.1.0 python-openstackclient urllib3~=1.26.15 
04:52:20  lf-activate-venv(): INFO: Adding /tmp/venv-wryl/bin to PATH
04:52:20  DEBUG: total: 0
04:52:20  INFO: Retrieving Stack Cost...
04:52:21  INFO: Retrieving Pricing Info for: v3-standard-2
04:52:21  INFO: Archiving Costs
04:52:21  [openflowplugin-csit-3node-clustering-only-calcium] $ /bin/bash -l /tmp/jenkins4496500795685688286.sh
04:52:21  ---> logs-deploy.sh
04:52:21  Setup pyenv:
04:52:21    system
04:52:21    3.8.13
04:52:21    3.9.13
04:52:21    3.10.13
04:52:21  * 3.11.7 (set by /w/workspace/openflowplugin-csit-3node-clustering-only-calcium/.python-version)
04:52:22  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-wryl from file:/tmp/.os_lf_venv
04:52:22  lf-activate-venv(): INFO: Installing: lftools 
04:52:31  lf-activate-venv(): INFO: Adding /tmp/venv-wryl/bin to PATH
04:52:31  WARNING: Nexus logging server not set
04:52:31  INFO: S3 path logs/releng/vex-yul-odl-jenkins-1/openflowplugin-csit-3node-clustering-only-calcium/651/
04:52:31  INFO: archiving logs to S3
04:52:32  ---> uname -a:
04:52:32  Linux prd-centos8-robot-2c-8g-3701.novalocal 4.18.0-553.5.1.el8.x86_64 #1 SMP Tue May 21 05:46:01 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
04:52:32  
04:52:32  
04:52:32  ---> lscpu:
04:52:32  Architecture:        x86_64
04:52:32  CPU op-mode(s):      32-bit, 64-bit
04:52:32  Byte Order:          Little Endian
04:52:32  CPU(s):              2
04:52:32  On-line CPU(s) list: 0,1
04:52:32  Thread(s) per core:  1
04:52:32  Core(s) per socket:  1
04:52:32  Socket(s):           2
04:52:32  NUMA node(s):        1
04:52:32  Vendor ID:           AuthenticAMD
04:52:32  CPU family:          23
04:52:32  Model:               49
04:52:32  Model name:          AMD EPYC-Rome Processor
04:52:32  Stepping:            0
04:52:32  CPU MHz:             2800.000
04:52:32  BogoMIPS:            5600.00
04:52:32  Virtualization:      AMD-V
04:52:32  Hypervisor vendor:   KVM
04:52:32  Virtualization type: full
04:52:32  L1d cache:           32K
04:52:32  L1i cache:           32K
04:52:32  L2 cache:            512K
04:52:32  L3 cache:            16384K
04:52:32  NUMA node0 CPU(s):   0,1
04:52:32  Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr wbnoinvd arat npt nrip_save umip rdpid arch_capabilities
04:52:32  
04:52:32  
04:52:32  ---> nproc:
04:52:32  2
04:52:32  
04:52:32  
04:52:32  ---> df -h:
04:52:32  Filesystem      Size  Used Avail Use% Mounted on
04:52:32  devtmpfs        3.8G     0  3.8G   0% /dev
04:52:32  tmpfs           3.8G     0  3.8G   0% /dev/shm
04:52:32  tmpfs           3.8G   17M  3.8G   1% /run
04:52:32  tmpfs           3.8G     0  3.8G   0% /sys/fs/cgroup
04:52:32  /dev/vda1        40G  8.3G   32G  21% /
04:52:32  tmpfs           770M     0  770M   0% /run/user/1001
04:52:32  
04:52:32  
04:52:32  ---> free -m:
04:52:32                total        used        free      shared  buff/cache   available
04:52:32  Mem:           7697         567        5055          19        2074        6834
04:52:32  Swap:          1023           0        1023
04:52:32  
04:52:32  
04:52:32  ---> ip addr:
04:52:32  1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
04:52:32      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
04:52:32      inet 127.0.0.1/8 scope host lo
04:52:32         valid_lft forever preferred_lft forever
04:52:32      inet6 ::1/128 scope host 
04:52:32         valid_lft forever preferred_lft forever
04:52:32  2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1458 qdisc mq state UP group default qlen 1000
04:52:32      link/ether fa:16:3e:3f:56:c5 brd ff:ff:ff:ff:ff:ff
04:52:32      altname enp0s3
04:52:32      altname ens3
04:52:32      inet 10.30.171.223/23 brd 10.30.171.255 scope global dynamic noprefixroute eth0
04:52:32         valid_lft 86185sec preferred_lft 86185sec
04:52:32      inet6 fe80::f816:3eff:fe3f:56c5/64 scope link 
04:52:32         valid_lft forever preferred_lft forever
04:52:32  
04:52:32  
04:52:32  ---> sar -b -r -n DEV:
04:52:32  Linux 4.18.0-553.5.1.el8.x86_64 (centos-stream-8-robot-7d7a37eb-bc14-4dd6-9530-dc22c5eae738.noval) 	03/01/2025 	_x86_64_	(2 CPU)
04:52:32  
04:52:32  04:48:53     LINUX RESTART	(2 CPU)
04:52:32  
04:52:32  04:50:01 AM       tps      rtps      wtps   bread/s   bwrtn/s
04:52:32  04:51:01 AM    117.59      0.45    117.14     16.79  11083.28
04:52:32  04:52:01 AM     45.88      0.97     44.92     96.23   5349.43
04:52:32  Average:        81.74      0.71     81.04     56.50   8217.07
04:52:32  
04:52:32  04:50:01 AM kbmemfree   kbavail kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit  kbactive   kbinact   kbdirty
04:52:32  04:51:01 AM   5183656   7024988   2698768     34.24      2688   2019540    695604      7.79    191160   2167892    166372
04:52:32  04:52:01 AM   5209220   7023368   2673204     33.91      2688   1991764    702656      7.87    382996   1944868       152
04:52:32  Average:      5196438   7024178   2685986     34.08      2688   2005652    699130      7.83    287078   2056380     83262
04:52:32  
04:52:32  04:50:01 AM     IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
04:52:32  04:51:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
04:52:32  04:51:01 AM      eth0    131.26    101.88   1458.70     13.41      0.00      0.00      0.00      0.00
04:52:32  04:52:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
04:52:32  04:52:01 AM      eth0     26.21     17.63     20.62     16.61      0.00      0.00      0.00      0.00
04:52:32  Average:           lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
04:52:32  Average:         eth0     78.75     59.77    739.84     15.01      0.00      0.00      0.00      0.00
04:52:32  
04:52:32  
04:52:32  ---> sar -P ALL:
04:52:32  Linux 4.18.0-553.5.1.el8.x86_64 (centos-stream-8-robot-7d7a37eb-bc14-4dd6-9530-dc22c5eae738.noval) 	03/01/2025 	_x86_64_	(2 CPU)
04:52:32  
04:52:32  04:48:53     LINUX RESTART	(2 CPU)
04:52:32  
04:52:32  04:50:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
04:52:32  04:51:01 AM     all     37.74      0.00      5.62      3.16      0.13     53.35
04:52:32  04:51:01 AM       0     38.79      0.00      5.59      2.57      0.13     52.91
04:52:32  04:51:01 AM       1     36.70      0.00      5.65      3.75      0.12     53.79
04:52:32  04:52:01 AM     all     22.17      0.00      2.67      2.03      0.08     73.05
04:52:32  04:52:01 AM       0      9.94      0.00      2.16      2.24      0.10     85.56
04:52:32  04:52:01 AM       1     34.38      0.00      3.17      1.82      0.07     60.56
04:52:32  Average:        all     29.97      0.00      4.14      2.60      0.10     63.19
04:52:32  Average:          0     24.38      0.00      3.88      2.41      0.12     69.22
04:52:32  Average:          1     35.54      0.00      4.41      2.78      0.09     57.17
04:52:32  
04:52:32  
04:52:32