23:35:43  Started by upstream project "integration-distribution-mri-test-calcium" build number 65
23:35:43  originally caused by:
23:35:43   Started by timer
23:35:43  Running as SYSTEM
23:35:43  [EnvInject] - Loading node environment variables.
23:35:43  Building remotely on prd-centos8-robot-2c-8g-884 (centos8-robot-2c-8g) in workspace /w/workspace/controller-csit-3node-rest-clust-cars-perf-tell-only-calcium
23:35:44  [ssh-agent] Looking for ssh-agent implementation...
23:35:44  [ssh-agent]   Exec ssh-agent (binary ssh-agent on a remote machine)
23:35:44  $ ssh-agent
23:35:44  SSH_AUTH_SOCK=/tmp/ssh-5pj0wqK7AY2B/agent.5293
23:35:44  SSH_AGENT_PID=5294
23:35:44  [ssh-agent] Started.
23:35:44  Running ssh-add (command line suppressed)
23:35:44  Identity added: /w/workspace/controller-csit-3node-rest-clust-cars-perf-tell-only-calcium@tmp/private_key_1815442868657831347.key (/w/workspace/controller-csit-3node-rest-clust-cars-perf-tell-only-calcium@tmp/private_key_1815442868657831347.key)
23:35:44  [ssh-agent] Using credentials jenkins (Release Engineering Jenkins Key)
23:35:44  The recommended git tool is: NONE
23:35:46  using credential opendaylight-jenkins-ssh
23:35:46  Wiping out workspace first.
23:35:46  Cloning the remote Git repository
23:35:46  Cloning repository git://devvexx.opendaylight.org/mirror/integration/test
23:35:46   > git init /w/workspace/controller-csit-3node-rest-clust-cars-perf-tell-only-calcium/test # timeout=10
23:35:46  Fetching upstream changes from git://devvexx.opendaylight.org/mirror/integration/test
23:35:46   > git --version # timeout=10
23:35:46   > git --version # 'git version 2.43.0'
23:35:46  using GIT_SSH to set credentials Release Engineering Jenkins Key
23:35:46  [INFO] Currently running in a labeled security context
23:35:46  [INFO] Currently SELinux is 'enforcing' on the host
23:35:46   > /usr/bin/chcon --type=ssh_home_t /w/workspace/controller-csit-3node-rest-clust-cars-perf-tell-only-calcium/test@tmp/jenkins-gitclient-ssh15807763982621949308.key
23:35:46  Verifying host key using known hosts file
23:35:47  You're using 'Known hosts file' strategy to verify ssh host keys, but your known_hosts file does not exist, please go to 'Manage Jenkins' -> 'Security' -> 'Git Host Key Verification Configuration' and configure host key verification.
23:35:47   > git fetch --tags --force --progress -- git://devvexx.opendaylight.org/mirror/integration/test +refs/heads/*:refs/remotes/origin/* # timeout=10
23:35:50   > git config remote.origin.url git://devvexx.opendaylight.org/mirror/integration/test # timeout=10
23:35:50   > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
23:35:50   > git config remote.origin.url git://devvexx.opendaylight.org/mirror/integration/test # timeout=10
23:35:50  Fetching upstream changes from git://devvexx.opendaylight.org/mirror/integration/test
23:35:50  using GIT_SSH to set credentials Release Engineering Jenkins Key
23:35:50  [INFO] Currently running in a labeled security context
23:35:50  [INFO] Currently SELinux is 'enforcing' on the host
23:35:50   > /usr/bin/chcon --type=ssh_home_t /w/workspace/controller-csit-3node-rest-clust-cars-perf-tell-only-calcium/test@tmp/jenkins-gitclient-ssh8913379973223977435.key
23:35:50  Verifying host key using known hosts file
23:35:50  You're using 'Known hosts file' strategy to verify ssh host keys, but your known_hosts file does not exist, please go to 'Manage Jenkins' -> 'Security' -> 'Git Host Key Verification Configuration' and configure host key verification.
23:35:50   > git fetch --tags --force --progress -- git://devvexx.opendaylight.org/mirror/integration/test master # timeout=10
23:35:50   > git rev-parse FETCH_HEAD^{commit} # timeout=10
23:35:50  Checking out Revision 2398daf6144d1f3f14dd440639962fb247a8c4d5 (origin/master)
23:35:50   > git config core.sparsecheckout # timeout=10
23:35:50   > git checkout -f 2398daf6144d1f3f14dd440639962fb247a8c4d5 # timeout=10
23:35:51  Commit message: "Adapt scale tests for new connection-status value"
23:35:51   > git rev-parse FETCH_HEAD^{commit} # timeout=10
23:35:51   > git rev-list --no-walk 950cd65cd21fcdb2f4ef581980b030ffbe85b94b # timeout=10
23:35:51  No emails were triggered.
23:35:51  provisioning config files...
23:35:51  copy managed file [npmrc] to file:/home/jenkins/.npmrc
23:35:51  copy managed file [pipconf] to file:/home/jenkins/.config/pip/pip.conf
23:35:52  copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
23:35:52  [controller-csit-3node-rest-clust-cars-perf-tell-only-calcium] $ /bin/bash /tmp/jenkins16833318831162650210.sh
23:35:52  ---> python-tools-install.sh
23:35:52  Setup pyenv:
23:35:52    system
23:35:52  * 3.8.13 (set by /opt/pyenv/version)
23:35:52  * 3.9.13 (set by /opt/pyenv/version)
23:35:52  * 3.10.13 (set by /opt/pyenv/version)
23:35:52  * 3.11.7 (set by /opt/pyenv/version)
23:35:57  lf-activate-venv(): INFO: Creating python3 venv at /tmp/venv-rmsb
23:35:57  lf-activate-venv(): INFO: Save venv in file: /tmp/.os_lf_venv
23:36:00  lf-activate-venv(): INFO: Installing: lftools 
23:36:36  lf-activate-venv(): INFO: Adding /tmp/venv-rmsb/bin to PATH
23:36:36  Generating Requirements File
23:36:57  Python 3.11.7
23:36:57  pip 24.3.1 from /tmp/venv-rmsb/lib/python3.11/site-packages/pip (python 3.11)
23:36:57  appdirs==1.4.4
23:36:57  argcomplete==3.5.2
23:36:57  aspy.yaml==1.3.0
23:36:57  attrs==24.2.0
23:36:57  autopage==0.5.2
23:36:57  beautifulsoup4==4.12.3
23:36:57  boto3==1.35.76
23:36:57  botocore==1.35.76
23:36:57  bs4==0.0.2
23:36:57  cachetools==5.5.0
23:36:57  certifi==2024.8.30
23:36:57  cffi==1.17.1
23:36:57  cfgv==3.4.0
23:36:57  chardet==5.2.0
23:36:57  charset-normalizer==3.4.0
23:36:57  click==8.1.7
23:36:57  cliff==4.8.0
23:36:57  cmd2==2.5.7
23:36:57  cryptography==3.3.2
23:36:57  debtcollector==3.0.0
23:36:57  decorator==5.1.1
23:36:57  defusedxml==0.7.1
23:36:57  Deprecated==1.2.15
23:36:57  distlib==0.3.9
23:36:57  dnspython==2.7.0
23:36:57  docker==4.2.2
23:36:57  dogpile.cache==1.3.3
23:36:57  durationpy==0.9
23:36:57  email_validator==2.2.0
23:36:57  filelock==3.16.1
23:36:57  future==1.0.0
23:36:57  gitdb==4.0.11
23:36:57  GitPython==3.1.43
23:36:57  google-auth==2.36.0
23:36:57  httplib2==0.22.0
23:36:57  identify==2.6.3
23:36:57  idna==3.10
23:36:57  importlib-resources==1.5.0
23:36:57  iso8601==2.1.0
23:36:57  Jinja2==3.1.4
23:36:57  jmespath==1.0.1
23:36:57  jsonpatch==1.33
23:36:57  jsonpointer==3.0.0
23:36:57  jsonschema==4.23.0
23:36:57  jsonschema-specifications==2024.10.1
23:36:57  keystoneauth1==5.9.1
23:36:57  kubernetes==31.0.0
23:36:57  lftools==0.37.10
23:36:57  lxml==5.3.0
23:36:57  MarkupSafe==3.0.2
23:36:57  msgpack==1.1.0
23:36:57  multi_key_dict==2.0.3
23:36:57  munch==4.0.0
23:36:57  netaddr==1.3.0
23:36:57  netifaces==0.11.0
23:36:57  niet==1.4.2
23:36:57  nodeenv==1.9.1
23:36:57  oauth2client==4.1.3
23:36:57  oauthlib==3.2.2
23:36:57  openstacksdk==4.1.0
23:36:57  os-client-config==2.1.0
23:36:57  os-service-types==1.7.0
23:36:57  osc-lib==3.2.0
23:36:57  oslo.config==9.7.0
23:36:57  oslo.context==5.7.0
23:36:57  oslo.i18n==6.5.0
23:36:57  oslo.log==6.2.0
23:36:57  oslo.serialization==5.6.0
23:36:57  oslo.utils==8.0.0
23:36:57  packaging==24.2
23:36:57  pbr==6.1.0
23:36:57  platformdirs==4.3.6
23:36:57  prettytable==3.12.0
23:36:57  psutil==6.1.0
23:36:57  pyasn1==0.6.1
23:36:57  pyasn1_modules==0.4.1
23:36:57  pycparser==2.22
23:36:57  pygerrit2==2.0.15
23:36:57  PyGithub==2.5.0
23:36:57  PyJWT==2.10.1
23:36:57  PyNaCl==1.5.0
23:36:57  pyparsing==2.4.7
23:36:57  pyperclip==1.9.0
23:36:57  pyrsistent==0.20.0
23:36:57  python-cinderclient==9.6.0
23:36:57  python-dateutil==2.9.0.post0
23:36:57  python-heatclient==4.0.0
23:36:57  python-jenkins==1.8.2
23:36:57  python-keystoneclient==5.5.0
23:36:57  python-magnumclient==4.7.0
23:36:57  python-openstackclient==7.2.1
23:36:57  python-swiftclient==4.6.0
23:36:57  PyYAML==6.0.2
23:36:57  referencing==0.35.1
23:36:57  requests==2.32.3
23:36:57  requests-oauthlib==2.0.0
23:36:57  requestsexceptions==1.4.0
23:36:57  rfc3986==2.0.0
23:36:57  rpds-py==0.22.3
23:36:57  rsa==4.9
23:36:57  ruamel.yaml==0.18.6
23:36:57  ruamel.yaml.clib==0.2.12
23:36:57  s3transfer==0.10.4
23:36:57  simplejson==3.19.3
23:36:57  six==1.17.0
23:36:57  smmap==5.0.1
23:36:57  soupsieve==2.6
23:36:57  stevedore==5.4.0
23:36:57  tabulate==0.9.0
23:36:57  toml==0.10.2
23:36:57  tomlkit==0.13.2
23:36:57  tqdm==4.67.1
23:36:57  typing_extensions==4.12.2
23:36:57  tzdata==2024.2
23:36:57  urllib3==1.26.20
23:36:57  virtualenv==20.28.0
23:36:57  wcwidth==0.2.13
23:36:57  websocket-client==1.8.0
23:36:57  wrapt==1.17.0
23:36:57  xdg==6.0.0
23:36:57  xmltodict==0.14.2
23:36:57  yq==3.4.3
23:36:57  [EnvInject] - Injecting environment variables from a build step.
23:36:57  [EnvInject] - Injecting as environment variables the properties content 
23:36:57  OS_STACK_TEMPLATE=csit-2-instance-type.yaml
23:36:57  OS_CLOUD=vex
23:36:57  OS_STACK_NAME=releng-controller-csit-3node-rest-clust-cars-perf-tell-only-calcium-65
23:36:57  OS_STACK_TEMPLATE_DIR=openstack-hot
23:36:57  
23:36:57  [EnvInject] - Variables injected successfully.
23:36:57  provisioning config files...
23:36:57  copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
23:36:57  [controller-csit-3node-rest-clust-cars-perf-tell-only-calcium] $ /bin/bash /tmp/jenkins6336792934955737475.sh
23:36:57  ---> Create parameters file for OpenStack HOT
23:36:57  OpenStack Heat parameters generated
23:36:57  -----------------------------------
23:36:58  parameters:
23:36:58      vm_0_count: '3'
23:36:58      vm_0_flavor: 'v3-standard-4'
23:36:58      vm_0_image: 'ZZCI - Ubuntu 20.04 - builder - x86_64 - 20241201-160151.330'
23:36:58      vm_1_count: '1'
23:36:58      vm_1_flavor: 'v3-standard-2'
23:36:58      vm_1_image: 'ZZCI - Ubuntu 22.04 - mininet-ovs-217 - x86_64 - 20241201-060156.310'
23:36:58      
23:36:58      job_name: '12539-65'
23:36:58      silo: 'releng'
23:36:58  [controller-csit-3node-rest-clust-cars-perf-tell-only-calcium] $ /bin/bash -l /tmp/jenkins13253304536365080905.sh
23:36:58  ---> Create HEAT stack
23:36:58  + source /home/jenkins/lf-env.sh
23:36:58  + lf-activate-venv --python python3 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
23:36:58  ++ mktemp -d /tmp/venv-XXXX
23:36:58  + lf_venv=/tmp/venv-F0h1
23:36:58  + local venv_file=/tmp/.os_lf_venv
23:36:58  + local python=python3
23:36:58  + local options
23:36:58  + local set_path=true
23:36:58  + local install_args=
23:36:58  ++ getopt -o np:v: -l no-path,system-site-packages,python:,venv-file: -n lf-activate-venv -- --python python3 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
23:36:58  + options=' --python '\''python3'\'' -- '\''lftools[openstack]'\'' '\''kubernetes'\'' '\''niet'\'' '\''python-heatclient'\'' '\''python-openstackclient'\'' '\''python-magnumclient'\'' '\''yq'\'''
23:36:58  + eval set -- ' --python '\''python3'\'' -- '\''lftools[openstack]'\'' '\''kubernetes'\'' '\''niet'\'' '\''python-heatclient'\'' '\''python-openstackclient'\'' '\''python-magnumclient'\'' '\''yq'\'''
23:36:58  ++ set -- --python python3 -- 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
23:36:58  + true
23:36:58  + case $1 in
23:36:58  + python=python3
23:36:58  + shift 2
23:36:58  + true
23:36:58  + case $1 in
23:36:58  + shift
23:36:58  + break
23:36:58  + case $python in
23:36:58  + local pkg_list=
23:36:58  + [[ -d /opt/pyenv ]]
23:36:58  + echo 'Setup pyenv:'
23:36:58  Setup pyenv:
23:36:58  + export PYENV_ROOT=/opt/pyenv
23:36:58  + PYENV_ROOT=/opt/pyenv
23:36:58  + export PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
23:36:58  + PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
23:36:58  + pyenv versions
23:36:58    system
23:36:58    3.8.13
23:36:58    3.9.13
23:36:58    3.10.13
23:36:58  * 3.11.7 (set by /w/workspace/controller-csit-3node-rest-clust-cars-perf-tell-only-calcium/.python-version)
23:36:58  + command -v pyenv
23:36:58  ++ pyenv init - --no-rehash
23:36:58  + eval 'PATH="$(bash --norc -ec '\''IFS=:; paths=($PATH); 
23:36:58  for i in ${!paths[@]}; do 
23:36:58  if [[ ${paths[i]} == "'\'''\''/opt/pyenv/shims'\'''\''" ]]; then unset '\''\'\'''\''paths[i]'\''\'\'''\''; 
23:36:58  fi; done; 
23:36:58  echo "${paths[*]}"'\'')"
23:36:58  export PATH="/opt/pyenv/shims:${PATH}"
23:36:58  export PYENV_SHELL=bash
23:36:58  source '\''/opt/pyenv/libexec/../completions/pyenv.bash'\''
23:36:58  pyenv() {
23:36:58    local command
23:36:58    command="${1:-}"
23:36:58    if [ "$#" -gt 0 ]; then
23:36:58      shift
23:36:58    fi
23:36:58  
23:36:58    case "$command" in
23:36:58    rehash|shell)
23:36:58      eval "$(pyenv "sh-$command" "$@")"
23:36:58      ;;
23:36:58    *)
23:36:58      command pyenv "$command" "$@"
23:36:58      ;;
23:36:58    esac
23:36:58  }'
23:36:58  +++ bash --norc -ec 'IFS=:; paths=($PATH); 
23:36:58  for i in ${!paths[@]}; do 
23:36:58  if [[ ${paths[i]} == "/opt/pyenv/shims" ]]; then unset '\''paths[i]'\''; 
23:36:58  fi; done; 
23:36:58  echo "${paths[*]}"'
23:36:58  ++ PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
23:36:58  ++ export PATH=/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
23:36:58  ++ PATH=/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
23:36:58  ++ export PYENV_SHELL=bash
23:36:58  ++ PYENV_SHELL=bash
23:36:58  ++ source /opt/pyenv/libexec/../completions/pyenv.bash
23:36:58  +++ complete -F _pyenv pyenv
23:36:58  ++ lf-pyver python3
23:36:58  ++ local py_version_xy=python3
23:36:58  ++ local py_version_xyz=
23:36:58  ++ awk '{ print $1 }'
23:36:58  ++ pyenv versions
23:36:58  ++ local command
23:36:58  ++ command=versions
23:36:58  ++ '[' 1 -gt 0 ']'
23:36:58  ++ shift
23:36:58  ++ case "$command" in
23:36:58  ++ command pyenv versions
23:36:58  ++ pyenv versions
23:36:58  ++ sed 's/^[ *]* //'
23:36:58  ++ grep -E '^[0-9.]*[0-9]$'
23:36:58  ++ [[ ! -s /tmp/.pyenv_versions ]]
23:36:58  +++ tail -n 1
23:36:58  +++ grep '^3' /tmp/.pyenv_versions
23:36:58  +++ sort -V
23:36:58  ++ py_version_xyz=3.11.7
23:36:58  ++ [[ -z 3.11.7 ]]
23:36:58  ++ echo 3.11.7
23:36:58  ++ return 0
23:36:58  + pyenv local 3.11.7
23:36:58  + local command
23:36:58  + command=local
23:36:58  + '[' 2 -gt 0 ']'
23:36:58  + shift
23:36:58  + case "$command" in
23:36:58  + command pyenv local 3.11.7
23:36:58  + pyenv local 3.11.7
23:36:58  + for arg in "$@"
23:36:58  + case $arg in
23:36:58  + pkg_list+='lftools[openstack] '
23:36:58  + for arg in "$@"
23:36:58  + case $arg in
23:36:58  + pkg_list+='kubernetes '
23:36:58  + for arg in "$@"
23:36:58  + case $arg in
23:36:58  + pkg_list+='niet '
23:36:58  + for arg in "$@"
23:36:58  + case $arg in
23:36:58  + pkg_list+='python-heatclient '
23:36:58  + for arg in "$@"
23:36:58  + case $arg in
23:36:58  + pkg_list+='python-openstackclient '
23:36:58  + for arg in "$@"
23:36:58  + case $arg in
23:36:58  + pkg_list+='python-magnumclient '
23:36:58  + for arg in "$@"
23:36:58  + case $arg in
23:36:58  + pkg_list+='yq '
23:36:58  + [[ -f /tmp/.os_lf_venv ]]
23:36:58  ++ cat /tmp/.os_lf_venv
23:36:58  + lf_venv=/tmp/venv-rmsb
23:36:58  + echo 'lf-activate-venv(): INFO: Reuse venv:/tmp/venv-rmsb from' file:/tmp/.os_lf_venv
23:36:58  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-rmsb from file:/tmp/.os_lf_venv
23:36:58  + /tmp/venv-rmsb/bin/python3 -m pip install --upgrade --quiet pip virtualenv
23:36:59  + [[ -z lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq  ]]
23:36:59  + echo 'lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq '
23:36:59  lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq 
23:36:59  + /tmp/venv-rmsb/bin/python3 -m pip install --upgrade --quiet --upgrade-strategy eager 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
23:37:17  + type python3
23:37:17  + true
23:37:17  + echo 'lf-activate-venv(): INFO: Adding /tmp/venv-rmsb/bin to PATH'
23:37:17  lf-activate-venv(): INFO: Adding /tmp/venv-rmsb/bin to PATH
23:37:17  + PATH=/tmp/venv-rmsb/bin:/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
23:37:17  + return 0
23:37:17  + openstack --os-cloud vex limits show --absolute
23:37:18  +--------------------------+---------+
23:37:18  | Name                     |   Value |
23:37:18  +--------------------------+---------+
23:37:18  | maxTotalInstances        |      -1 |
23:37:18  | maxTotalCores            |     450 |
23:37:18  | maxTotalRAMSize          | 1000000 |
23:37:18  | maxServerMeta            |     128 |
23:37:18  | maxImageMeta             |     128 |
23:37:18  | maxPersonality           |       5 |
23:37:18  | maxPersonalitySize       |   10240 |
23:37:18  | maxTotalKeypairs         |     100 |
23:37:18  | maxServerGroups          |      10 |
23:37:18  | maxServerGroupMembers    |      10 |
23:37:18  | maxTotalFloatingIps      |      -1 |
23:37:18  | maxSecurityGroups        |      -1 |
23:37:18  | maxSecurityGroupRules    |      -1 |
23:37:18  | totalRAMUsed             |  868352 |
23:37:18  | totalCoresUsed           |     212 |
23:37:18  | totalInstancesUsed       |      70 |
23:37:18  | totalFloatingIpsUsed     |       0 |
23:37:18  | totalSecurityGroupsUsed  |       0 |
23:37:18  | totalServerGroupsUsed    |       0 |
23:37:18  | maxTotalVolumes          |      -1 |
23:37:18  | maxTotalSnapshots        |      10 |
23:37:18  | maxTotalVolumeGigabytes  |    4096 |
23:37:18  | maxTotalBackups          |      10 |
23:37:18  | maxTotalBackupGigabytes  |    1000 |
23:37:18  | totalVolumesUsed         |       3 |
23:37:18  | totalGigabytesUsed       |      60 |
23:37:18  | totalSnapshotsUsed       |       0 |
23:37:18  | totalBackupsUsed         |       0 |
23:37:18  | totalBackupGigabytesUsed |       0 |
23:37:18  +--------------------------+---------+
23:37:18  + pushd /opt/ciman/openstack-hot
23:37:18  /opt/ciman/openstack-hot /w/workspace/controller-csit-3node-rest-clust-cars-perf-tell-only-calcium
23:37:18  + lftools openstack --os-cloud vex stack create releng-controller-csit-3node-rest-clust-cars-perf-tell-only-calcium-65 csit-2-instance-type.yaml /w/workspace/controller-csit-3node-rest-clust-cars-perf-tell-only-calcium/stack-parameters.yaml
23:37:33  Creating stack releng-controller-csit-3node-rest-clust-cars-perf-tell-only-calcium-65
23:37:33  WARN: Failed to initialize stack. Reason: Resource CREATE failed: Forbidden: resources.vm_0_group.resources[1].resources.instance: Quota exceeded for ram: Requested 16384, but already used 991232 of 1000000 ram (HTTP 403) (Request-ID: req-185aebb4-32f3-41c4-bb5c-57a5dddd9fdd)
23:37:33  TypeError: delete() missing 1 required positional argument: 'force'
23:37:34  Build step 'Execute shell' marked build as failure
23:37:34  $ ssh-agent -k
23:37:34  unset SSH_AUTH_SOCK;
23:37:34  unset SSH_AGENT_PID;
23:37:34  echo Agent pid 5294 killed;
23:37:34  [ssh-agent] Stopped.
23:37:34  Recording plot data
23:37:34  Robot results publisher started...
23:37:34  INFO: Checking test criticality is deprecated and will be dropped in a future release!
23:37:34  -Parsing output xml:
23:37:34  Failed!
23:37:34  hudson.AbortException: No files found in path /w/workspace/controller-csit-3node-rest-clust-cars-perf-tell-only-calcium with configured filemask: output.xml
23:37:34  	at hudson.plugins.robot.RobotParser$RobotParserCallable.invoke(RobotParser.java:81)
23:37:34  	at hudson.plugins.robot.RobotParser$RobotParserCallable.invoke(RobotParser.java:52)
23:37:34  	at hudson.FilePath$FileCallableWrapper.call(FilePath.java:3616)
23:37:34  	at hudson.remoting.UserRequest.perform(UserRequest.java:211)
23:37:34  	at hudson.remoting.UserRequest.perform(UserRequest.java:54)
23:37:34  	at hudson.remoting.Request$2.run(Request.java:377)
23:37:34  	at hudson.remoting.InterceptingExecutorService.lambda$wrap$0(InterceptingExecutorService.java:78)
23:37:34  	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
23:37:34  	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
23:37:34  	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
23:37:34  	at java.base/java.lang.Thread.run(Thread.java:829)
23:37:34  	Suppressed: hudson.remoting.Channel$CallSiteStackTrace: Remote call to prd-centos8-robot-2c-8g-884
23:37:34  		at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1787)
23:37:34  		at hudson.remoting.UserRequest$ExceptionResponse.retrieve(UserRequest.java:356)
23:37:34  		at hudson.remoting.Channel.call(Channel.java:1003)
23:37:34  		at hudson.FilePath.act(FilePath.java:1230)
23:37:34  		at hudson.FilePath.act(FilePath.java:1219)
23:37:34  		at hudson.plugins.robot.RobotParser.parse(RobotParser.java:48)
23:37:34  		at hudson.plugins.robot.RobotPublisher.parse(RobotPublisher.java:246)
23:37:34  		at hudson.plugins.robot.RobotPublisher.perform(RobotPublisher.java:268)
23:37:34  		at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:80)
23:37:34  		at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
23:37:34  		at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:818)
23:37:34  		at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:767)
23:37:34  		at hudson.model.Build$BuildExecution.post2(Build.java:179)
23:37:34  		at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:711)
23:37:34  		at hudson.model.Run.execute(Run.java:1918)
23:37:34  		at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:44)
23:37:34  		at hudson.model.ResourceController.execute(ResourceController.java:101)
23:37:34  		at hudson.model.Executor.run(Executor.java:442)
23:37:34  [PostBuildScript] - [INFO] Executing post build scripts.
23:37:34  [controller-csit-3node-rest-clust-cars-perf-tell-only-calcium] $ /bin/bash /tmp/jenkins3242076188966762370.sh
23:37:34  Archiving csit artifacts
23:37:34  seq: invalid floating point argument: ‘’
23:37:34  Try 'seq --help' for more information.
23:37:34    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
23:37:34                                   Dload  Upload   Total   Spent    Left  Speed
23:37:34  
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100   237    0   237    0     0   2633      0 --:--:-- --:--:-- --:--:--  2633
23:37:34  Archive:  robot-plugin.zip
23:37:34    End-of-central-directory signature not found.  Either this file is not
23:37:34    a zipfile, or it constitutes one disk of a multi-part archive.  In the
23:37:34    latter case the central directory and zipfile comment will be found on
23:37:34    the last disk(s) of this archive.
23:37:34  unzip:  cannot find zipfile directory in one of robot-plugin.zip or
23:37:34          robot-plugin.zip.zip, and cannot find robot-plugin.zip.ZIP, period.
23:37:34  mv: cannot stat '*.log': No such file or directory
23:37:34  mv: cannot stat '*.log.gz': No such file or directory
23:37:34  mv: cannot stat '*.csv': No such file or directory
23:37:34  mv: cannot stat '*.png': No such file or directory
23:37:34  [PostBuildScript] - [INFO] Executing post build scripts.
23:37:34  [controller-csit-3node-rest-clust-cars-perf-tell-only-calcium] $ /bin/bash /tmp/jenkins2480488659301544792.sh
23:37:34  [PostBuildScript] - [INFO] Executing post build scripts.
23:37:34  [EnvInject] - Injecting environment variables from a build step.
23:37:34  [EnvInject] - Injecting as environment variables the properties content 
23:37:34  OS_CLOUD=vex
23:37:34  OS_STACK_NAME=releng-controller-csit-3node-rest-clust-cars-perf-tell-only-calcium-65
23:37:34  
23:37:34  [EnvInject] - Variables injected successfully.
23:37:34  provisioning config files...
23:37:35  copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
23:37:35  [controller-csit-3node-rest-clust-cars-perf-tell-only-calcium] $ /bin/bash /tmp/jenkins582832396144400106.sh
23:37:35  ---> openstack-stack-delete.sh
23:37:35  Setup pyenv:
23:37:35    system
23:37:35    3.8.13
23:37:35    3.9.13
23:37:35    3.10.13
23:37:35  * 3.11.7 (set by /w/workspace/controller-csit-3node-rest-clust-cars-perf-tell-only-calcium/.python-version)
23:37:35  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-rmsb from file:/tmp/.os_lf_venv
23:37:36  lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes python-heatclient python-openstackclient 
23:37:49  lf-activate-venv(): INFO: Adding /tmp/venv-rmsb/bin to PATH
23:37:49  INFO: Retrieving stack cost for: releng-controller-csit-3node-rest-clust-cars-perf-tell-only-calcium-65
23:37:54  openstack.exceptions.BadRequestException: BadRequestException: 400: Client Error for url: https://compute.public.mtl1.vexxhost.net/v2.1/12c36e260d8e4bb2913965203b1b491f/servers/detail?name=, Invalid input for query parameters name. Value: . '' is not a 'regex'
23:37:54  WARNING: Unable to get stack costs, continuing anyway
23:38:06  INFO: Deleting stack releng-controller-csit-3node-rest-clust-cars-perf-tell-only-calcium-65
23:38:06  Successfully deleted stack releng-controller-csit-3node-rest-clust-cars-perf-tell-only-calcium-65
23:38:06  [PostBuildScript] - [INFO] Executing post build scripts.
23:38:06  [controller-csit-3node-rest-clust-cars-perf-tell-only-calcium] $ /bin/bash /tmp/jenkins762608856091423596.sh
23:38:06  ---> sysstat.sh
23:38:07  [controller-csit-3node-rest-clust-cars-perf-tell-only-calcium] $ /bin/bash /tmp/jenkins2814913214623682324.sh
23:38:07  ---> package-listing.sh
23:38:07  ++ facter osfamily
23:38:07  ++ tr '[:upper:]' '[:lower:]'
23:38:07  + OS_FAMILY=redhat
23:38:07  + workspace=/w/workspace/controller-csit-3node-rest-clust-cars-perf-tell-only-calcium
23:38:07  + START_PACKAGES=/tmp/packages_start.txt
23:38:07  + END_PACKAGES=/tmp/packages_end.txt
23:38:07  + DIFF_PACKAGES=/tmp/packages_diff.txt
23:38:07  + PACKAGES=/tmp/packages_start.txt
23:38:07  + '[' /w/workspace/controller-csit-3node-rest-clust-cars-perf-tell-only-calcium ']'
23:38:07  + PACKAGES=/tmp/packages_end.txt
23:38:07  + case "${OS_FAMILY}" in
23:38:07  + rpm -qa
23:38:07  + sort
23:38:08  + '[' -f /tmp/packages_start.txt ']'
23:38:08  + '[' -f /tmp/packages_end.txt ']'
23:38:08  + diff /tmp/packages_start.txt /tmp/packages_end.txt
23:38:08  + '[' /w/workspace/controller-csit-3node-rest-clust-cars-perf-tell-only-calcium ']'
23:38:08  + mkdir -p /w/workspace/controller-csit-3node-rest-clust-cars-perf-tell-only-calcium/archives/
23:38:08  + cp -f /tmp/packages_diff.txt /tmp/packages_end.txt /tmp/packages_start.txt /w/workspace/controller-csit-3node-rest-clust-cars-perf-tell-only-calcium/archives/
23:38:08  [controller-csit-3node-rest-clust-cars-perf-tell-only-calcium] $ /bin/bash /tmp/jenkins12892838595019798156.sh
23:38:08  ---> capture-instance-metadata.sh
23:38:08  Setup pyenv:
23:38:08    system
23:38:08    3.8.13
23:38:08    3.9.13
23:38:08    3.10.13
23:38:08  * 3.11.7 (set by /w/workspace/controller-csit-3node-rest-clust-cars-perf-tell-only-calcium/.python-version)
23:38:08  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-rmsb from file:/tmp/.os_lf_venv
23:38:09  lf-activate-venv(): INFO: Installing: lftools 
23:38:19  lf-activate-venv(): INFO: Adding /tmp/venv-rmsb/bin to PATH
23:38:19  INFO: Running in OpenStack, capturing instance metadata
23:38:19  [controller-csit-3node-rest-clust-cars-perf-tell-only-calcium] $ /bin/bash /tmp/jenkins16645074320912192637.sh
23:38:19  provisioning config files...
23:38:20  Could not find credentials [logs] for controller-csit-3node-rest-clust-cars-perf-tell-only-calcium #65
23:38:20  copy managed file [jenkins-log-archives-settings] to file:/w/workspace/controller-csit-3node-rest-clust-cars-perf-tell-only-calcium@tmp/config14436288904624998331tmp
23:38:20  Regular expression run condition: Expression=[^.*logs-s3.*], Label=[odl-logs-s3-cloudfront-index]
23:38:20  Run condition [Regular expression match] enabling perform for step [Provide Configuration files]
23:38:20  provisioning config files...
23:38:20  copy managed file [jenkins-s3-log-ship] to file:/home/jenkins/.aws/credentials
23:38:20  [EnvInject] - Injecting environment variables from a build step.
23:38:20  [EnvInject] - Injecting as environment variables the properties content 
23:38:20  SERVER_ID=logs
23:38:20  
23:38:20  [EnvInject] - Variables injected successfully.
23:38:20  [controller-csit-3node-rest-clust-cars-perf-tell-only-calcium] $ /bin/bash /tmp/jenkins1289268239581882996.sh
23:38:20  ---> create-netrc.sh
23:38:20  WARN: Log server credential not found.
23:38:20  [controller-csit-3node-rest-clust-cars-perf-tell-only-calcium] $ /bin/bash /tmp/jenkins465641778071947096.sh
23:38:20  ---> python-tools-install.sh
23:38:20  Setup pyenv:
23:38:20    system
23:38:20    3.8.13
23:38:20    3.9.13
23:38:20    3.10.13
23:38:20  * 3.11.7 (set by /w/workspace/controller-csit-3node-rest-clust-cars-perf-tell-only-calcium/.python-version)
23:38:21  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-rmsb from file:/tmp/.os_lf_venv
23:38:21  lf-activate-venv(): INFO: Installing: lftools 
23:38:30  lf-activate-venv(): INFO: Adding /tmp/venv-rmsb/bin to PATH
23:38:30  [controller-csit-3node-rest-clust-cars-perf-tell-only-calcium] $ /bin/bash /tmp/jenkins4345252597058141360.sh
23:38:30  ---> sudo-logs.sh
23:38:30  Archiving 'sudo' log..
23:38:30  [controller-csit-3node-rest-clust-cars-perf-tell-only-calcium] $ /bin/bash /tmp/jenkins4700700843699301341.sh
23:38:30  ---> job-cost.sh
23:38:30  Setup pyenv:
23:38:31    system
23:38:31    3.8.13
23:38:31    3.9.13
23:38:31    3.10.13
23:38:31  * 3.11.7 (set by /w/workspace/controller-csit-3node-rest-clust-cars-perf-tell-only-calcium/.python-version)
23:38:31  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-rmsb from file:/tmp/.os_lf_venv
23:38:32  lf-activate-venv(): INFO: Installing: zipp==1.1.0 python-openstackclient urllib3~=1.26.15 
23:38:37  lf-activate-venv(): INFO: Adding /tmp/venv-rmsb/bin to PATH
23:38:37  DEBUG: total: 0
23:38:37  INFO: Retrieving Stack Cost...
23:38:37  INFO: Retrieving Pricing Info for: v3-standard-2
23:38:38  INFO: Archiving Costs
23:38:38  [controller-csit-3node-rest-clust-cars-perf-tell-only-calcium] $ /bin/bash -l /tmp/jenkins2721052042308799827.sh
23:38:38  ---> logs-deploy.sh
23:38:38  Setup pyenv:
23:38:38    system
23:38:38    3.8.13
23:38:38    3.9.13
23:38:38    3.10.13
23:38:38  * 3.11.7 (set by /w/workspace/controller-csit-3node-rest-clust-cars-perf-tell-only-calcium/.python-version)
23:38:38  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-rmsb from file:/tmp/.os_lf_venv
23:38:39  lf-activate-venv(): INFO: Installing: lftools 
23:38:48  lf-activate-venv(): INFO: Adding /tmp/venv-rmsb/bin to PATH
23:38:48  WARNING: Nexus logging server not set
23:38:48  INFO: S3 path logs/releng/vex-yul-odl-jenkins-1/controller-csit-3node-rest-clust-cars-perf-tell-only-calcium/65/
23:38:48  INFO: archiving logs to S3
23:38:49  ---> uname -a:
23:38:49  Linux prd-centos8-robot-2c-8g-884.novalocal 4.18.0-553.5.1.el8.x86_64 #1 SMP Tue May 21 05:46:01 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
23:38:49  
23:38:49  
23:38:49  ---> lscpu:
23:38:49  Architecture:        x86_64
23:38:49  CPU op-mode(s):      32-bit, 64-bit
23:38:49  Byte Order:          Little Endian
23:38:49  CPU(s):              2
23:38:49  On-line CPU(s) list: 0,1
23:38:49  Thread(s) per core:  1
23:38:49  Core(s) per socket:  1
23:38:49  Socket(s):           2
23:38:49  NUMA node(s):        1
23:38:49  Vendor ID:           AuthenticAMD
23:38:49  CPU family:          23
23:38:49  Model:               49
23:38:49  Model name:          AMD EPYC-Rome Processor
23:38:49  Stepping:            0
23:38:49  CPU MHz:             2799.998
23:38:49  BogoMIPS:            5599.99
23:38:49  Virtualization:      AMD-V
23:38:49  Hypervisor vendor:   KVM
23:38:49  Virtualization type: full
23:38:49  L1d cache:           32K
23:38:49  L1i cache:           32K
23:38:49  L2 cache:            512K
23:38:49  L3 cache:            16384K
23:38:49  NUMA node0 CPU(s):   0,1
23:38:49  Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr wbnoinvd arat npt nrip_save umip rdpid arch_capabilities
23:38:49  
23:38:49  
23:38:49  ---> nproc:
23:38:49  2
23:38:49  
23:38:49  
23:38:49  ---> df -h:
23:38:49  Filesystem      Size  Used Avail Use% Mounted on
23:38:49  devtmpfs        3.8G     0  3.8G   0% /dev
23:38:49  tmpfs           3.8G     0  3.8G   0% /dev/shm
23:38:49  tmpfs           3.8G   17M  3.8G   1% /run
23:38:49  tmpfs           3.8G     0  3.8G   0% /sys/fs/cgroup
23:38:49  /dev/vda1        40G  8.3G   32G  21% /
23:38:49  tmpfs           770M     0  770M   0% /run/user/1001
23:38:49  
23:38:49  
23:38:49  ---> free -m:
23:38:49                total        used        free      shared  buff/cache   available
23:38:49  Mem:           7697         593        5001          19        2102        6808
23:38:49  Swap:          1023           0        1023
23:38:49  
23:38:49  
23:38:49  ---> ip addr:
23:38:49  1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
23:38:49      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
23:38:49      inet 127.0.0.1/8 scope host lo
23:38:49         valid_lft forever preferred_lft forever
23:38:49      inet6 ::1/128 scope host 
23:38:49         valid_lft forever preferred_lft forever
23:38:49  2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1458 qdisc mq state UP group default qlen 1000
23:38:49      link/ether fa:16:3e:4b:7c:f4 brd ff:ff:ff:ff:ff:ff
23:38:49      altname enp0s3
23:38:49      altname ens3
23:38:49      inet 10.30.170.143/23 brd 10.30.171.255 scope global dynamic noprefixroute eth0
23:38:49         valid_lft 86175sec preferred_lft 86175sec
23:38:49      inet6 fe80::f816:3eff:fe4b:7cf4/64 scope link 
23:38:49         valid_lft forever preferred_lft forever
23:38:49  
23:38:49  
23:38:49  ---> sar -b -r -n DEV:
23:38:49  Linux 4.18.0-553.5.1.el8.x86_64 (centos-stream-8-robot-7d7a37eb-bc14-4dd6-9530-dc22c5eae738.noval) 	12/07/2024 	_x86_64_	(2 CPU)
23:38:49  
23:38:49  23:34:59     LINUX RESTART	(2 CPU)
23:38:49  
23:38:49  11:36:01 PM       tps      rtps      wtps   bread/s   bwrtn/s
23:38:49  11:37:01 PM     81.27      6.95     74.33   1257.58   5243.10
23:38:49  11:38:01 PM     79.29      0.87     78.42     84.24   9761.66
23:38:49  Average:        80.28      3.91     76.37    670.91   7502.38
23:38:49  
23:38:49  11:36:01 PM kbmemfree   kbavail kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit  kbactive   kbinact   kbdirty
23:38:49  11:37:01 PM   5227048   7036108   2655376     33.69      2688   1987060    740700      8.29    210164   2109468    227860
23:38:49  11:38:01 PM   5146756   6989648   2735668     34.71      2688   2020028    778648      8.72    249616   2140332       164
23:38:49  Average:      5186902   7012878   2695522     34.20      2688   2003544    759674      8.51    229890   2124900    114012
23:38:49  
23:38:49  11:36:01 PM     IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
23:38:49  11:37:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
23:38:49  11:37:01 PM      eth0     77.21     63.88    825.35      9.29      0.00      0.00      0.00      0.00
23:38:49  11:38:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
23:38:49  11:38:01 PM      eth0     49.80     36.45    449.13      9.08      0.00      0.00      0.00      0.00
23:38:49  Average:           lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
23:38:49  Average:         eth0     63.50     50.17    637.24      9.18      0.00      0.00      0.00      0.00
23:38:49  
23:38:49  
23:38:49  ---> sar -P ALL:
23:38:49  Linux 4.18.0-553.5.1.el8.x86_64 (centos-stream-8-robot-7d7a37eb-bc14-4dd6-9530-dc22c5eae738.noval) 	12/07/2024 	_x86_64_	(2 CPU)
23:38:49  
23:38:49  23:34:59     LINUX RESTART	(2 CPU)
23:38:49  
23:38:49  11:36:01 PM     CPU     %user     %nice   %system   %iowait    %steal     %idle
23:38:49  11:37:01 PM     all     34.52      0.00      5.68      3.59      0.13     56.08
23:38:49  11:37:01 PM       0     29.83      0.00      5.41      2.85      0.12     61.80
23:38:49  11:37:01 PM       1     39.22      0.00      5.96      4.32      0.13     50.37
23:38:49  11:38:01 PM     all     22.36      0.00      2.63      3.79      0.09     71.13
23:38:49  11:38:01 PM       0     13.18      0.00      2.40      3.58      0.08     80.75
23:38:49  11:38:01 PM       1     31.51      0.00      2.85      3.99      0.10     61.54
23:38:49  Average:        all     28.46      0.00      4.16      3.69      0.11     63.58
23:38:49  Average:          0     21.54      0.00      3.91      3.22      0.10     71.23
23:38:49  Average:          1     35.37      0.00      4.41      4.16      0.12     55.94
23:38:49  
23:38:49  
23:38:49