Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on prd-centos8-builder-2c-2g-25898 (centos8-builder-2c-2g) in workspace /w/workspace/builder-openstack-cron
[ssh-agent] Looking for ssh-agent implementation...
[ssh-agent]   Exec ssh-agent (binary ssh-agent on a remote machine)
$ ssh-agent
SSH_AUTH_SOCK=/tmp/ssh-auZ1E3wNxBxY/agent.5442
SSH_AGENT_PID=5443
[ssh-agent] Started.
Running ssh-add (command line suppressed)
Identity added: /w/workspace/builder-openstack-cron@tmp/private_key_7180430022218844699.key (/w/workspace/builder-openstack-cron@tmp/private_key_7180430022218844699.key)
[ssh-agent] Using credentials jenkins (jenkins-ssh)
The recommended git tool is: NONE
using credential jenkins-ssh
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository git://devvexx.opendaylight.org/mirror/releng/builder
 > git init /w/workspace/builder-openstack-cron # timeout=10
Fetching upstream changes from git://devvexx.opendaylight.org/mirror/releng/builder
 > git --version # timeout=10
 > git --version # 'git version 2.43.0'
using GIT_SSH to set credentials jenkins-ssh
[INFO] Currently running in a labeled security context
[INFO] Currently SELinux is 'enforcing' on the host
 > /usr/bin/chcon --type=ssh_home_t /w/workspace/builder-openstack-cron@tmp/jenkins-gitclient-ssh18247541400362168595.key
Verifying host key using known hosts file
You're using 'Known hosts file' strategy to verify ssh host keys, but your known_hosts file does not exist, please go to 'Manage Jenkins' -> 'Security' -> 'Git Host Key Verification Configuration' and configure host key verification.
 > git fetch --tags --force --progress -- git://devvexx.opendaylight.org/mirror/releng/builder +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url git://devvexx.opendaylight.org/mirror/releng/builder # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
Avoid second fetch
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
Checking out Revision 65b2ba49ef2671c250317b25f759069affcf44cf (refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 65b2ba49ef2671c250317b25f759069affcf44cf # timeout=10
Commit message: "Merge "Run gate-userfeatures-netty-all as part of mri""
 > git rev-list --no-walk 65b2ba49ef2671c250317b25f759069affcf44cf # timeout=10
 > git remote # timeout=10
 > git submodule init # timeout=10
 > git submodule sync # timeout=10
 > git config --get remote.origin.url # timeout=10
 > git submodule init # timeout=10
 > git config -f .gitmodules --get-regexp ^submodule\.(.+)\.url # timeout=10
 > git config --get submodule.global-jjb.url # timeout=10
 > git config -f .gitmodules --get submodule.global-jjb.path # timeout=10
 > git config --get submodule.packer/common-packer.url # timeout=10
 > git config -f .gitmodules --get submodule.packer/common-packer.path # timeout=10
 > git submodule update --init --recursive global-jjb # timeout=10
 > git submodule update --init --recursive packer/common-packer # timeout=10
Run condition [Boolean condition] enabling prebuild for step [Execute shell]
Run condition [Boolean condition] enabling prebuild for step [Execute shell]
Run condition [Boolean condition] enabling prebuild for step [Execute shell]
Run condition [Boolean condition] enabling prebuild for step [Execute shell]
Run condition [Boolean condition] enabling prebuild for step [Execute shell]
Run condition [Boolean condition] enabling prebuild for step [Execute shell]
Run condition [Boolean condition] enabling prebuild for step [BuilderChain]
No emails were triggered.
provisioning config files...
copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
copy managed file [npmrc] to file:/home/jenkins/.npmrc
copy managed file [pipconf] to file:/home/jenkins/.config/pip/pip.conf
[builder-openstack-cron] $ /bin/bash /tmp/jenkins17711343298975893513.sh
---> python-tools-install.sh
Setup pyenv:
  system
* 3.8.13 (set by /opt/pyenv/version)
* 3.9.13 (set by /opt/pyenv/version)
* 3.10.13 (set by /opt/pyenv/version)
* 3.11.7 (set by /opt/pyenv/version)
lf-activate-venv(): INFO: Creating python3 venv at /tmp/venv-ewQY
lf-activate-venv(): INFO: Save venv in file: /tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing: lftools 
lf-activate-venv(): INFO: Adding /tmp/venv-ewQY/bin to PATH
Generating Requirements File
Python 3.11.7
pip 24.2 from /tmp/venv-ewQY/lib/python3.11/site-packages/pip (python 3.11)
appdirs==1.4.4
argcomplete==3.5.0
aspy.yaml==1.3.0
attrs==24.2.0
autopage==0.5.2
beautifulsoup4==4.12.3
boto3==1.35.24
botocore==1.35.24
bs4==0.0.2
cachetools==5.5.0
certifi==2024.8.30
cffi==1.17.1
cfgv==3.4.0
chardet==5.2.0
charset-normalizer==3.3.2
click==8.1.7
cliff==4.7.0
cmd2==2.4.3
cryptography==3.3.2
debtcollector==3.0.0
decorator==5.1.1
defusedxml==0.7.1
Deprecated==1.2.14
distlib==0.3.8
dnspython==2.6.1
docker==4.2.2
dogpile.cache==1.3.3
durationpy==0.7
email_validator==2.2.0
filelock==3.16.1
future==1.0.0
gitdb==4.0.11
GitPython==3.1.43
google-auth==2.35.0
httplib2==0.22.0
identify==2.6.1
idna==3.10
importlib-resources==1.5.0
iso8601==2.1.0
Jinja2==3.1.4
jmespath==1.0.1
jsonpatch==1.33
jsonpointer==3.0.0
jsonschema==4.23.0
jsonschema-specifications==2023.12.1
keystoneauth1==5.8.0
kubernetes==31.0.0
lftools==0.37.10
lxml==5.3.0
MarkupSafe==2.1.5
msgpack==1.1.0
multi_key_dict==2.0.3
munch==4.0.0
netaddr==1.3.0
netifaces==0.11.0
niet==1.4.2
nodeenv==1.9.1
oauth2client==4.1.3
oauthlib==3.2.2
openstacksdk==4.0.0
os-client-config==2.1.0
os-service-types==1.7.0
osc-lib==3.1.0
oslo.config==9.6.0
oslo.context==5.6.0
oslo.i18n==6.4.0
oslo.log==6.1.2
oslo.serialization==5.5.0
oslo.utils==7.3.0
packaging==24.1
pbr==6.1.0
platformdirs==4.3.6
prettytable==3.11.0
pyasn1==0.6.1
pyasn1_modules==0.4.1
pycparser==2.22
pygerrit2==2.0.15
PyGithub==2.4.0
PyJWT==2.9.0
PyNaCl==1.5.0
pyparsing==2.4.7
pyperclip==1.9.0
pyrsistent==0.20.0
python-cinderclient==9.6.0
python-dateutil==2.9.0.post0
python-heatclient==4.0.0
python-jenkins==1.8.2
python-keystoneclient==5.5.0
python-magnumclient==4.7.0
python-openstackclient==7.1.2
python-swiftclient==4.6.0
PyYAML==6.0.2
referencing==0.35.1
requests==2.32.3
requests-oauthlib==2.0.0
requestsexceptions==1.4.0
rfc3986==2.0.0
rpds-py==0.20.0
rsa==4.9
ruamel.yaml==0.18.6
ruamel.yaml.clib==0.2.8
s3transfer==0.10.2
simplejson==3.19.3
six==1.16.0
smmap==5.0.1
soupsieve==2.6
stevedore==5.3.0
tabulate==0.9.0
toml==0.10.2
tomlkit==0.13.2
tqdm==4.66.5
typing_extensions==4.12.2
tzdata==2024.1
urllib3==1.26.20
virtualenv==20.26.5
wcwidth==0.2.13
websocket-client==1.8.0
wrapt==1.16.0
xdg==6.0.0
xmltodict==0.13.0
yq==3.4.3
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties content 
OS_CLOUD=vex

[EnvInject] - Variables injected successfully.
[Boolean condition] checking [True] against [^(1|y|yes|t|true|on|run)$] (origin token: True)
Run condition [Boolean condition] enabling perform for step [Execute shell]
[builder-openstack-cron] $ /bin/bash -l /tmp/jenkins7009134509185934180.sh
---> Orphaned k8s clusters
Setup pyenv:
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/builder-openstack-cron/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-ewQY from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing: kubernetes python-heatclient python-openstackclient python-magnumclient 
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
lftools 0.37.10 requires urllib3<2.1.0, but you have urllib3 2.2.3 which is incompatible.
lf-activate-venv(): INFO: Adding /tmp/venv-ewQY/bin to PATH
-----> Delete orphaned cluster
[Boolean condition] checking [True] against [^(1|y|yes|t|true|on|run)$] (origin token: True)
Run condition [Boolean condition] enabling perform for step [Execute shell]
[builder-openstack-cron] $ /bin/bash -l /tmp/jenkins10203664442967042860.sh
---> Orphaned stacks
Setup pyenv:
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/builder-openstack-cron/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-ewQY from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq 
lf-activate-venv(): INFO: Adding /tmp/venv-ewQY/bin to PATH
+ mapfile -t OS_COE_CLUSTERS_ID
++ openstack --os-cloud vex coe cluster list -f value -c uuid -c name
++ awk '{print $1}'
++ grep -E '(DELETE_FAILED|UNKNOWN|UNHEALTHY)'
+ echo '-----> Active clusters -> stacks'
-----> Active clusters -> stacks
+ OS_COE_STACKS=()
+ [[ 0 -gt 0 ]]
+ mapfile -t OS_STACKS
++ openstack --os-cloud vex stack list -f value -c 'Stack Name' -c 'Stack Status' --property stack_status=CREATE_COMPLETE --property stack_status=DELETE_FAILED --property stack_status=CREATE_FAILED
++ awk '{print $1}'
+ echo '-----> Active stacks'
-----> Active stacks
+ for stack in "${OS_STACKS[@]}"
+ echo releng-openflowplugin-csit-1node-perf-bulkomatic-only-potassium-400
releng-openflowplugin-csit-1node-perf-bulkomatic-only-potassium-400
+ for stack in "${OS_STACKS[@]}"
+ echo releng-daexim-csit-1node-basic-only-potassium-442
releng-daexim-csit-1node-basic-only-potassium-442
+ for stack in "${OS_STACKS[@]}"
+ echo releng-bgpcep-csit-1node-bgp-ingest-mixed-all-titanium-25
releng-bgpcep-csit-1node-bgp-ingest-mixed-all-titanium-25
+ echo '-----> Delete orphaned stacks'
-----> Delete orphaned stacks
+ for STACK_NAME in "${OS_STACKS[@]}"
+ [[ 0 -gt 0 ]]
+ stack_in_jenkins releng-openflowplugin-csit-1node-perf-bulkomatic-only-potassium-400 https://jenkins.opendaylight.org/releng https://jenkins.opendaylight.org/sandbox
+ STACK_NAME=releng-openflowplugin-csit-1node-perf-bulkomatic-only-potassium-400
+ builds=()
+ for jenkins in "${@:2}"
+ PARAMS='tree=computer[executors[currentExecutable[url]],'
+ PARAMS='tree=computer[executors[currentExecutable[url]],oneOffExecutors[currentExecutable[url]]]'
+ PARAMS='tree=computer[executors[currentExecutable[url]],oneOffExecutors[currentExecutable[url]]]&xpath=//url&wrapper=builds'
+ JENKINS_URL='https://jenkins.opendaylight.org/releng/computer/api/json?tree=computer[executors[currentExecutable[url]],oneOffExecutors[currentExecutable[url]]]&xpath=//url&wrapper=builds'
++ curl -s -w '\n\n%{http_code}' --globoff -H Content-Type:application/json 'https://jenkins.opendaylight.org/releng/computer/api/json?tree=computer[executors[currentExecutable[url]],oneOffExecutors[currentExecutable[url]]]&xpath=//url&wrapper=builds'
+ resp='{"_class":"hudson.model.ComputerSet","computer":[{"_class":"hudson.model.Hudson$MasterComputer","executors":[],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":{"_class":"hudson.model.FreeStyleBuild","url":"https://jenkins.opendaylight.org/releng/job/builder-openstack-cron/65171/"}}],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":{"_class":"hudson.model.FreeStyleBuild","url":"https://jenkins.opendaylight.org/releng/job/lispflowmapping-maven-merge-potassium/564/"}}],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":{"_class":"hudson.maven.MavenModuleSetBuild","url":"https://jenkins.opendaylight.org/releng/job/distribution-merge-managed-scandium/839/"}}],"oneOffExecutors":[]}]}

200'
++ echo '{"_class":"hudson.model.ComputerSet","computer":[{"_class":"hudson.model.Hudson$MasterComputer","executors":[],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":{"_class":"hudson.model.FreeStyleBuild","url":"https://jenkins.opendaylight.org/releng/job/builder-openstack-cron/65171/"}}],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":{"_class":"hudson.model.FreeStyleBuild","url":"https://jenkins.opendaylight.org/releng/job/lispflowmapping-maven-merge-potassium/564/"}}],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":{"_class":"hudson.maven.MavenModuleSetBuild","url":"https://jenkins.opendaylight.org/releng/job/distribution-merge-managed-scandium/839/"}}],"oneOffExecutors":[]}]}

200'
++ head -n1
+ json_data='{"_class":"hudson.model.ComputerSet","computer":[{"_class":"hudson.model.Hudson$MasterComputer","executors":[],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":{"_class":"hudson.model.FreeStyleBuild","url":"https://jenkins.opendaylight.org/releng/job/builder-openstack-cron/65171/"}}],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":{"_class":"hudson.model.FreeStyleBuild","url":"https://jenkins.opendaylight.org/releng/job/lispflowmapping-maven-merge-potassium/564/"}}],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":{"_class":"hudson.maven.MavenModuleSetBuild","url":"https://jenkins.opendaylight.org/releng/job/distribution-merge-managed-scandium/839/"}}],"oneOffExecutors":[]}]}'
++ echo '{"_class":"hudson.model.ComputerSet","computer":[{"_class":"hudson.model.Hudson$MasterComputer","executors":[],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":{"_class":"hudson.model.FreeStyleBuild","url":"https://jenkins.opendaylight.org/releng/job/builder-openstack-cron/65171/"}}],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":{"_class":"hudson.model.FreeStyleBuild","url":"https://jenkins.opendaylight.org/releng/job/lispflowmapping-maven-merge-potassium/564/"}}],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":{"_class":"hudson.maven.MavenModuleSetBuild","url":"https://jenkins.opendaylight.org/releng/job/distribution-merge-managed-scandium/839/"}}],"oneOffExecutors":[]}]}

200'
++ awk 'END {print $NF}'
+ status=200
+ '[' 200 '!=' 200 ']'
+ [[ https://jenkins.opendaylight.org/releng == *\j\e\n\k\i\n\s\.*\.\o\r\g ]]
+ [[ https://jenkins.opendaylight.org/releng == *\j\e\n\k\i\n\s\.*\.\i\o ]]
++ echo https://jenkins.opendaylight.org/releng
++ awk -F/ '{print $NF}'
++ sed 's/\/*$//'
+ silo=releng
+ export silo
+ builds=(${builds[@]} $(echo "$json_data" |             jq -r '.computer[].executors[].currentExecutable.url'             | grep -v null | awk -F'/' '{print ENVIRON["silo"] "-" $6 "-" $7}'))
++ grep -v null
++ echo '{"_class":"hudson.model.ComputerSet","computer":[{"_class":"hudson.model.Hudson$MasterComputer","executors":[],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":{"_class":"hudson.model.FreeStyleBuild","url":"https://jenkins.opendaylight.org/releng/job/builder-openstack-cron/65171/"}}],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":{"_class":"hudson.model.FreeStyleBuild","url":"https://jenkins.opendaylight.org/releng/job/lispflowmapping-maven-merge-potassium/564/"}}],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":{"_class":"hudson.maven.MavenModuleSetBuild","url":"https://jenkins.opendaylight.org/releng/job/distribution-merge-managed-scandium/839/"}}],"oneOffExecutors":[]}]}'
++ jq -r '.computer[].executors[].currentExecutable.url'
++ awk -F/ '{print ENVIRON["silo"] "-" $6 "-" $7}'
+ for jenkins in "${@:2}"
+ PARAMS='tree=computer[executors[currentExecutable[url]],'
+ PARAMS='tree=computer[executors[currentExecutable[url]],oneOffExecutors[currentExecutable[url]]]'
+ PARAMS='tree=computer[executors[currentExecutable[url]],oneOffExecutors[currentExecutable[url]]]&xpath=//url&wrapper=builds'
+ JENKINS_URL='https://jenkins.opendaylight.org/sandbox/computer/api/json?tree=computer[executors[currentExecutable[url]],oneOffExecutors[currentExecutable[url]]]&xpath=//url&wrapper=builds'
++ curl -s -w '\n\n%{http_code}' --globoff -H Content-Type:application/json 'https://jenkins.opendaylight.org/sandbox/computer/api/json?tree=computer[executors[currentExecutable[url]],oneOffExecutors[currentExecutable[url]]]&xpath=//url&wrapper=builds'
+ resp='{"_class":"hudson.model.ComputerSet","computer":[{"_class":"hudson.model.Hudson$MasterComputer","executors":[],"oneOffExecutors":[]}]}

200'
++ echo '{"_class":"hudson.model.ComputerSet","computer":[{"_class":"hudson.model.Hudson$MasterComputer","executors":[],"oneOffExecutors":[]}]}

200'
++ head -n1
+ json_data='{"_class":"hudson.model.ComputerSet","computer":[{"_class":"hudson.model.Hudson$MasterComputer","executors":[],"oneOffExecutors":[]}]}'
++ echo '{"_class":"hudson.model.ComputerSet","computer":[{"_class":"hudson.model.Hudson$MasterComputer","executors":[],"oneOffExecutors":[]}]}

200'
++ awk 'END {print $NF}'
+ status=200
+ '[' 200 '!=' 200 ']'
+ [[ https://jenkins.opendaylight.org/sandbox == *\j\e\n\k\i\n\s\.*\.\o\r\g ]]
+ [[ https://jenkins.opendaylight.org/sandbox == *\j\e\n\k\i\n\s\.*\.\i\o ]]
++ echo https://jenkins.opendaylight.org/sandbox
++ awk -F/ '{print $NF}'
++ sed 's/\/*$//'
+ silo=sandbox
+ export silo
+ builds=(${builds[@]} $(echo "$json_data" |             jq -r '.computer[].executors[].currentExecutable.url'             | grep -v null | awk -F'/' '{print ENVIRON["silo"] "-" $6 "-" $7}'))
++ echo '{"_class":"hudson.model.ComputerSet","computer":[{"_class":"hudson.model.Hudson$MasterComputer","executors":[],"oneOffExecutors":[]}]}'
++ awk -F/ '{print ENVIRON["silo"] "-" $6 "-" $7}'
++ jq -r '.computer[].executors[].currentExecutable.url'
++ grep -v null
+ [[ releng-builder-openstack-cron-65171 releng-lispflowmapping-maven-merge-potassium-564 releng-distribution-merge-managed-scandium-839 =~ releng-openflowplugin-csit-1node-perf-bulkomatic-only-potassium-400 ]]
+ return 1
+ echo 'Deleting orphaned stack: releng-openflowplugin-csit-1node-perf-bulkomatic-only-potassium-400'
Deleting orphaned stack: releng-openflowplugin-csit-1node-perf-bulkomatic-only-potassium-400
+ lftools openstack --os-cloud vex stack delete --force releng-openflowplugin-csit-1node-perf-bulkomatic-only-potassium-400
Build timed out (after 10 minutes). Marking the build as failed.
$ ssh-agent -k
unset SSH_AUTH_SOCK;
unset SSH_AGENT_PID;
echo Agent pid 5443 killed;
[ssh-agent] Stopped.
Build was aborted
[PostBuildScript] - [INFO] Executing post build scripts.
[builder-openstack-cron] $ /bin/bash /tmp/jenkins13484702930405596251.sh
/tmp/jenkins10203664442967042860.sh: line 129:  7086 Terminated              lftools openstack --os-cloud "${os_cloud}" stack delete --force "${STACK_NAME}"
+ for STACK_NAME in "${OS_STACKS[@]}"
+ [[ 0 -gt 0 ]]
+ stack_in_jenkins releng-daexim-csit-1node-basic-only-potassium-442 https://jenkins.opendaylight.org/releng https://jenkins.opendaylight.org/sandbox
+ STACK_NAME=releng-daexim-csit-1node-basic-only-potassium-442
+ builds=()
+ for jenkins in "${@:2}"
+ PARAMS='tree=computer[executors[currentExecutable[url]],'
+ PARAMS='tree=computer[executors[currentExecutable[url]],oneOffExecutors[currentExecutable[url]]]'
+ PARAMS='tree=computer[executors[currentExecutable[url]],oneOffExecutors[currentExecutable[url]]]&xpath=//url&wrapper=builds'
+ JENKINS_URL='https://jenkins.opendaylight.org/releng/computer/api/json?tree=computer[executors[currentExecutable[url]],oneOffExecutors[currentExecutable[url]]]&xpath=//url&wrapper=builds'
++ curl -s -w '\n\n%{http_code}' --globoff -H Content-Type:application/json 'https://jenkins.opendaylight.org/releng/computer/api/json?tree=computer[executors[currentExecutable[url]],oneOffExecutors[currentExecutable[url]]]&xpath=//url&wrapper=builds'
---> sysstat.sh
(23) Failed writing body
[builder-openstack-cron] $ /bin/bash /tmp/jenkins15549544668377119124.sh
---> package-listing.sh
++ facter osfamily
++ tr '[:upper:]' '[:lower:]'
+ OS_FAMILY=redhat
+ workspace=/w/workspace/builder-openstack-cron
+ START_PACKAGES=/tmp/packages_start.txt
+ END_PACKAGES=/tmp/packages_end.txt
+ DIFF_PACKAGES=/tmp/packages_diff.txt
+ PACKAGES=/tmp/packages_start.txt
+ '[' /w/workspace/builder-openstack-cron ']'
+ PACKAGES=/tmp/packages_end.txt
+ case "${OS_FAMILY}" in
+ rpm -qa
+ sort
+ '[' -f /tmp/packages_start.txt ']'
+ '[' -f /tmp/packages_end.txt ']'
+ diff /tmp/packages_start.txt /tmp/packages_end.txt
+ '[' /w/workspace/builder-openstack-cron ']'
+ mkdir -p /w/workspace/builder-openstack-cron/archives/
+ cp -f /tmp/packages_diff.txt /tmp/packages_end.txt /tmp/packages_start.txt /w/workspace/builder-openstack-cron/archives/
[builder-openstack-cron] $ /bin/bash /tmp/jenkins7202407690942010145.sh
---> capture-instance-metadata.sh
Setup pyenv:
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/builder-openstack-cron/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-ewQY from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing: lftools 
lf-activate-venv(): INFO: Adding /tmp/venv-ewQY/bin to PATH
INFO: Running in OpenStack, capturing instance metadata
[builder-openstack-cron] $ /bin/bash /tmp/jenkins11161679989081996952.sh
provisioning config files...
Could not find credentials [logs] for builder-openstack-cron #65171
copy managed file [jenkins-log-archives-settings] to file:/w/workspace/builder-openstack-cron@tmp/config160005952263891452tmp
Regular expression run condition: Expression=[^.*logs-s3.*], Label=[odl-logs-s3-cloudfront-index]
Run condition [Regular expression match] enabling perform for step [Provide Configuration files]
provisioning config files...
copy managed file [jenkins-s3-log-ship] to file:/home/jenkins/.aws/credentials
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties content 
SERVER_ID=logs

[EnvInject] - Variables injected successfully.
[builder-openstack-cron] $ /bin/bash /tmp/jenkins3024304414501876773.sh
---> create-netrc.sh
WARN: Log server credential not found.
[builder-openstack-cron] $ /bin/bash /tmp/jenkins17815550368270184985.sh
---> python-tools-install.sh
Setup pyenv:
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/builder-openstack-cron/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-ewQY from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing: lftools 
lf-activate-venv(): INFO: Adding /tmp/venv-ewQY/bin to PATH
[builder-openstack-cron] $ /bin/bash /tmp/jenkins12505745533291393934.sh
---> sudo-logs.sh
Archiving 'sudo' log..
[builder-openstack-cron] $ /bin/bash /tmp/jenkins3519535709286191734.sh
---> job-cost.sh
Setup pyenv:
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/builder-openstack-cron/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-ewQY from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing: zipp==1.1.0 python-openstackclient urllib3~=1.26.15 
lf-activate-venv(): INFO: Adding /tmp/venv-ewQY/bin to PATH
INFO: No Stack...
INFO: Retrieving Pricing Info for: v3-standard-2
INFO: Archiving Costs
[builder-openstack-cron] $ /bin/bash -l /tmp/jenkins2449552607211615421.sh
---> logs-deploy.sh
Setup pyenv:
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/builder-openstack-cron/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-ewQY from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing: lftools 
lf-activate-venv(): INFO: Adding /tmp/venv-ewQY/bin to PATH
WARNING: Nexus logging server not set
INFO: S3 path logs/releng/vex-yul-odl-jenkins-1/builder-openstack-cron/65171/
INFO: archiving logs to S3
---> uname -a:
Linux prd-centos8-builder-2c-2g-25898.novalocal 4.18.0-553.5.1.el8.x86_64 #1 SMP Tue May 21 05:46:01 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux


---> lscpu:
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              2
On-line CPU(s) list: 0,1
Thread(s) per core:  1
Core(s) per socket:  1
Socket(s):           2
NUMA node(s):        1
Vendor ID:           AuthenticAMD
CPU family:          23
Model:               49
Model name:          AMD EPYC-Rome Processor
Stepping:            0
CPU MHz:             2800.000
BogoMIPS:            5600.00
Virtualization:      AMD-V
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           32K
L1i cache:           32K
L2 cache:            512K
L3 cache:            16384K
NUMA node0 CPU(s):   0,1
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr wbnoinvd arat npt nrip_save umip rdpid arch_capabilities


---> nproc:
2


---> df -h:
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        3.8G     0  3.8G   0% /dev
tmpfs           3.8G     0  3.8G   0% /dev/shm
tmpfs           3.8G   17M  3.8G   1% /run
tmpfs           3.8G     0  3.8G   0% /sys/fs/cgroup
/dev/vda1        40G  8.4G   32G  21% /
tmpfs           770M     0  770M   0% /run/user/1001


---> free -m:
              total        used        free      shared  buff/cache   available
Mem:           7697         649        4800          19        2247        6753
Swap:          1023           0        1023


---> ip addr:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1458 qdisc mq state UP group default qlen 1000
    link/ether fa:16:3e:e5:f8:d9 brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    altname ens3
    inet 10.30.170.236/23 brd 10.30.171.255 scope global dynamic noprefixroute eth0
       valid_lft 85701sec preferred_lft 85701sec
    inet6 fe80::f816:3eff:fee5:f8d9/64 scope link 
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1458 qdisc noqueue state DOWN group default 
    link/ether 02:42:74:df:d8:85 brd ff:ff:ff:ff:ff:ff
    inet 10.250.0.254/24 brd 10.250.0.255 scope global docker0
       valid_lft forever preferred_lft forever


---> sar -b -r -n DEV:
Linux 4.18.0-553.5.1.el8.x86_64 (centos-stream-8-builder-5e6ee0fb-b6d9-4116-90d4-35c890e2b0f7.nov) 	09/22/2024 	_x86_64_	(2 CPU)

18:31:32     LINUX RESTART	(2 CPU)

06:32:01 PM       tps      rtps      wtps   bread/s   bwrtn/s
06:33:01 PM    161.46     37.54    123.93   8460.25  20579.31
06:34:01 PM    115.96      0.20    115.76     17.20  11479.04
06:35:01 PM     17.58      0.70     16.88     71.31   1530.67
06:36:01 PM      0.40      0.00      0.40      0.00      4.32
06:37:01 PM      0.37      0.00      0.37      0.00      3.82
06:38:01 PM      0.53      0.00      0.53      0.00      7.61
06:39:01 PM      0.33      0.00      0.33      0.00      3.33
06:40:01 PM      0.45      0.00      0.45      0.00      5.83
06:41:01 PM      0.37      0.00      0.37      0.00      4.37
06:42:01 PM      0.37      0.00      0.37      0.00      4.08
06:43:01 PM      8.37      0.30      8.07     36.13    141.76
Average:        27.84      3.52     24.31    780.51   3069.57

06:32:01 PM kbmemfree   kbavail kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit  kbactive   kbinact   kbdirty
06:33:01 PM   5126600   6947884   2755832     34.96      2688   2010288   1041876     11.67    239952   2216068    130888
06:34:01 PM   4982204   6973768   2900228     36.79      2688   2168360    958732     10.73    235508   2313380     34100
06:35:01 PM   4936908   6930004   2945524     37.37      2688   2170456   1016796     11.39    251924   2343984         4
06:36:01 PM   4934020   6927116   2948412     37.40      2688   2170456   1016796     11.39    251924   2347100         4
06:37:01 PM   4931804   6924904   2950628     37.43      2688   2170460   1016796     11.39    251924   2349588         4
06:38:01 PM   4931684   6924784   2950748     37.43      2688   2170460   1016796     11.39    251924   2349680         4
06:39:01 PM   4931828   6924924   2950604     37.43      2688   2170460   1016796     11.39    251924   2349620         4
06:40:01 PM   4931556   6924656   2950876     37.44      2688   2170464   1037620     11.62    251924   2349980         4
06:41:01 PM   4932180   6925316   2950252     37.43      2688   2170464   1015460     11.37    251924   2349540         4
06:42:01 PM   4931948   6925088   2950484     37.43      2688   2170468   1015460     11.37    251924   2349960         4
06:43:01 PM   5005012   6999348   2877420     36.50      2688   2171720    999168     11.19    409380   2119824        36
Average:      4961431   6938890   2921001     37.06      2688   2155823   1013845     11.35    263657   2312611     15005

06:32:01 PM     IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
06:33:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
06:33:01 PM      eth0    302.08    162.81   1693.90     60.93      0.00      0.00      0.00      0.00
06:33:01 PM   docker0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
06:34:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
06:34:01 PM      eth0     55.17     46.36    614.86      8.58      0.00      0.00      0.00      0.00
06:34:01 PM   docker0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
06:35:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
06:35:01 PM      eth0      3.93      3.40      2.65      0.87      0.00      0.00      0.00      0.00
06:35:01 PM   docker0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
06:36:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
06:36:01 PM      eth0      0.57      0.38      0.34      0.14      0.00      0.00      0.00      0.00
06:36:01 PM   docker0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
06:37:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
06:37:01 PM      eth0      0.62      0.42      0.35      0.15      0.00      0.00      0.00      0.00
06:37:01 PM   docker0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
06:38:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
06:38:01 PM      eth0      0.78      0.53      0.47      0.21      0.00      0.00      0.00      0.00
06:38:01 PM   docker0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
06:39:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
06:39:01 PM      eth0      0.52      0.48      0.41      0.17      0.00      0.00      0.00      0.00
06:39:01 PM   docker0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
06:40:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
06:40:01 PM      eth0      0.68      0.35      0.30      0.12      0.00      0.00      0.00      0.00
06:40:01 PM   docker0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
06:41:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
06:41:01 PM      eth0      0.75      0.40      0.38      0.15      0.00      0.00      0.00      0.00
06:41:01 PM   docker0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
06:42:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
06:42:01 PM      eth0      0.92      0.58      0.60      0.34      0.00      0.00      0.00      0.00
06:42:01 PM   docker0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
06:43:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
06:43:01 PM      eth0     13.93     11.31     11.41      6.21      0.00      0.00      0.00      0.00
06:43:01 PM   docker0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
Average:           lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
Average:         eth0     34.54     20.64    211.43      7.08      0.00      0.00      0.00      0.00
Average:      docker0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00


---> sar -P ALL:
Linux 4.18.0-553.5.1.el8.x86_64 (centos-stream-8-builder-5e6ee0fb-b6d9-4116-90d4-35c890e2b0f7.nov) 	09/22/2024 	_x86_64_	(2 CPU)

18:31:32     LINUX RESTART	(2 CPU)

06:32:01 PM     CPU     %user     %nice   %system   %iowait    %steal     %idle
06:33:01 PM     all     43.19      0.00      8.74      6.13      0.11     41.83
06:33:01 PM       0     46.14      0.00      8.47      4.28      0.10     41.00
06:33:01 PM       1     40.25      0.00      9.02      7.96      0.12     42.65
06:34:01 PM     all     35.25      0.00      5.44      3.33      0.13     55.85
06:34:01 PM       0     37.25      0.00      5.53      3.37      0.13     53.71
06:34:01 PM       1     33.24      0.00      5.35      3.29      0.12     57.99
06:35:01 PM     all      2.17      0.00      0.54      0.61      0.03     96.64
06:35:01 PM       0      1.19      0.00      0.32      0.72      0.03     97.74
06:35:01 PM       1      3.16      0.00      0.77      0.50      0.03     95.54
06:36:01 PM     all      0.39      0.00      0.09      0.00      0.02     99.50
06:36:01 PM       0      0.57      0.00      0.10      0.00      0.00     99.33
06:36:01 PM       1      0.22      0.00      0.08      0.00      0.03     99.67
06:37:01 PM     all      0.27      0.00      0.09      0.00      0.03     99.61
06:37:01 PM       0      0.25      0.00      0.10      0.00      0.02     99.63
06:37:01 PM       1      0.28      0.00      0.08      0.00      0.05     99.58
06:38:01 PM     all      0.41      0.00      0.07      0.00      0.03     99.50
06:38:01 PM       0      0.20      0.00      0.05      0.00      0.02     99.73
06:38:01 PM       1      0.62      0.00      0.08      0.00      0.03     99.27
06:39:01 PM     all      0.43      0.00      0.04      0.00      0.03     99.51
06:39:01 PM       0      0.17      0.00      0.07      0.00      0.02     99.75
06:39:01 PM       1      0.68      0.00      0.02      0.00      0.03     99.27
06:40:01 PM     all      0.27      0.00      0.73      0.00      0.06     98.94
06:40:01 PM       0      0.18      0.00      1.35      0.00      0.10     98.36
06:40:01 PM       1      0.36      0.00      0.09      0.00      0.02     99.54
06:41:01 PM     all      0.42      0.00      0.07      0.00      0.02     99.50
06:41:01 PM       0      0.28      0.00      0.07      0.00      0.02     99.63
06:41:01 PM       1      0.55      0.00      0.07      0.00      0.02     99.37
06:42:01 PM     all      0.32      0.00      0.06      0.00      0.03     99.60
06:42:01 PM       0      0.15      0.00      0.07      0.00      0.02     99.77
06:42:01 PM       1      0.49      0.00      0.05      0.00      0.03     99.43
06:43:01 PM     all     14.94      0.00      1.61      0.76      0.06     82.63
06:43:01 PM       0     18.13      0.00      1.47      1.45      0.07     78.88
06:43:01 PM       1     11.74      0.00      1.74      0.07      0.05     86.39

Average:        CPU     %user     %nice   %system   %iowait    %steal     %idle
Average:        all      8.91      0.00      1.59      0.98      0.05     88.48
Average:          0      9.48      0.00      1.59      0.89      0.05     87.99
Average:          1      8.33      0.00      1.58      1.08      0.05     88.96