Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on prd-centos8-builder-2c-2g-30955 (centos8-builder-2c-2g) in workspace /w/workspace/builder-openstack-cron [ssh-agent] Looking for ssh-agent implementation... [ssh-agent] Exec ssh-agent (binary ssh-agent on a remote machine) $ ssh-agent SSH_AUTH_SOCK=/tmp/ssh-rv48gG37fTT8/agent.5441 SSH_AGENT_PID=5442 [ssh-agent] Started. Running ssh-add (command line suppressed) Identity added: /w/workspace/builder-openstack-cron@tmp/private_key_7381997049514356701.key (/w/workspace/builder-openstack-cron@tmp/private_key_7381997049514356701.key) [ssh-agent] Using credentials jenkins (jenkins-ssh) The recommended git tool is: NONE using credential jenkins-ssh Wiping out workspace first. Cloning the remote Git repository Cloning repository git://devvexx.opendaylight.org/mirror/releng/builder > git init /w/workspace/builder-openstack-cron # timeout=10 Fetching upstream changes from git://devvexx.opendaylight.org/mirror/releng/builder > git --version # timeout=10 > git --version # 'git version 2.43.0' using GIT_SSH to set credentials jenkins-ssh [INFO] Currently running in a labeled security context [INFO] Currently SELinux is 'enforcing' on the host > /usr/bin/chcon --type=ssh_home_t /w/workspace/builder-openstack-cron@tmp/jenkins-gitclient-ssh9940273942152460427.key Verifying host key using known hosts file You're using 'Known hosts file' strategy to verify ssh host keys, but your known_hosts file does not exist, please go to 'Manage Jenkins' -> 'Security' -> 'Git Host Key Verification Configuration' and configure host key verification. > git fetch --tags --force --progress -- git://devvexx.opendaylight.org/mirror/releng/builder +refs/heads/*:refs/remotes/origin/* # timeout=10 > git config remote.origin.url git://devvexx.opendaylight.org/mirror/releng/builder # timeout=10 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 Avoid second fetch > git rev-parse refs/remotes/origin/master^{commit} # timeout=10 Checking out Revision 1d63b900750173f3a58231e6e4d029581de2b124 (refs/remotes/origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 1d63b900750173f3a58231e6e4d029581de2b124 # timeout=10 Commit message: "Chore: Add Ubuntu 20.04 and 22.04 jobs for mininet" > git rev-list --no-walk 1d63b900750173f3a58231e6e4d029581de2b124 # timeout=10 > git remote # timeout=10 > git submodule init # timeout=10 > git submodule sync # timeout=10 > git config --get remote.origin.url # timeout=10 > git submodule init # timeout=10 > git config -f .gitmodules --get-regexp ^submodule\.(.+)\.url # timeout=10 > git config --get submodule.global-jjb.url # timeout=10 > git config -f .gitmodules --get submodule.global-jjb.path # timeout=10 > git config --get submodule.packer/common-packer.url # timeout=10 > git config -f .gitmodules --get submodule.packer/common-packer.path # timeout=10 > git submodule update --init --recursive global-jjb # timeout=10 > git submodule update --init --recursive packer/common-packer # timeout=10 Run condition [Boolean condition] enabling prebuild for step [Execute shell] Run condition [Boolean condition] enabling prebuild for step [Execute shell] Run condition [Boolean condition] enabling prebuild for step [Execute shell] Run condition [Boolean condition] enabling prebuild for step [Execute shell] Run condition [Boolean condition] enabling prebuild for step [Execute shell] Run condition [Boolean condition] enabling prebuild for step [Execute shell] Run condition [Boolean condition] enabling prebuild for step [BuilderChain] No emails were triggered. provisioning config files... copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml copy managed file [npmrc] to file:/home/jenkins/.npmrc copy managed file [pipconf] to file:/home/jenkins/.config/pip/pip.conf [builder-openstack-cron] $ /bin/bash /tmp/jenkins355310688084610819.sh ---> python-tools-install.sh Setup pyenv: system * 3.8.13 (set by /opt/pyenv/version) * 3.9.13 (set by /opt/pyenv/version) * 3.10.13 (set by /opt/pyenv/version) * 3.11.7 (set by /opt/pyenv/version) lf-activate-venv(): INFO: Creating python3 venv at /tmp/venv-2EqV lf-activate-venv(): INFO: Save venv in file: /tmp/.os_lf_venv lf-activate-venv(): INFO: Installing: lftools lf-activate-venv(): INFO: Adding /tmp/venv-2EqV/bin to PATH Generating Requirements File Python 3.11.7 pip 24.2 from /tmp/venv-2EqV/lib/python3.11/site-packages/pip (python 3.11) appdirs==1.4.4 argcomplete==3.5.0 aspy.yaml==1.3.0 attrs==24.2.0 autopage==0.5.2 beautifulsoup4==4.12.3 boto3==1.35.29 botocore==1.35.29 bs4==0.0.2 cachetools==5.5.0 certifi==2024.8.30 cffi==1.17.1 cfgv==3.4.0 chardet==5.2.0 charset-normalizer==3.3.2 click==8.1.7 cliff==4.7.0 cmd2==2.4.3 cryptography==3.3.2 debtcollector==3.0.0 decorator==5.1.1 defusedxml==0.7.1 Deprecated==1.2.14 distlib==0.3.8 dnspython==2.6.1 docker==4.2.2 dogpile.cache==1.3.3 durationpy==0.7 email_validator==2.2.0 filelock==3.16.1 future==1.0.0 gitdb==4.0.11 GitPython==3.1.43 google-auth==2.35.0 httplib2==0.22.0 identify==2.6.1 idna==3.10 importlib-resources==1.5.0 iso8601==2.1.0 Jinja2==3.1.4 jmespath==1.0.1 jsonpatch==1.33 jsonpointer==3.0.0 jsonschema==4.23.0 jsonschema-specifications==2023.12.1 keystoneauth1==5.8.0 kubernetes==31.0.0 lftools==0.37.10 lxml==5.3.0 MarkupSafe==2.1.5 msgpack==1.1.0 multi_key_dict==2.0.3 munch==4.0.0 netaddr==1.3.0 netifaces==0.11.0 niet==1.4.2 nodeenv==1.9.1 oauth2client==4.1.3 oauthlib==3.2.2 openstacksdk==4.0.0 os-client-config==2.1.0 os-service-types==1.7.0 osc-lib==3.1.0 oslo.config==9.6.0 oslo.context==5.6.0 oslo.i18n==6.4.0 oslo.log==6.1.2 oslo.serialization==5.5.0 oslo.utils==7.3.0 packaging==24.1 pbr==6.1.0 platformdirs==4.3.6 prettytable==3.11.0 pyasn1==0.6.1 pyasn1_modules==0.4.1 pycparser==2.22 pygerrit2==2.0.15 PyGithub==2.4.0 PyJWT==2.9.0 PyNaCl==1.5.0 pyparsing==2.4.7 pyperclip==1.9.0 pyrsistent==0.20.0 python-cinderclient==9.6.0 python-dateutil==2.9.0.post0 python-heatclient==4.0.0 python-jenkins==1.8.2 python-keystoneclient==5.5.0 python-magnumclient==4.7.0 python-openstackclient==7.1.2 python-swiftclient==4.6.0 PyYAML==6.0.2 referencing==0.35.1 requests==2.32.3 requests-oauthlib==2.0.0 requestsexceptions==1.4.0 rfc3986==2.0.0 rpds-py==0.20.0 rsa==4.9 ruamel.yaml==0.18.6 ruamel.yaml.clib==0.2.8 s3transfer==0.10.2 simplejson==3.19.3 six==1.16.0 smmap==5.0.1 soupsieve==2.6 stevedore==5.3.0 tabulate==0.9.0 toml==0.10.2 tomlkit==0.13.2 tqdm==4.66.5 typing_extensions==4.12.2 tzdata==2024.2 urllib3==1.26.20 virtualenv==20.26.6 wcwidth==0.2.13 websocket-client==1.8.0 wrapt==1.16.0 xdg==6.0.0 xmltodict==0.13.0 yq==3.4.3 [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties content OS_CLOUD=vex [EnvInject] - Variables injected successfully. [Boolean condition] checking [True] against [^(1|y|yes|t|true|on|run)$] (origin token: True) Run condition [Boolean condition] enabling perform for step [Execute shell] [builder-openstack-cron] $ /bin/bash -l /tmp/jenkins8626271671875942775.sh ---> Orphaned k8s clusters Setup pyenv: system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/builder-openstack-cron/.python-version) lf-activate-venv(): INFO: Reuse venv:/tmp/venv-2EqV from file:/tmp/.os_lf_venv lf-activate-venv(): INFO: Installing: kubernetes python-heatclient python-openstackclient python-magnumclient ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. lftools 0.37.10 requires urllib3<2.1.0, but you have urllib3 2.2.3 which is incompatible. lf-activate-venv(): INFO: Adding /tmp/venv-2EqV/bin to PATH -----> Delete orphaned cluster [Boolean condition] checking [True] against [^(1|y|yes|t|true|on|run)$] (origin token: True) Run condition [Boolean condition] enabling perform for step [Execute shell] [builder-openstack-cron] $ /bin/bash -l /tmp/jenkins7874002827296299710.sh ---> Orphaned stacks Setup pyenv: system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/builder-openstack-cron/.python-version) lf-activate-venv(): INFO: Reuse venv:/tmp/venv-2EqV from file:/tmp/.os_lf_venv lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq lf-activate-venv(): INFO: Adding /tmp/venv-2EqV/bin to PATH + mapfile -t OS_COE_CLUSTERS_ID ++ openstack --os-cloud vex coe cluster list -f value -c uuid -c name ++ awk '{print $1}' ++ grep -E '(DELETE_FAILED|UNKNOWN|UNHEALTHY)' + echo '-----> Active clusters -> stacks' -----> Active clusters -> stacks + OS_COE_STACKS=() + [[ 0 -gt 0 ]] + mapfile -t OS_STACKS ++ openstack --os-cloud vex stack list -f value -c 'Stack Name' -c 'Stack Status' --property stack_status=CREATE_COMPLETE --property stack_status=DELETE_FAILED --property stack_status=CREATE_FAILED ++ awk '{print $1}' + echo '-----> Active stacks' -----> Active stacks + for stack in "${OS_STACKS[@]}" + echo releng-openflowplugin-csit-1node-perf-bulkomatic-only-potassium-400 releng-openflowplugin-csit-1node-perf-bulkomatic-only-potassium-400 + echo '-----> Delete orphaned stacks' -----> Delete orphaned stacks + for STACK_NAME in "${OS_STACKS[@]}" + [[ 0 -gt 0 ]] + stack_in_jenkins releng-openflowplugin-csit-1node-perf-bulkomatic-only-potassium-400 https://jenkins.opendaylight.org/releng https://jenkins.opendaylight.org/sandbox + STACK_NAME=releng-openflowplugin-csit-1node-perf-bulkomatic-only-potassium-400 + builds=() + for jenkins in "${@:2}" + PARAMS='tree=computer[executors[currentExecutable[url]],' + PARAMS='tree=computer[executors[currentExecutable[url]],oneOffExecutors[currentExecutable[url]]]' + PARAMS='tree=computer[executors[currentExecutable[url]],oneOffExecutors[currentExecutable[url]]]&xpath=//url&wrapper=builds' + JENKINS_URL='https://jenkins.opendaylight.org/releng/computer/api/json?tree=computer[executors[currentExecutable[url]],oneOffExecutors[currentExecutable[url]]]&xpath=//url&wrapper=builds' ++ curl -s -w '\n\n%{http_code}' --globoff -H Content-Type:application/json 'https://jenkins.opendaylight.org/releng/computer/api/json?tree=computer[executors[currentExecutable[url]],oneOffExecutors[currentExecutable[url]]]&xpath=//url&wrapper=builds' + resp='{"_class":"hudson.model.ComputerSet","computer":[{"_class":"hudson.model.Hudson$MasterComputer","executors":[],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":{"_class":"hudson.model.FreeStyleBuild","url":"https://jenkins.opendaylight.org/releng/job/builder-openstack-cron/65362/"}}],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":{"_class":"hudson.model.FreeStyleBuild","url":"https://jenkins.opendaylight.org/releng/job/ovsdb-maven-merge-titanium/42/"}}],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":null}],"oneOffExecutors":[]}]} 200' ++ echo '{"_class":"hudson.model.ComputerSet","computer":[{"_class":"hudson.model.Hudson$MasterComputer","executors":[],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":{"_class":"hudson.model.FreeStyleBuild","url":"https://jenkins.opendaylight.org/releng/job/builder-openstack-cron/65362/"}}],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":{"_class":"hudson.model.FreeStyleBuild","url":"https://jenkins.opendaylight.org/releng/job/ovsdb-maven-merge-titanium/42/"}}],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":null}],"oneOffExecutors":[]}]} 200' ++ head -n1 + json_data='{"_class":"hudson.model.ComputerSet","computer":[{"_class":"hudson.model.Hudson$MasterComputer","executors":[],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":{"_class":"hudson.model.FreeStyleBuild","url":"https://jenkins.opendaylight.org/releng/job/builder-openstack-cron/65362/"}}],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":{"_class":"hudson.model.FreeStyleBuild","url":"https://jenkins.opendaylight.org/releng/job/ovsdb-maven-merge-titanium/42/"}}],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":null}],"oneOffExecutors":[]}]}' ++ echo '{"_class":"hudson.model.ComputerSet","computer":[{"_class":"hudson.model.Hudson$MasterComputer","executors":[],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":{"_class":"hudson.model.FreeStyleBuild","url":"https://jenkins.opendaylight.org/releng/job/builder-openstack-cron/65362/"}}],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":{"_class":"hudson.model.FreeStyleBuild","url":"https://jenkins.opendaylight.org/releng/job/ovsdb-maven-merge-titanium/42/"}}],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":null}],"oneOffExecutors":[]}]} 200' ++ awk 'END {print $NF}' + status=200 + '[' 200 '!=' 200 ']' + [[ https://jenkins.opendaylight.org/releng == *\j\e\n\k\i\n\s\.*\.\o\r\g ]] + [[ https://jenkins.opendaylight.org/releng == *\j\e\n\k\i\n\s\.*\.\i\o ]] ++ echo https://jenkins.opendaylight.org/releng ++ awk -F/ '{print $NF}' ++ sed 's/\/*$//' + silo=releng + export silo + builds=(${builds[@]} $(echo "$json_data" | jq -r '.computer[].executors[].currentExecutable.url' | grep -v null | awk -F'/' '{print ENVIRON["silo"] "-" $6 "-" $7}')) ++ echo '{"_class":"hudson.model.ComputerSet","computer":[{"_class":"hudson.model.Hudson$MasterComputer","executors":[],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":{"_class":"hudson.model.FreeStyleBuild","url":"https://jenkins.opendaylight.org/releng/job/builder-openstack-cron/65362/"}}],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":{"_class":"hudson.model.FreeStyleBuild","url":"https://jenkins.opendaylight.org/releng/job/ovsdb-maven-merge-titanium/42/"}}],"oneOffExecutors":[]},{"_class":"jenkins.plugins.openstack.compute.JCloudsComputer","executors":[{"currentExecutable":null}],"oneOffExecutors":[]}]}' ++ awk -F/ '{print ENVIRON["silo"] "-" $6 "-" $7}' ++ jq -r '.computer[].executors[].currentExecutable.url' ++ grep -v null + for jenkins in "${@:2}" + PARAMS='tree=computer[executors[currentExecutable[url]],' + PARAMS='tree=computer[executors[currentExecutable[url]],oneOffExecutors[currentExecutable[url]]]' + PARAMS='tree=computer[executors[currentExecutable[url]],oneOffExecutors[currentExecutable[url]]]&xpath=//url&wrapper=builds' + JENKINS_URL='https://jenkins.opendaylight.org/sandbox/computer/api/json?tree=computer[executors[currentExecutable[url]],oneOffExecutors[currentExecutable[url]]]&xpath=//url&wrapper=builds' ++ curl -s -w '\n\n%{http_code}' --globoff -H Content-Type:application/json 'https://jenkins.opendaylight.org/sandbox/computer/api/json?tree=computer[executors[currentExecutable[url]],oneOffExecutors[currentExecutable[url]]]&xpath=//url&wrapper=builds' + resp='{"_class":"hudson.model.ComputerSet","computer":[{"_class":"hudson.model.Hudson$MasterComputer","executors":[],"oneOffExecutors":[]}]} 200' ++ echo '{"_class":"hudson.model.ComputerSet","computer":[{"_class":"hudson.model.Hudson$MasterComputer","executors":[],"oneOffExecutors":[]}]} 200' ++ head -n1 + json_data='{"_class":"hudson.model.ComputerSet","computer":[{"_class":"hudson.model.Hudson$MasterComputer","executors":[],"oneOffExecutors":[]}]}' ++ echo '{"_class":"hudson.model.ComputerSet","computer":[{"_class":"hudson.model.Hudson$MasterComputer","executors":[],"oneOffExecutors":[]}]} 200' ++ awk 'END {print $NF}' + status=200 + '[' 200 '!=' 200 ']' + [[ https://jenkins.opendaylight.org/sandbox == *\j\e\n\k\i\n\s\.*\.\o\r\g ]] + [[ https://jenkins.opendaylight.org/sandbox == *\j\e\n\k\i\n\s\.*\.\i\o ]] ++ echo https://jenkins.opendaylight.org/sandbox ++ sed 's/\/*$//' ++ awk -F/ '{print $NF}' + silo=sandbox + export silo + builds=(${builds[@]} $(echo "$json_data" | jq -r '.computer[].executors[].currentExecutable.url' | grep -v null | awk -F'/' '{print ENVIRON["silo"] "-" $6 "-" $7}')) ++ grep -v null ++ echo '{"_class":"hudson.model.ComputerSet","computer":[{"_class":"hudson.model.Hudson$MasterComputer","executors":[],"oneOffExecutors":[]}]}' ++ jq -r '.computer[].executors[].currentExecutable.url' ++ awk -F/ '{print ENVIRON["silo"] "-" $6 "-" $7}' + [[ releng-builder-openstack-cron-65362 releng-ovsdb-maven-merge-titanium-42 =~ releng-openflowplugin-csit-1node-perf-bulkomatic-only-potassium-400 ]] + return 1 + echo 'Deleting orphaned stack: releng-openflowplugin-csit-1node-perf-bulkomatic-only-potassium-400' Deleting orphaned stack: releng-openflowplugin-csit-1node-perf-bulkomatic-only-potassium-400 + lftools openstack --os-cloud vex stack delete --force releng-openflowplugin-csit-1node-perf-bulkomatic-only-potassium-400 Deleting stack releng-openflowplugin-csit-1node-perf-bulkomatic-only-potassium-400 Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Waiting for stack to delete... Successfully deleted stack releng-openflowplugin-csit-1node-perf-bulkomatic-only-potassium-400 [Boolean condition] checking [True] against [^(1|y|yes|t|true|on|run)$] (origin token: True) Run condition [Boolean condition] enabling perform for step [Execute shell] [builder-openstack-cron] $ /bin/bash -l /tmp/jenkins12432255600846959997.sh ---> Orphaned servers Setup pyenv: system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/builder-openstack-cron/.python-version) lf-activate-venv(): INFO: Reuse venv:/tmp/venv-2EqV from file:/tmp/.os_lf_venv lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq Build timed out (after 10 minutes). Marking the build as failed. $ ssh-agent -k unset SSH_AUTH_SOCK; unset SSH_AGENT_PID; echo Agent pid 5442 killed; [ssh-agent] Stopped. Build was aborted [PostBuildScript] - [INFO] Executing post build scripts. /home/jenkins/lf-env.sh: line 187: 7449 Terminated "$lf_venv/bin/python3" -m pip install --upgrade --quiet --upgrade-strategy eager $pkg_list /tmp/jenkins12432255600846959997.sh: line 67: openstack: command not found -----> Active servers -----> Delete orphaned servers [builder-openstack-cron] $ /bin/bash /tmp/jenkins5419345009041698440.sh ---> sysstat.sh [builder-openstack-cron] $ /bin/bash /tmp/jenkins13102554152635985762.sh ---> package-listing.sh ++ tr '[:upper:]' '[:lower:]' ++ facter osfamily + OS_FAMILY=redhat + workspace=/w/workspace/builder-openstack-cron + START_PACKAGES=/tmp/packages_start.txt + END_PACKAGES=/tmp/packages_end.txt + DIFF_PACKAGES=/tmp/packages_diff.txt + PACKAGES=/tmp/packages_start.txt + '[' /w/workspace/builder-openstack-cron ']' + PACKAGES=/tmp/packages_end.txt + case "${OS_FAMILY}" in + rpm -qa + sort + '[' -f /tmp/packages_start.txt ']' + '[' -f /tmp/packages_end.txt ']' + diff /tmp/packages_start.txt /tmp/packages_end.txt + '[' /w/workspace/builder-openstack-cron ']' + mkdir -p /w/workspace/builder-openstack-cron/archives/ + cp -f /tmp/packages_diff.txt /tmp/packages_end.txt /tmp/packages_start.txt /w/workspace/builder-openstack-cron/archives/ [builder-openstack-cron] $ /bin/bash /tmp/jenkins743110062866042527.sh ---> capture-instance-metadata.sh Setup pyenv: system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/builder-openstack-cron/.python-version) lf-activate-venv(): INFO: Reuse venv:/tmp/venv-2EqV from file:/tmp/.os_lf_venv lf-activate-venv(): INFO: Installing: lftools lf-activate-venv(): INFO: Adding /tmp/venv-2EqV/bin to PATH INFO: Running in OpenStack, capturing instance metadata [builder-openstack-cron] $ /bin/bash /tmp/jenkins16359533496902243256.sh provisioning config files... Could not find credentials [logs] for builder-openstack-cron #65362 copy managed file [jenkins-log-archives-settings] to file:/w/workspace/builder-openstack-cron@tmp/config6791131035208614629tmp Regular expression run condition: Expression=[^.*logs-s3.*], Label=[odl-logs-s3-cloudfront-index] Run condition [Regular expression match] enabling perform for step [Provide Configuration files] provisioning config files... copy managed file [jenkins-s3-log-ship] to file:/home/jenkins/.aws/credentials [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties content SERVER_ID=logs [EnvInject] - Variables injected successfully. [builder-openstack-cron] $ /bin/bash /tmp/jenkins9954315792433461311.sh ---> create-netrc.sh WARN: Log server credential not found. [builder-openstack-cron] $ /bin/bash /tmp/jenkins2492530623409838940.sh ---> python-tools-install.sh Setup pyenv: system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/builder-openstack-cron/.python-version) lf-activate-venv(): INFO: Reuse venv:/tmp/venv-2EqV from file:/tmp/.os_lf_venv lf-activate-venv(): INFO: Installing: lftools lf-activate-venv(): INFO: Adding /tmp/venv-2EqV/bin to PATH [builder-openstack-cron] $ /bin/bash /tmp/jenkins8556449570655273485.sh ---> sudo-logs.sh Archiving 'sudo' log.. [builder-openstack-cron] $ /bin/bash /tmp/jenkins5084057244607292383.sh ---> job-cost.sh Setup pyenv: system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/builder-openstack-cron/.python-version) lf-activate-venv(): INFO: Reuse venv:/tmp/venv-2EqV from file:/tmp/.os_lf_venv lf-activate-venv(): INFO: Installing: zipp==1.1.0 python-openstackclient urllib3~=1.26.15 lf-activate-venv(): INFO: Adding /tmp/venv-2EqV/bin to PATH INFO: No Stack... INFO: Retrieving Pricing Info for: v3-standard-2 INFO: Archiving Costs [builder-openstack-cron] $ /bin/bash -l /tmp/jenkins3109626265663669980.sh ---> logs-deploy.sh Setup pyenv: system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/builder-openstack-cron/.python-version) lf-activate-venv(): INFO: Reuse venv:/tmp/venv-2EqV from file:/tmp/.os_lf_venv lf-activate-venv(): INFO: Installing: lftools lf-activate-venv(): INFO: Adding /tmp/venv-2EqV/bin to PATH WARNING: Nexus logging server not set INFO: S3 path logs/releng/vex-yul-odl-jenkins-1/builder-openstack-cron/65362/ INFO: archiving logs to S3 ---> uname -a: Linux prd-centos8-builder-2c-2g-30955.novalocal 4.18.0-553.5.1.el8.x86_64 #1 SMP Tue May 21 05:46:01 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux ---> lscpu: Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 2 On-line CPU(s) list: 0,1 Thread(s) per core: 1 Core(s) per socket: 1 Socket(s): 2 NUMA node(s): 1 Vendor ID: AuthenticAMD CPU family: 23 Model: 49 Model name: AMD EPYC-Rome Processor Stepping: 0 CPU MHz: 2800.000 BogoMIPS: 5600.00 Virtualization: AMD-V Hypervisor vendor: KVM Virtualization type: full L1d cache: 32K L1i cache: 32K L2 cache: 512K L3 cache: 16384K NUMA node0 CPU(s): 0,1 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr wbnoinvd arat npt nrip_save umip rdpid arch_capabilities ---> nproc: 2 ---> df -h: Filesystem Size Used Avail Use% Mounted on devtmpfs 3.8G 0 3.8G 0% /dev tmpfs 3.8G 0 3.8G 0% /dev/shm tmpfs 3.8G 17M 3.8G 1% /run tmpfs 3.8G 0 3.8G 0% /sys/fs/cgroup /dev/vda1 40G 8.4G 32G 21% / tmpfs 770M 0 770M 0% /run/user/1001 ---> free -m: total used free shared buff/cache available Mem: 7697 633 4810 19 2254 6769 Swap: 1023 0 1023 ---> ip addr: 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: mtu 1458 qdisc mq state UP group default qlen 1000 link/ether fa:16:3e:2a:60:94 brd ff:ff:ff:ff:ff:ff altname enp0s3 altname ens3 inet 10.30.171.112/23 brd 10.30.171.255 scope global dynamic noprefixroute eth0 valid_lft 85702sec preferred_lft 85702sec inet6 fe80::f816:3eff:fe2a:6094/64 scope link valid_lft forever preferred_lft forever 3: docker0: mtu 1458 qdisc noqueue state DOWN group default link/ether 02:42:0a:b6:4b:3d brd ff:ff:ff:ff:ff:ff inet 10.250.0.254/24 brd 10.250.0.255 scope global docker0 valid_lft forever preferred_lft forever ---> sar -b -r -n DEV: Linux 4.18.0-553.5.1.el8.x86_64 (centos-stream-8-builder-5e6ee0fb-b6d9-4116-90d4-35c890e2b0f7.nov) 09/30/2024 _x86_64_ (2 CPU) 17:31:33 LINUX RESTART (2 CPU) 05:32:01 PM tps rtps wtps bread/s bwrtn/s 05:33:01 PM 132.99 26.06 106.93 6172.74 18345.95 05:34:01 PM 109.56 6.96 102.60 1264.78 8809.18 05:35:01 PM 36.94 0.70 36.24 71.32 5011.38 05:36:01 PM 4.82 0.00 4.82 0.00 180.22 05:37:01 PM 0.20 0.00 0.20 0.00 2.55 05:38:01 PM 0.33 0.00 0.33 0.00 5.52 05:39:01 PM 0.12 0.00 0.12 0.00 0.77 05:40:01 PM 0.10 0.00 0.10 0.00 1.18 05:41:01 PM 0.17 0.00 0.17 0.00 1.38 05:42:01 PM 0.20 0.00 0.20 0.00 2.38 05:43:01 PM 10.60 0.50 10.10 63.71 168.53 Average: 26.91 3.11 23.80 688.47 2957.33 05:32:01 PM kbmemfree kbavail kbmemused %memused kbbuffers kbcached kbcommit %commit kbactive kbinact kbdirty 05:33:01 PM 5263980 6988832 2618452 33.22 2688 1915880 974964 10.92 205000 2119596 83108 05:34:01 PM 5011668 6985556 2870764 36.42 2688 2151276 930232 10.42 226096 2297280 89536 05:35:01 PM 4949184 6948692 2933248 37.21 2688 2176496 964568 10.80 251576 2332840 8 05:36:01 PM 4948188 6947240 2934244 37.23 2688 2176496 964568 10.80 251576 2334776 4 05:37:01 PM 4942988 6942044 2939444 37.29 2688 2176500 964568 10.80 251576 2340124 4 05:38:01 PM 4942908 6941964 2939524 37.29 2688 2176500 964568 10.80 251576 2340200 4 05:39:01 PM 4943204 6942260 2939228 37.29 2688 2176500 980364 10.98 251580 2340140 4 05:40:01 PM 4943132 6942152 2939300 37.29 2688 2176504 983616 11.01 251580 2340264 4 05:41:01 PM 4943076 6942116 2939356 37.29 2688 2176508 983616 11.01 251580 2340192 4 05:42:01 PM 4943288 6942332 2939144 37.29 2688 2176512 983616 11.01 251580 2340176 4 05:43:01 PM 4968464 6972052 2913968 36.97 2688 2181056 949876 10.64 411448 2155704 1388 Average: 4981825 6954113 2900607 36.80 2688 2150930 967687 10.84 259561 2298299 15824 05:32:01 PM IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s %ifutil 05:33:01 PM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 05:33:01 PM eth0 297.52 181.11 1441.53 62.48 0.00 0.00 0.00 0.00 05:33:01 PM docker0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 05:34:01 PM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 05:34:01 PM eth0 47.93 40.04 491.40 6.94 0.00 0.00 0.00 0.00 05:34:01 PM docker0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 05:35:01 PM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 05:35:01 PM eth0 32.43 25.76 380.15 3.82 0.00 0.00 0.00 0.00 05:35:01 PM docker0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 05:36:01 PM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 05:36:01 PM eth0 0.58 0.35 0.34 0.12 0.00 0.00 0.00 0.00 05:36:01 PM docker0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 05:37:01 PM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 05:37:01 PM eth0 0.57 0.42 0.34 0.15 0.00 0.00 0.00 0.00 05:37:01 PM docker0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 05:38:01 PM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 05:38:01 PM eth0 0.62 0.52 0.47 0.21 0.00 0.00 0.00 0.00 05:38:01 PM docker0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 05:39:01 PM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 05:39:01 PM eth0 0.62 0.38 0.35 0.14 0.00 0.00 0.00 0.00 05:39:01 PM docker0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 05:40:01 PM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 05:40:01 PM eth0 0.55 0.42 0.34 0.15 0.00 0.00 0.00 0.00 05:40:01 PM docker0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 05:41:01 PM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 05:41:01 PM eth0 0.50 0.40 0.34 0.14 0.00 0.00 0.00 0.00 05:41:01 PM docker0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 05:42:01 PM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 05:42:01 PM eth0 0.57 0.38 0.34 0.14 0.00 0.00 0.00 0.00 05:42:01 PM docker0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 05:43:01 PM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 05:43:01 PM eth0 16.69 14.78 14.23 7.77 0.00 0.00 0.00 0.00 05:43:01 PM docker0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: eth0 36.24 24.05 211.81 7.46 0.00 0.00 0.00 0.00 Average: docker0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 ---> sar -P ALL: Linux 4.18.0-553.5.1.el8.x86_64 (centos-stream-8-builder-5e6ee0fb-b6d9-4116-90d4-35c890e2b0f7.nov) 09/30/2024 _x86_64_ (2 CPU) 17:31:33 LINUX RESTART (2 CPU) 05:32:01 PM CPU %user %nice %system %iowait %steal %idle 05:33:01 PM all 34.69 0.00 6.81 4.26 0.11 54.14 05:33:01 PM 0 38.83 0.00 6.66 3.66 0.12 50.73 05:33:01 PM 1 30.55 0.00 6.95 4.86 0.10 57.54 05:34:01 PM all 31.35 0.00 5.63 6.30 0.12 56.60 05:34:01 PM 0 42.21 0.00 7.14 7.91 0.12 42.61 05:34:01 PM 1 20.50 0.00 4.13 4.70 0.12 70.56 05:35:01 PM all 10.68 0.00 1.36 3.37 0.07 84.52 05:35:01 PM 0 19.31 0.00 1.85 3.47 0.07 75.30 05:35:01 PM 1 2.04 0.00 0.87 3.28 0.07 93.75 05:36:01 PM all 0.33 0.00 0.08 0.08 0.04 99.47 05:36:01 PM 0 0.55 0.00 0.05 0.17 0.03 99.20 05:36:01 PM 1 0.12 0.00 0.10 0.00 0.05 99.73 05:37:01 PM all 0.40 0.00 0.08 0.00 0.03 99.49 05:37:01 PM 0 0.48 0.00 0.07 0.00 0.03 99.42 05:37:01 PM 1 0.32 0.00 0.08 0.00 0.03 99.57 05:38:01 PM all 0.41 0.00 0.05 0.00 0.03 99.51 05:38:01 PM 0 0.20 0.00 0.03 0.00 0.02 99.75 05:38:01 PM 1 0.62 0.00 0.07 0.00 0.05 99.26 05:39:01 PM all 0.38 0.00 0.05 0.00 0.04 99.53 05:39:01 PM 0 0.08 0.00 0.05 0.00 0.03 99.83 05:39:01 PM 1 0.67 0.00 0.05 0.00 0.05 99.23 05:40:01 PM all 0.38 0.00 0.08 0.00 0.04 99.51 05:40:01 PM 0 0.22 0.00 0.08 0.00 0.03 99.67 05:40:01 PM 1 0.53 0.00 0.07 0.00 0.05 99.35 05:41:01 PM all 0.38 0.00 0.05 0.00 0.03 99.55 05:41:01 PM 0 0.22 0.00 0.07 0.00 0.03 99.68 05:41:01 PM 1 0.53 0.00 0.03 0.00 0.02 99.42 05:42:01 PM all 0.35 0.00 0.06 0.01 0.03 99.55 05:42:01 PM 0 0.22 0.00 0.07 0.00 0.02 99.70 05:42:01 PM 1 0.48 0.00 0.05 0.02 0.05 99.40 05:43:01 PM all 18.76 0.00 2.47 0.18 0.06 78.53 05:43:01 PM 0 19.00 0.00 2.30 0.22 0.05 78.42 05:43:01 PM 1 18.51 0.00 2.64 0.13 0.07 78.64 Average: CPU %user %nice %system %iowait %steal %idle Average: all 8.89 0.00 1.51 1.29 0.05 88.25 Average: 0 10.99 0.00 1.66 1.40 0.05 85.89 Average: 1 6.79 0.00 1.36 1.18 0.06 90.61