13:32:54 Started by timer 13:32:54 Running as SYSTEM 13:32:54 [EnvInject] - Loading node environment variables. 13:32:54 Building remotely on prd-centos8-builder-2c-2g-10414 (centos8-builder-2c-2g) in workspace /w/workspace/builder-openstack-cron 13:32:54 [ssh-agent] Looking for ssh-agent implementation... 13:32:54 [ssh-agent] Exec ssh-agent (binary ssh-agent on a remote machine) 13:32:54 $ ssh-agent 13:32:54 SSH_AUTH_SOCK=/tmp/ssh-ZkboK8pB6cdd/agent.5448 13:32:54 SSH_AGENT_PID=5449 13:32:54 [ssh-agent] Started. 13:32:54 Running ssh-add (command line suppressed) 13:32:54 Identity added: /w/workspace/builder-openstack-cron@tmp/private_key_12216067819623856045.key (/w/workspace/builder-openstack-cron@tmp/private_key_12216067819623856045.key) 13:32:54 [ssh-agent] Using credentials jenkins (jenkins-ssh) 13:32:54 The recommended git tool is: NONE 13:32:57 using credential jenkins-ssh 13:32:57 Wiping out workspace first. 13:32:57 Cloning the remote Git repository 13:32:57 Cloning repository git://devvexx.opendaylight.org/mirror/releng/builder 13:32:57 > git init /w/workspace/builder-openstack-cron # timeout=10 13:32:57 Fetching upstream changes from git://devvexx.opendaylight.org/mirror/releng/builder 13:32:57 > git --version # timeout=10 13:32:57 > git --version # 'git version 2.43.0' 13:32:57 using GIT_SSH to set credentials jenkins-ssh 13:32:57 [INFO] Currently running in a labeled security context 13:32:57 [INFO] Currently SELinux is 'enforcing' on the host 13:32:57 > /usr/bin/chcon --type=ssh_home_t /w/workspace/builder-openstack-cron@tmp/jenkins-gitclient-ssh12176091274666603374.key 13:32:57 Verifying host key using known hosts file 13:32:57 You're using 'Known hosts file' strategy to verify ssh host keys, but your known_hosts file does not exist, please go to 'Manage Jenkins' -> 'Security' -> 'Git Host Key Verification Configuration' and configure host key verification. 13:32:57 > git fetch --tags --force --progress -- git://devvexx.opendaylight.org/mirror/releng/builder +refs/heads/*:refs/remotes/origin/* # timeout=10 13:32:59 > git config remote.origin.url git://devvexx.opendaylight.org/mirror/releng/builder # timeout=10 13:32:59 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 13:33:00 Avoid second fetch 13:33:00 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10 13:33:00 Checking out Revision 0b38abbc5ab476f968e8079adeba7618d4ea8419 (refs/remotes/origin/master) 13:33:00 > git config core.sparsecheckout # timeout=10 13:33:00 > git checkout -f 0b38abbc5ab476f968e8079adeba7618d4ea8419 # timeout=10 13:33:00 Commit message: "Merge "Fix distribution-merge jobs"" 13:33:00 > git rev-list --no-walk 0b38abbc5ab476f968e8079adeba7618d4ea8419 # timeout=10 13:33:00 > git remote # timeout=10 13:33:00 > git submodule init # timeout=10 13:33:00 > git submodule sync # timeout=10 13:33:00 > git config --get remote.origin.url # timeout=10 13:33:00 > git submodule init # timeout=10 13:33:01 > git config -f .gitmodules --get-regexp ^submodule\.(.+)\.url # timeout=10 13:33:01 > git config --get submodule.global-jjb.url # timeout=10 13:33:01 > git config -f .gitmodules --get submodule.global-jjb.path # timeout=10 13:33:01 > git config --get submodule.packer/common-packer.url # timeout=10 13:33:01 > git config -f .gitmodules --get submodule.packer/common-packer.path # timeout=10 13:33:01 > git submodule update --init --recursive global-jjb # timeout=10 13:33:02 > git submodule update --init --recursive packer/common-packer # timeout=10 13:33:06 Run condition [Boolean condition] enabling prebuild for step [Execute shell] 13:33:06 Run condition [Boolean condition] enabling prebuild for step [Execute shell] 13:33:06 Run condition [Boolean condition] enabling prebuild for step [Execute shell] 13:33:06 Run condition [Boolean condition] enabling prebuild for step [Execute shell] 13:33:06 Run condition [Boolean condition] enabling prebuild for step [Execute shell] 13:33:06 Run condition [Boolean condition] enabling prebuild for step [Execute shell] 13:33:06 Run condition [Boolean condition] enabling prebuild for step [BuilderChain] 13:33:06 No emails were triggered. 13:33:06 provisioning config files... 13:33:06 copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml 13:33:06 copy managed file [npmrc] to file:/home/jenkins/.npmrc 13:33:06 copy managed file [pipconf] to file:/home/jenkins/.config/pip/pip.conf 13:33:06 [builder-openstack-cron] $ /bin/bash /tmp/jenkins16190489728665109933.sh 13:33:06 ---> python-tools-install.sh 13:33:06 Setup pyenv: 13:33:06 system 13:33:06 * 3.8.13 (set by /opt/pyenv/version) 13:33:06 * 3.9.13 (set by /opt/pyenv/version) 13:33:06 * 3.10.13 (set by /opt/pyenv/version) 13:33:06 * 3.11.7 (set by /opt/pyenv/version) 13:33:11 lf-activate-venv(): INFO: Creating python3 venv at /tmp/venv-Ue9O 13:33:11 lf-activate-venv(): INFO: Save venv in file: /tmp/.os_lf_venv 13:33:14 lf-activate-venv(): INFO: Installing: lftools 13:33:45 lf-activate-venv(): INFO: Adding /tmp/venv-Ue9O/bin to PATH 13:33:45 Generating Requirements File 13:34:05 Python 3.11.7 13:34:05 pip 24.2 from /tmp/venv-Ue9O/lib/python3.11/site-packages/pip (python 3.11) 13:34:05 appdirs==1.4.4 13:34:05 argcomplete==3.5.0 13:34:05 aspy.yaml==1.3.0 13:34:05 attrs==24.2.0 13:34:05 autopage==0.5.2 13:34:05 beautifulsoup4==4.12.3 13:34:05 boto3==1.34.158 13:34:05 botocore==1.34.158 13:34:05 bs4==0.0.2 13:34:05 cachetools==5.4.0 13:34:05 certifi==2024.7.4 13:34:05 cffi==1.17.0 13:34:05 cfgv==3.4.0 13:34:05 chardet==5.2.0 13:34:05 charset-normalizer==3.3.2 13:34:05 click==8.1.7 13:34:05 cliff==4.7.0 13:34:05 cmd2==2.4.3 13:34:05 cryptography==3.3.2 13:34:05 debtcollector==3.0.0 13:34:05 decorator==5.1.1 13:34:05 defusedxml==0.7.1 13:34:05 Deprecated==1.2.14 13:34:05 distlib==0.3.8 13:34:05 dnspython==2.6.1 13:34:05 docker==4.2.2 13:34:05 dogpile.cache==1.3.3 13:34:05 email_validator==2.2.0 13:34:05 filelock==3.15.4 13:34:05 future==1.0.0 13:34:05 gitdb==4.0.11 13:34:05 GitPython==3.1.43 13:34:05 google-auth==2.33.0 13:34:05 httplib2==0.22.0 13:34:05 identify==2.6.0 13:34:05 idna==3.7 13:34:05 importlib-resources==1.5.0 13:34:05 iso8601==2.1.0 13:34:05 Jinja2==3.1.4 13:34:05 jmespath==1.0.1 13:34:05 jsonpatch==1.33 13:34:05 jsonpointer==3.0.0 13:34:05 jsonschema==4.23.0 13:34:05 jsonschema-specifications==2023.12.1 13:34:05 keystoneauth1==5.7.0 13:34:05 kubernetes==30.1.0 13:34:05 lftools==0.37.10 13:34:05 lxml==5.3.0 13:34:05 MarkupSafe==2.1.5 13:34:05 msgpack==1.0.8 13:34:05 multi_key_dict==2.0.3 13:34:05 munch==4.0.0 13:34:05 netaddr==1.3.0 13:34:05 netifaces==0.11.0 13:34:05 niet==1.4.2 13:34:05 nodeenv==1.9.1 13:34:05 oauth2client==4.1.3 13:34:05 oauthlib==3.2.2 13:34:05 openstacksdk==3.3.0 13:34:05 os-client-config==2.1.0 13:34:05 os-service-types==1.7.0 13:34:05 osc-lib==3.1.0 13:34:05 oslo.config==9.5.0 13:34:05 oslo.context==5.5.0 13:34:05 oslo.i18n==6.3.0 13:34:05 oslo.log==6.1.1 13:34:05 oslo.serialization==5.4.0 13:34:05 oslo.utils==7.2.0 13:34:05 packaging==24.1 13:34:05 pbr==6.0.0 13:34:05 platformdirs==4.2.2 13:34:05 prettytable==3.11.0 13:34:05 pyasn1==0.6.0 13:34:05 pyasn1_modules==0.4.0 13:34:05 pycparser==2.22 13:34:05 pygerrit2==2.0.15 13:34:05 PyGithub==2.3.0 13:34:05 PyJWT==2.9.0 13:34:05 PyNaCl==1.5.0 13:34:05 pyparsing==2.4.7 13:34:05 pyperclip==1.9.0 13:34:05 pyrsistent==0.20.0 13:34:05 python-cinderclient==9.5.0 13:34:05 python-dateutil==2.9.0.post0 13:34:05 python-heatclient==3.5.0 13:34:05 python-jenkins==1.8.2 13:34:05 python-keystoneclient==5.4.0 13:34:05 python-magnumclient==4.6.0 13:34:05 python-openstackclient==7.0.0 13:34:05 python-swiftclient==4.6.0 13:34:05 PyYAML==6.0.2 13:34:05 referencing==0.35.1 13:34:05 requests==2.32.3 13:34:05 requests-oauthlib==2.0.0 13:34:05 requestsexceptions==1.4.0 13:34:05 rfc3986==2.0.0 13:34:05 rpds-py==0.20.0 13:34:05 rsa==4.9 13:34:05 ruamel.yaml==0.18.6 13:34:05 ruamel.yaml.clib==0.2.8 13:34:05 s3transfer==0.10.2 13:34:05 simplejson==3.19.2 13:34:05 six==1.16.0 13:34:05 smmap==5.0.1 13:34:05 soupsieve==2.5 13:34:05 stevedore==5.2.0 13:34:05 tabulate==0.9.0 13:34:05 toml==0.10.2 13:34:05 tomlkit==0.13.0 13:34:05 tqdm==4.66.5 13:34:05 typing_extensions==4.12.2 13:34:05 tzdata==2024.1 13:34:05 urllib3==1.26.19 13:34:05 virtualenv==20.26.3 13:34:05 wcwidth==0.2.13 13:34:05 websocket-client==1.8.0 13:34:05 wrapt==1.16.0 13:34:05 xdg==6.0.0 13:34:05 xmltodict==0.13.0 13:34:05 yq==3.4.3 13:34:05 [EnvInject] - Injecting environment variables from a build step. 13:34:05 [EnvInject] - Injecting as environment variables the properties content 13:34:05 OS_CLOUD=vex 13:34:05 13:34:05 [EnvInject] - Variables injected successfully. 13:34:05 [Boolean condition] checking [True] against [^(1|y|yes|t|true|on|run)$] (origin token: True) 13:34:05 Run condition [Boolean condition] enabling perform for step [Execute shell] 13:34:05 [builder-openstack-cron] $ /bin/bash -l /tmp/jenkins9386884559896252191.sh 13:34:05 ---> Orphaned k8s clusters 13:34:05 Setup pyenv: 13:34:05 system 13:34:05 3.8.13 13:34:05 3.9.13 13:34:05 3.10.13 13:34:05 * 3.11.7 (set by /w/workspace/builder-openstack-cron/.python-version) 13:34:06 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-Ue9O from file:/tmp/.os_lf_venv 13:34:12 lf-activate-venv(): INFO: Installing: kubernetes python-heatclient python-openstackclient python-magnumclient 13:34:18 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. 13:34:18 lftools 0.37.10 requires urllib3<2.1.0, but you have urllib3 2.2.2 which is incompatible. 13:34:18 lf-activate-venv(): INFO: Adding /tmp/venv-Ue9O/bin to PATH 13:34:21 -----> Delete orphaned cluster 13:34:21 [Boolean condition] checking [True] against [^(1|y|yes|t|true|on|run)$] (origin token: True) 13:34:21 Run condition [Boolean condition] enabling perform for step [Execute shell] 13:34:21 [builder-openstack-cron] $ /bin/bash -l /tmp/jenkins2121240403535777579.sh 13:34:21 ---> Orphaned stacks 13:34:21 Setup pyenv: 13:34:21 system 13:34:21 3.8.13 13:34:21 3.9.13 13:34:21 3.10.13 13:34:21 * 3.11.7 (set by /w/workspace/builder-openstack-cron/.python-version) 13:34:22 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-Ue9O from file:/tmp/.os_lf_venv 13:34:23 lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq 13:34:43 lf-activate-venv(): INFO: Adding /tmp/venv-Ue9O/bin to PATH 13:34:43 + mapfile -t OS_COE_CLUSTERS_ID 13:34:43 ++ grep -E '(DELETE_FAILED|UNKNOWN|UNHEALTHY)' 13:34:43 ++ openstack --os-cloud vex coe cluster list -f value -c uuid -c name 13:34:43 ++ awk '{print $1}' 13:34:46 + echo '-----> Active clusters -> stacks' 13:34:46 -----> Active clusters -> stacks 13:34:46 + OS_COE_STACKS=() 13:34:46 + [[ 0 -gt 0 ]] 13:34:46 + mapfile -t OS_STACKS 13:34:46 ++ awk '{print $1}' 13:34:46 ++ openstack --os-cloud vex stack list -f value -c 'Stack Name' -c 'Stack Status' --property stack_status=CREATE_COMPLETE --property stack_status=DELETE_FAILED --property stack_status=CREATE_FAILED 13:34:47 + echo '-----> Active stacks' 13:34:47 -----> Active stacks 13:34:47 + echo '-----> Delete orphaned stacks' 13:34:47 -----> Delete orphaned stacks 13:34:47 [Boolean condition] checking [True] against [^(1|y|yes|t|true|on|run)$] (origin token: True) 13:34:47 Run condition [Boolean condition] enabling perform for step [Execute shell] 13:34:47 [builder-openstack-cron] $ /bin/bash -l /tmp/jenkins12053438253789285369.sh 13:34:47 ---> Orphaned servers 13:34:47 Setup pyenv: 13:34:47 system 13:34:47 3.8.13 13:34:47 3.9.13 13:34:47 3.10.13 13:34:47 * 3.11.7 (set by /w/workspace/builder-openstack-cron/.python-version) 13:34:48 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-Ue9O from file:/tmp/.os_lf_venv 13:34:49 lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq 13:35:13 lf-activate-venv(): INFO: Adding /tmp/venv-Ue9O/bin to PATH 13:35:18 -----> Active servers 13:35:18 prd-centos8-builder-2c-2g-10414 13:35:18 prd-centos8-builder-2c-2g-10413 13:35:18 prd-ubuntu2004-docker-4c-4g-10412 13:35:18 prd-ubuntu2004-docker-4c-4g-10411 13:35:18 prd-centos8-builder-2c-2g-10410 13:35:18 prd-centos8-builder-2c-2g-10408 13:35:18 prd-ubuntu2004-docker-8c-8g-10406 13:35:18 prd-queue-intque-2c-1g-10405 13:35:18 prd-ubuntu2004-docker-8c-8g-10400 13:35:18 prd-centos8-robot-2c-8g-10363 13:35:18 prd-centos8-robot-2c-8g-10362 13:35:18 prd-centos7-docker-1c-4g-10305 13:35:18 prd-centos8-builder-2c-2g-8811 13:35:18 prd-centos8-builder-2c-2g-8810 13:35:18 prd-ubuntu2004-docker-4c-4g-8809 13:35:18 prd-centos8-builder-2c-2g-8807 13:35:18 prd-ubuntu2004-docker-4c-4g-8804 13:35:18 prd-ubuntu2004-docker-2c-2g-8788 13:35:18 -----> Delete orphaned servers 13:35:19 Deleting orphaned server: prd-centos8-builder-2c-2g-8811 13:35:22 AttributeError: 'Server' object has no attribute 'created'. Did you mean: 'create'? 13:35:22 Deleting orphaned server: prd-centos8-builder-2c-2g-8810 13:35:25 AttributeError: 'Server' object has no attribute 'created'. Did you mean: 'create'? 13:35:25 Deleting orphaned server: prd-ubuntu2004-docker-4c-4g-8809 13:35:28 AttributeError: 'Server' object has no attribute 'created'. Did you mean: 'create'? 13:35:28 Deleting orphaned server: prd-centos8-builder-2c-2g-8807 13:35:31 AttributeError: 'Server' object has no attribute 'created'. Did you mean: 'create'? 13:35:31 Deleting orphaned server: prd-ubuntu2004-docker-4c-4g-8804 13:35:34 AttributeError: 'Server' object has no attribute 'created'. Did you mean: 'create'? 13:35:34 Deleting orphaned server: prd-ubuntu2004-docker-2c-2g-8788 13:35:37 AttributeError: 'Server' object has no attribute 'created'. Did you mean: 'create'? 13:35:37 Build step 'Conditional step (single)' marked build as failure 13:35:37 $ ssh-agent -k 13:35:37 unset SSH_AUTH_SOCK; 13:35:37 unset SSH_AGENT_PID; 13:35:37 echo Agent pid 5449 killed; 13:35:37 [ssh-agent] Stopped. 13:35:37 [PostBuildScript] - [INFO] Executing post build scripts. 13:35:37 [builder-openstack-cron] $ /bin/bash /tmp/jenkins6697920194931024092.sh 13:35:37 ---> sysstat.sh 13:35:37 [builder-openstack-cron] $ /bin/bash /tmp/jenkins15924621785665367383.sh 13:35:37 ---> package-listing.sh 13:35:37 ++ tr '[:upper:]' '[:lower:]' 13:35:37 ++ facter osfamily 13:35:38 + OS_FAMILY=redhat 13:35:38 + workspace=/w/workspace/builder-openstack-cron 13:35:38 + START_PACKAGES=/tmp/packages_start.txt 13:35:38 + END_PACKAGES=/tmp/packages_end.txt 13:35:38 + DIFF_PACKAGES=/tmp/packages_diff.txt 13:35:38 + PACKAGES=/tmp/packages_start.txt 13:35:38 + '[' /w/workspace/builder-openstack-cron ']' 13:35:38 + PACKAGES=/tmp/packages_end.txt 13:35:38 + case "${OS_FAMILY}" in 13:35:38 + rpm -qa 13:35:38 + sort 13:35:38 + '[' -f /tmp/packages_start.txt ']' 13:35:38 + '[' -f /tmp/packages_end.txt ']' 13:35:38 + diff /tmp/packages_start.txt /tmp/packages_end.txt 13:35:38 + '[' /w/workspace/builder-openstack-cron ']' 13:35:38 + mkdir -p /w/workspace/builder-openstack-cron/archives/ 13:35:38 + cp -f /tmp/packages_diff.txt /tmp/packages_end.txt /tmp/packages_start.txt /w/workspace/builder-openstack-cron/archives/ 13:35:38 [builder-openstack-cron] $ /bin/bash /tmp/jenkins7455562313367280479.sh 13:35:38 ---> capture-instance-metadata.sh 13:35:38 Setup pyenv: 13:35:38 system 13:35:38 3.8.13 13:35:38 3.9.13 13:35:38 3.10.13 13:35:38 * 3.11.7 (set by /w/workspace/builder-openstack-cron/.python-version) 13:35:39 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-Ue9O from file:/tmp/.os_lf_venv 13:35:40 lf-activate-venv(): INFO: Installing: lftools 13:35:50 lf-activate-venv(): INFO: Adding /tmp/venv-Ue9O/bin to PATH 13:35:50 INFO: Running in OpenStack, capturing instance metadata 13:35:50 [builder-openstack-cron] $ /bin/bash /tmp/jenkins1117470793851825618.sh 13:35:50 provisioning config files... 13:35:50 Could not find credentials [logs] for builder-openstack-cron #64182 13:35:50 copy managed file [jenkins-log-archives-settings] to file:/w/workspace/builder-openstack-cron@tmp/config14231315951644629577tmp 13:35:50 Regular expression run condition: Expression=[^.*logs-s3.*], Label=[odl-logs-s3-cloudfront-index] 13:35:50 Run condition [Regular expression match] enabling perform for step [Provide Configuration files] 13:35:50 provisioning config files... 13:35:51 copy managed file [jenkins-s3-log-ship] to file:/home/jenkins/.aws/credentials 13:35:51 [EnvInject] - Injecting environment variables from a build step. 13:35:51 [EnvInject] - Injecting as environment variables the properties content 13:35:51 SERVER_ID=logs 13:35:51 13:35:51 [EnvInject] - Variables injected successfully. 13:35:51 [builder-openstack-cron] $ /bin/bash /tmp/jenkins669075683588027905.sh 13:35:51 ---> create-netrc.sh 13:35:51 WARN: Log server credential not found. 13:35:51 [builder-openstack-cron] $ /bin/bash /tmp/jenkins18438297840205970403.sh 13:35:51 ---> python-tools-install.sh 13:35:51 Setup pyenv: 13:35:51 system 13:35:51 3.8.13 13:35:51 3.9.13 13:35:51 3.10.13 13:35:51 * 3.11.7 (set by /w/workspace/builder-openstack-cron/.python-version) 13:35:51 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-Ue9O from file:/tmp/.os_lf_venv 13:35:52 lf-activate-venv(): INFO: Installing: lftools 13:36:00 lf-activate-venv(): INFO: Adding /tmp/venv-Ue9O/bin to PATH 13:36:00 [builder-openstack-cron] $ /bin/bash /tmp/jenkins14289802806424124594.sh 13:36:00 ---> sudo-logs.sh 13:36:00 Archiving 'sudo' log.. 13:36:00 [builder-openstack-cron] $ /bin/bash /tmp/jenkins18268659456769009277.sh 13:36:00 ---> job-cost.sh 13:36:00 Setup pyenv: 13:36:00 system 13:36:00 3.8.13 13:36:00 3.9.13 13:36:00 3.10.13 13:36:01 * 3.11.7 (set by /w/workspace/builder-openstack-cron/.python-version) 13:36:01 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-Ue9O from file:/tmp/.os_lf_venv 13:36:02 lf-activate-venv(): INFO: Installing: zipp==1.1.0 python-openstackclient urllib3~=1.26.15 13:36:07 lf-activate-venv(): INFO: Adding /tmp/venv-Ue9O/bin to PATH 13:36:07 INFO: No Stack... 13:36:07 INFO: Retrieving Pricing Info for: v3-standard-2 13:36:07 INFO: Archiving Costs 13:36:07 [builder-openstack-cron] $ /bin/bash -l /tmp/jenkins2452722484803166710.sh 13:36:07 ---> logs-deploy.sh 13:36:07 Setup pyenv: 13:36:08 system 13:36:08 3.8.13 13:36:08 3.9.13 13:36:08 3.10.13 13:36:08 * 3.11.7 (set by /w/workspace/builder-openstack-cron/.python-version) 13:36:08 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-Ue9O from file:/tmp/.os_lf_venv 13:36:09 lf-activate-venv(): INFO: Installing: lftools 13:36:17 lf-activate-venv(): INFO: Adding /tmp/venv-Ue9O/bin to PATH 13:36:17 WARNING: Nexus logging server not set 13:36:17 INFO: S3 path logs/releng/vex-yul-odl-jenkins-1/builder-openstack-cron/64182/ 13:36:17 INFO: archiving logs to S3 13:36:18 ---> uname -a: 13:36:18 Linux prd-centos8-builder-2c-2g-10414.novalocal 4.18.0-553.5.1.el8.x86_64 #1 SMP Tue May 21 05:46:01 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 13:36:18 13:36:18 13:36:18 ---> lscpu: 13:36:18 Architecture: x86_64 13:36:18 CPU op-mode(s): 32-bit, 64-bit 13:36:18 Byte Order: Little Endian 13:36:18 CPU(s): 2 13:36:18 On-line CPU(s) list: 0,1 13:36:18 Thread(s) per core: 1 13:36:18 Core(s) per socket: 1 13:36:18 Socket(s): 2 13:36:18 NUMA node(s): 1 13:36:18 Vendor ID: AuthenticAMD 13:36:18 CPU family: 23 13:36:18 Model: 49 13:36:18 Model name: AMD EPYC-Rome Processor 13:36:18 Stepping: 0 13:36:18 CPU MHz: 2799.998 13:36:18 BogoMIPS: 5599.99 13:36:18 Virtualization: AMD-V 13:36:18 Hypervisor vendor: KVM 13:36:18 Virtualization type: full 13:36:18 L1d cache: 32K 13:36:18 L1i cache: 32K 13:36:18 L2 cache: 512K 13:36:18 L3 cache: 16384K 13:36:18 NUMA node0 CPU(s): 0,1 13:36:18 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr wbnoinvd arat npt nrip_save umip rdpid arch_capabilities 13:36:18 13:36:18 13:36:18 ---> nproc: 13:36:18 2 13:36:18 13:36:18 13:36:18 ---> df -h: 13:36:18 Filesystem Size Used Avail Use% Mounted on 13:36:18 devtmpfs 3.8G 0 3.8G 0% /dev 13:36:18 tmpfs 3.8G 0 3.8G 0% /dev/shm 13:36:18 tmpfs 3.8G 17M 3.8G 1% /run 13:36:18 tmpfs 3.8G 0 3.8G 0% /sys/fs/cgroup 13:36:18 /dev/vda1 40G 8.4G 32G 21% / 13:36:18 tmpfs 770M 0 770M 0% /run/user/1001 13:36:18 13:36:18 13:36:18 ---> free -m: 13:36:18 total used free shared buff/cache available 13:36:18 Mem: 7697 681 4777 19 2238 6722 13:36:18 Swap: 1023 0 1023 13:36:18 13:36:18 13:36:18 ---> ip addr: 13:36:18 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 13:36:18 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 13:36:18 inet 127.0.0.1/8 scope host lo 13:36:18 valid_lft forever preferred_lft forever 13:36:18 inet6 ::1/128 scope host 13:36:18 valid_lft forever preferred_lft forever 13:36:18 2: eth0: mtu 1458 qdisc mq state UP group default qlen 1000 13:36:18 link/ether fa:16:3e:8c:6e:88 brd ff:ff:ff:ff:ff:ff 13:36:18 altname enp0s3 13:36:18 altname ens3 13:36:18 inet 10.30.170.230/23 brd 10.30.171.255 scope global dynamic noprefixroute eth0 13:36:18 valid_lft 86158sec preferred_lft 86158sec 13:36:18 inet6 fe80::f816:3eff:fe8c:6e88/64 scope link 13:36:18 valid_lft forever preferred_lft forever 13:36:18 3: docker0: mtu 1458 qdisc noqueue state DOWN group default 13:36:18 link/ether 02:42:d7:eb:93:a5 brd ff:ff:ff:ff:ff:ff 13:36:18 inet 10.250.0.254/24 brd 10.250.0.255 scope global docker0 13:36:18 valid_lft forever preferred_lft forever 13:36:18 13:36:18 13:36:18 ---> sar -b -r -n DEV: 13:36:18 Linux 4.18.0-553.5.1.el8.x86_64 (centos-stream-8-builder-5e6ee0fb-b6d9-4116-90d4-35c890e2b0f7.nov) 08/12/2024 _x86_64_ (2 CPU) 13:36:18 13:36:18 13:32:12 LINUX RESTART (2 CPU) 13:36:18 13:36:18 01:33:01 PM tps rtps wtps bread/s bwrtn/s 13:36:18 01:34:01 PM 155.21 13.60 141.62 2375.87 7020.41 13:36:18 01:35:01 PM 80.97 0.25 80.72 18.53 9413.01 13:36:18 01:36:01 PM 24.16 1.15 23.01 133.69 1048.73 13:36:18 Average: 86.78 5.00 81.78 842.74 5827.19 13:36:18 13:36:18 01:33:01 PM kbmemfree kbavail kbmemused %memused kbbuffers kbcached kbcommit %commit kbactive kbinact kbdirty 13:36:18 01:34:01 PM 4960540 6888200 2921892 37.07 2688 2108848 1089212 12.20 233296 2358392 193124 13:36:18 01:35:01 PM 4923516 6907652 2958916 37.54 2688 2161328 1045292 11.70 254176 2357428 18216 13:36:18 01:36:01 PM 4953068 6939888 2929364 37.16 2688 2164356 1022996 11.45 430324 2152356 188 13:36:18 Average: 4945708 6911913 2936724 37.26 2688 2144844 1052500 11.78 305932 2289392 70509 13:36:18 13:36:18 01:33:01 PM IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s %ifutil 13:36:18 01:34:01 PM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 13:36:18 01:34:01 PM eth0 105.73 83.01 1065.05 11.97 0.00 0.00 0.00 0.00 13:36:18 01:34:01 PM docker0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 13:36:18 01:35:01 PM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 13:36:18 01:35:01 PM eth0 37.18 29.98 332.92 7.14 0.00 0.00 0.00 0.00 13:36:18 01:35:01 PM docker0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 13:36:18 01:36:01 PM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 13:36:18 01:36:01 PM eth0 32.71 30.26 39.69 9.71 0.00 0.00 0.00 0.00 13:36:18 01:36:01 PM docker0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 13:36:18 Average: lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 13:36:18 Average: eth0 58.54 47.75 479.23 9.61 0.00 0.00 0.00 0.00 13:36:18 Average: docker0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 13:36:18 13:36:18 13:36:18 ---> sar -P ALL: 13:36:18 Linux 4.18.0-553.5.1.el8.x86_64 (centos-stream-8-builder-5e6ee0fb-b6d9-4116-90d4-35c890e2b0f7.nov) 08/12/2024 _x86_64_ (2 CPU) 13:36:18 13:36:18 13:32:12 LINUX RESTART (2 CPU) 13:36:18 13:36:18 01:33:01 PM CPU %user %nice %system %iowait %steal %idle 13:36:18 01:34:01 PM all 34.06 0.00 6.26 3.79 0.09 55.80 13:36:18 01:34:01 PM 0 48.49 0.00 6.94 2.72 0.08 41.77 13:36:18 01:34:01 PM 1 19.61 0.00 5.58 4.86 0.10 69.86 13:36:18 01:35:01 PM all 23.55 0.00 3.34 6.59 0.14 66.37 13:36:18 01:35:01 PM 0 34.82 0.00 3.87 9.92 0.13 51.26 13:36:18 01:35:01 PM 1 12.32 0.00 2.81 3.28 0.15 81.45 13:36:18 01:36:01 PM all 28.71 0.00 3.87 1.45 0.11 65.86 13:36:18 01:36:01 PM 0 36.38 0.00 4.21 2.05 0.13 57.22 13:36:18 01:36:01 PM 1 21.05 0.00 3.52 0.85 0.08 74.49 13:36:18 Average: all 28.78 0.00 4.49 3.95 0.11 62.67 13:36:18 Average: 0 39.91 0.00 5.01 4.89 0.12 50.06 13:36:18 Average: 1 17.65 0.00 3.97 3.00 0.11 75.27 13:36:18 13:36:18 13:36:18