02:08:40 Started by upstream project "autorelease-release-titanium-mvn39-openjdk21" build number 28 02:08:40 originally caused by: 02:08:40 Started by timer 02:08:40 Running as SYSTEM 02:08:40 [EnvInject] - Loading node environment variables. 02:08:40 Building remotely on prd-queue-disttest-2c-1g-25974 (queue-disttest-2c-1g) in workspace /w/workspace/integration-distribution-test-titanium 02:08:41 [ssh-agent] Looking for ssh-agent implementation... 02:08:41 [ssh-agent] Exec ssh-agent (binary ssh-agent on a remote machine) 02:08:41 $ ssh-agent 02:08:41 SSH_AUTH_SOCK=/tmp/ssh-KXE4YHUPgRMl/agent.9223 02:08:41 SSH_AGENT_PID=9224 02:08:41 [ssh-agent] Started. 02:08:41 Running ssh-add (command line suppressed) 02:08:41 Identity added: /w/workspace/integration-distribution-test-titanium@tmp/private_key_2186626887872553521.key (/w/workspace/integration-distribution-test-titanium@tmp/private_key_2186626887872553521.key) 02:08:41 [ssh-agent] Using credentials jenkins (jenkins-ssh) 02:08:41 No emails were triggered. 02:08:41 provisioning config files... 02:08:41 copy managed file [npmrc] to file:/home/jenkins/.npmrc 02:08:41 copy managed file [pipconf] to file:/home/jenkins/.config/pip/pip.conf 02:08:41 [integration-distribution-test-titanium] $ /bin/bash /tmp/jenkins17116145358929233637.sh 02:08:41 ---> python-tools-install.sh 02:08:41 Setup pyenv: 02:08:41 system 02:08:41 * 3.8.13 (set by /opt/pyenv/version) 02:08:41 * 3.9.13 (set by /opt/pyenv/version) 02:08:41 3.10.13 02:08:48 lf-activate-venv(): INFO: Creating python3 venv at /tmp/venv-Pyyi 02:08:48 lf-activate-venv(): INFO: Save venv in file: /tmp/.os_lf_venv 02:08:53 lf-activate-venv(): INFO: Installing: lftools 02:09:34 lf-activate-venv(): INFO: Adding /tmp/venv-Pyyi/bin to PATH 02:09:34 Generating Requirements File 02:10:09 Python 3.10.13 02:10:09 pip 24.2 from /tmp/venv-Pyyi/lib/python3.10/site-packages/pip (python 3.10) 02:10:10 appdirs==1.4.4 02:10:10 argcomplete==3.5.0 02:10:10 aspy.yaml==1.3.0 02:10:10 attrs==24.2.0 02:10:10 autopage==0.5.2 02:10:10 beautifulsoup4==4.12.3 02:10:10 boto3==1.35.24 02:10:10 botocore==1.35.24 02:10:10 bs4==0.0.2 02:10:10 cachetools==5.5.0 02:10:10 certifi==2024.8.30 02:10:10 cffi==1.17.1 02:10:10 cfgv==3.4.0 02:10:10 chardet==5.2.0 02:10:10 charset-normalizer==3.3.2 02:10:10 click==8.1.7 02:10:10 cliff==4.7.0 02:10:10 cmd2==2.4.3 02:10:10 cryptography==3.3.2 02:10:10 debtcollector==3.0.0 02:10:10 decorator==5.1.1 02:10:10 defusedxml==0.7.1 02:10:10 Deprecated==1.2.14 02:10:10 distlib==0.3.8 02:10:10 dnspython==2.6.1 02:10:10 docker==4.2.2 02:10:10 dogpile.cache==1.3.3 02:10:10 durationpy==0.7 02:10:10 email_validator==2.2.0 02:10:10 filelock==3.16.1 02:10:10 future==1.0.0 02:10:10 gitdb==4.0.11 02:10:10 GitPython==3.1.43 02:10:10 google-auth==2.35.0 02:10:10 httplib2==0.22.0 02:10:10 identify==2.6.1 02:10:10 idna==3.10 02:10:10 importlib-resources==1.5.0 02:10:10 iso8601==2.1.0 02:10:10 Jinja2==3.1.4 02:10:10 jmespath==1.0.1 02:10:10 jsonpatch==1.33 02:10:10 jsonpointer==3.0.0 02:10:10 jsonschema==4.23.0 02:10:10 jsonschema-specifications==2023.12.1 02:10:10 keystoneauth1==5.8.0 02:10:10 kubernetes==31.0.0 02:10:10 lftools==0.37.10 02:10:10 lxml==5.3.0 02:10:10 MarkupSafe==2.1.5 02:10:10 msgpack==1.1.0 02:10:10 multi_key_dict==2.0.3 02:10:10 munch==4.0.0 02:10:10 netaddr==1.3.0 02:10:10 netifaces==0.11.0 02:10:10 niet==1.4.2 02:10:10 nodeenv==1.9.1 02:10:10 oauth2client==4.1.3 02:10:10 oauthlib==3.2.2 02:10:10 openstacksdk==4.0.0 02:10:10 os-client-config==2.1.0 02:10:10 os-service-types==1.7.0 02:10:10 osc-lib==3.1.0 02:10:10 oslo.config==9.6.0 02:10:10 oslo.context==5.6.0 02:10:10 oslo.i18n==6.4.0 02:10:10 oslo.log==6.1.2 02:10:10 oslo.serialization==5.5.0 02:10:10 oslo.utils==7.3.0 02:10:10 packaging==24.1 02:10:10 pbr==6.1.0 02:10:10 platformdirs==4.3.6 02:10:10 prettytable==3.11.0 02:10:10 pyasn1==0.6.1 02:10:10 pyasn1_modules==0.4.1 02:10:10 pycparser==2.22 02:10:10 pygerrit2==2.0.15 02:10:10 PyGithub==2.4.0 02:10:10 PyJWT==2.9.0 02:10:10 PyNaCl==1.5.0 02:10:10 pyparsing==2.4.7 02:10:10 pyperclip==1.9.0 02:10:10 pyrsistent==0.20.0 02:10:10 python-cinderclient==9.6.0 02:10:10 python-dateutil==2.9.0.post0 02:10:10 python-heatclient==4.0.0 02:10:10 python-jenkins==1.8.2 02:10:10 python-keystoneclient==5.5.0 02:10:10 python-magnumclient==4.7.0 02:10:10 python-openstackclient==7.1.2 02:10:10 python-swiftclient==4.6.0 02:10:10 PyYAML==6.0.2 02:10:10 referencing==0.35.1 02:10:10 requests==2.32.3 02:10:10 requests-oauthlib==2.0.0 02:10:10 requestsexceptions==1.4.0 02:10:10 rfc3986==2.0.0 02:10:10 rpds-py==0.20.0 02:10:10 rsa==4.9 02:10:10 ruamel.yaml==0.18.6 02:10:10 ruamel.yaml.clib==0.2.8 02:10:10 s3transfer==0.10.2 02:10:10 simplejson==3.19.3 02:10:10 six==1.16.0 02:10:10 smmap==5.0.1 02:10:10 soupsieve==2.6 02:10:10 stevedore==5.3.0 02:10:10 tabulate==0.9.0 02:10:10 toml==0.10.2 02:10:10 tomlkit==0.13.2 02:10:10 tqdm==4.66.5 02:10:10 typing_extensions==4.12.2 02:10:10 tzdata==2024.1 02:10:10 urllib3==1.26.20 02:10:10 virtualenv==20.26.5 02:10:10 wcwidth==0.2.13 02:10:10 websocket-client==1.8.0 02:10:10 wrapt==1.16.0 02:10:10 xdg==6.0.0 02:10:10 xmltodict==0.13.0 02:10:10 yq==3.4.3 02:10:10 $ ssh-agent -k 02:10:10 unset SSH_AUTH_SOCK; 02:10:10 unset SSH_AGENT_PID; 02:10:10 echo Agent pid 9224 killed; 02:10:10 [ssh-agent] Stopped. 02:10:10 ERROR: Build aborted. Can't trigger undefined projects. 1 of the below project(s) can't be resolved: 02:10:10 > distribution-csit-managed-titanium 02:10:10 Check your configuration! 02:10:10 [PostBuildScript] - [INFO] Executing post build scripts. 02:10:10 [integration-distribution-test-titanium] $ /bin/sh /tmp/jenkins8746108738907121932.sh 02:10:24 python /tmp/tmp.BXHD9JUsLI https://jenkins.opendaylight.org/releng/job/integration-distribution-test-titanium/28/ 02:10:24 [PostBuildScript] - [INFO] Executing post build scripts. 02:10:24 [integration-distribution-test-titanium] $ /bin/bash /tmp/jenkins17034857507074724771.sh 02:10:24 ---> sysstat.sh 02:10:24 [integration-distribution-test-titanium] $ /bin/bash /tmp/jenkins1070990154519223027.sh 02:10:24 ---> package-listing.sh 02:10:24 ++ facter osfamily 02:10:24 ++ tr '[:upper:]' '[:lower:]' 02:10:24 + OS_FAMILY=redhat 02:10:24 + workspace=/w/workspace/integration-distribution-test-titanium 02:10:24 + START_PACKAGES=/tmp/packages_start.txt 02:10:24 + END_PACKAGES=/tmp/packages_end.txt 02:10:24 + DIFF_PACKAGES=/tmp/packages_diff.txt 02:10:24 + PACKAGES=/tmp/packages_start.txt 02:10:24 + '[' /w/workspace/integration-distribution-test-titanium ']' 02:10:24 + PACKAGES=/tmp/packages_end.txt 02:10:24 + case "${OS_FAMILY}" in 02:10:24 + rpm -qa 02:10:24 + sort 02:10:25 + '[' -f /tmp/packages_start.txt ']' 02:10:25 + '[' -f /tmp/packages_end.txt ']' 02:10:25 + diff /tmp/packages_start.txt /tmp/packages_end.txt 02:10:25 + '[' /w/workspace/integration-distribution-test-titanium ']' 02:10:25 + mkdir -p /w/workspace/integration-distribution-test-titanium/archives/ 02:10:25 + cp -f /tmp/packages_diff.txt /tmp/packages_end.txt /tmp/packages_start.txt /w/workspace/integration-distribution-test-titanium/archives/ 02:10:25 [integration-distribution-test-titanium] $ /bin/bash /tmp/jenkins7616922156679211138.sh 02:10:25 ---> capture-instance-metadata.sh 02:10:25 Setup pyenv: 02:10:25 system 02:10:25 3.8.13 02:10:25 3.9.13 02:10:25 * 3.10.13 (set by /w/workspace/integration-distribution-test-titanium/.python-version) 02:10:25 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-Pyyi from file:/tmp/.os_lf_venv 02:10:27 lf-activate-venv(): INFO: Installing: lftools 02:10:43 lf-activate-venv(): INFO: Adding /tmp/venv-Pyyi/bin to PATH 02:10:43 INFO: Running in OpenStack, capturing instance metadata 02:10:43 [integration-distribution-test-titanium] $ /bin/bash /tmp/jenkins3242924747571286981.sh 02:10:44 provisioning config files... 02:10:44 Could not find credentials [logs] for integration-distribution-test-titanium #28 02:10:44 copy managed file [jenkins-log-archives-settings] to file:/w/workspace/integration-distribution-test-titanium@tmp/config13905866711739825024tmp 02:10:44 Regular expression run condition: Expression=[^.*logs-s3.*], Label=[odl-logs-s3-cloudfront-index] 02:10:44 Run condition [Regular expression match] enabling perform for step [Provide Configuration files] 02:10:44 provisioning config files... 02:10:44 copy managed file [jenkins-s3-log-ship] to file:/home/jenkins/.aws/credentials 02:10:44 [EnvInject] - Injecting environment variables from a build step. 02:10:44 [EnvInject] - Injecting as environment variables the properties content 02:10:44 SERVER_ID=logs 02:10:44 02:10:44 [EnvInject] - Variables injected successfully. 02:10:44 [integration-distribution-test-titanium] $ /bin/bash /tmp/jenkins4724355954977053072.sh 02:10:44 ---> create-netrc.sh 02:10:44 WARN: Log server credential not found. 02:10:44 [integration-distribution-test-titanium] $ /bin/bash /tmp/jenkins15068183885453559117.sh 02:10:44 ---> python-tools-install.sh 02:10:44 Setup pyenv: 02:10:44 system 02:10:44 3.8.13 02:10:44 3.9.13 02:10:44 * 3.10.13 (set by /w/workspace/integration-distribution-test-titanium/.python-version) 02:10:44 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-Pyyi from file:/tmp/.os_lf_venv 02:10:46 lf-activate-venv(): INFO: Installing: lftools 02:10:59 lf-activate-venv(): INFO: Adding /tmp/venv-Pyyi/bin to PATH 02:10:59 [integration-distribution-test-titanium] $ /bin/bash /tmp/jenkins10348144282855950148.sh 02:10:59 ---> sudo-logs.sh 02:10:59 Archiving 'sudo' log.. 02:10:59 [integration-distribution-test-titanium] $ /bin/bash /tmp/jenkins13427941256222833721.sh 02:10:59 ---> job-cost.sh 02:10:59 Setup pyenv: 02:10:59 system 02:10:59 3.8.13 02:10:59 3.9.13 02:10:59 * 3.10.13 (set by /w/workspace/integration-distribution-test-titanium/.python-version) 02:10:59 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-Pyyi from file:/tmp/.os_lf_venv 02:11:00 lf-activate-venv(): INFO: Installing: zipp==1.1.0 python-openstackclient urllib3~=1.26.15 02:11:06 lf-activate-venv(): INFO: Adding /tmp/venv-Pyyi/bin to PATH 02:11:06 INFO: No Stack... 02:11:06 INFO: Retrieving Pricing Info for: v3-standard-2 02:11:07 INFO: Archiving Costs 02:11:07 [integration-distribution-test-titanium] $ /bin/bash -l /tmp/jenkins7693157652855611306.sh 02:11:07 ---> logs-deploy.sh 02:11:07 Setup pyenv: 02:11:07 system 02:11:07 3.8.13 02:11:07 3.9.13 02:11:07 * 3.10.13 (set by /w/workspace/integration-distribution-test-titanium/.python-version) 02:11:07 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-Pyyi from file:/tmp/.os_lf_venv 02:11:08 lf-activate-venv(): INFO: Installing: lftools 02:11:18 lf-activate-venv(): INFO: Adding /tmp/venv-Pyyi/bin to PATH 02:11:18 WARNING: Nexus logging server not set 02:11:18 INFO: S3 path logs/releng/vex-yul-odl-jenkins-1/integration-distribution-test-titanium/28/ 02:11:18 INFO: archiving logs to S3 02:11:20 ---> uname -a: 02:11:20 Linux prd-queue-disttest-2c-1g-25974.novalocal 3.10.0-1160.118.1.el7.x86_64 #1 SMP Wed Apr 24 16:01:50 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux 02:11:20 02:11:20 02:11:20 ---> lscpu: 02:11:20 Architecture: x86_64 02:11:20 CPU op-mode(s): 32-bit, 64-bit 02:11:20 Byte Order: Little Endian 02:11:20 CPU(s): 2 02:11:20 On-line CPU(s) list: 0,1 02:11:20 Thread(s) per core: 1 02:11:20 Core(s) per socket: 1 02:11:20 Socket(s): 2 02:11:20 NUMA node(s): 1 02:11:20 Vendor ID: AuthenticAMD 02:11:20 CPU family: 23 02:11:20 Model: 49 02:11:20 Model name: AMD EPYC-Rome Processor 02:11:20 Stepping: 0 02:11:20 CPU MHz: 2799.998 02:11:20 BogoMIPS: 5599.99 02:11:20 Virtualization: AMD-V 02:11:20 Hypervisor vendor: KVM 02:11:20 Virtualization type: full 02:11:20 L1d cache: 32K 02:11:20 L1i cache: 32K 02:11:20 L2 cache: 512K 02:11:20 L3 cache: 16384K 02:11:20 NUMA node0 CPU(s): 0,1 02:11:20 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm art rep_good nopl extd_apicid eagerfpu pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ssbd rsb_ctxsw ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 clzero xsaveerptr arat npt nrip_save umip spec_ctrl intel_stibp arch_capabilities 02:11:20 02:11:20 02:11:20 ---> nproc: 02:11:20 2 02:11:20 02:11:20 02:11:20 ---> df -h: 02:11:20 Filesystem Size Used Avail Use% Mounted on 02:11:20 devtmpfs 3.8G 0 3.8G 0% /dev 02:11:20 tmpfs 3.9G 0 3.9G 0% /dev/shm 02:11:20 tmpfs 3.9G 17M 3.9G 1% /run 02:11:20 tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup 02:11:20 /dev/vda1 40G 5.8G 35G 15% / 02:11:20 tmpfs 783M 0 783M 0% /run/user/1001 02:11:20 02:11:20 02:11:20 ---> free -m: 02:11:20 total used free shared buff/cache available 02:11:20 Mem: 7821 451 5475 16 1893 7082 02:11:20 Swap: 1023 0 1023 02:11:20 02:11:20 02:11:20 ---> ip addr: 02:11:20 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 02:11:20 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 02:11:20 inet 127.0.0.1/8 scope host lo 02:11:20 valid_lft forever preferred_lft forever 02:11:20 inet6 ::1/128 scope host 02:11:20 valid_lft forever preferred_lft forever 02:11:20 2: eth0: mtu 1458 qdisc mq state UP group default qlen 1000 02:11:20 link/ether fa:16:3e:a9:0b:0f brd ff:ff:ff:ff:ff:ff 02:11:20 inet 10.30.171.77/23 brd 10.30.171.255 scope global dynamic eth0 02:11:20 valid_lft 86212sec preferred_lft 86212sec 02:11:20 inet6 fe80::f816:3eff:fea9:b0f/64 scope link 02:11:20 valid_lft forever preferred_lft forever 02:11:20 02:11:20 02:11:20 ---> sar -b -r -n DEV: 02:11:20 Linux 3.10.0-1160.118.1.el7.x86_64 (centos-7-robot-05cc0f9b-716b-4844-b983-0ce161c05ab8.novalocal) 09/23/2024 _x86_64_ (2 CPU) 02:11:20 02:11:20 02:08:06 AM LINUX RESTART 02:11:20 02:11:20 02:09:02 AM tps rtps wtps bread/s bwrtn/s 02:11:20 02:10:01 AM 130.08 6.52 123.56 746.08 6417.68 02:11:20 02:11:01 AM 288.79 14.79 274.00 676.88 8349.47 02:11:20 Average: 210.09 10.69 199.40 711.19 7391.60 02:11:20 02:11:20 02:09:02 AM kbmemfree kbmemused %memused kbbuffers kbcached kbcommit %commit kbactive kbinact kbdirty 02:11:20 02:10:01 AM 5646904 2361832 29.49 2620 1814256 587944 6.49 480708 1586244 158904 02:11:20 02:11:01 AM 5668624 2340112 29.22 2620 1824552 543812 6.00 561848 1473192 13652 02:11:20 Average: 5657764 2350972 29.36 2620 1819404 565878 6.25 521278 1529718 86278 02:11:20 02:11:20 02:09:02 AM IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s 02:11:20 02:10:01 AM eth0 64.72 59.63 692.55 6.99 0.00 0.00 0.00 02:11:20 02:10:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:11:20 02:11:01 AM eth0 53.38 49.05 339.03 9.44 0.00 0.00 0.00 02:11:20 02:11:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:11:20 Average: eth0 59.01 54.30 514.32 8.23 0.00 0.00 0.00 02:11:20 Average: lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:11:20 02:11:20 02:11:20 ---> sar -P ALL: 02:11:20 Linux 3.10.0-1160.118.1.el7.x86_64 (centos-7-robot-05cc0f9b-716b-4844-b983-0ce161c05ab8.novalocal) 09/23/2024 _x86_64_ (2 CPU) 02:11:20 02:11:20 02:08:06 AM LINUX RESTART 02:11:20 02:11:20 02:09:02 AM CPU %user %nice %system %iowait %steal %idle 02:11:20 02:10:01 AM all 31.32 0.00 2.97 5.17 0.12 60.42 02:11:20 02:10:01 AM 0 54.29 0.00 4.45 6.61 0.11 34.54 02:11:20 02:10:01 AM 1 9.66 0.00 1.59 3.80 0.12 84.83 02:11:20 02:11:01 AM all 28.55 0.00 2.12 10.36 0.11 58.86 02:11:20 02:11:01 AM 0 51.33 0.00 2.76 11.45 0.13 34.34 02:11:20 02:11:01 AM 1 7.14 0.00 1.51 9.34 0.10 81.92 02:11:20 Average: all 29.92 0.00 2.54 7.79 0.11 59.63 02:11:20 Average: 0 52.79 0.00 3.60 9.05 0.12 34.44 02:11:20 Average: 1 8.39 0.00 1.55 6.60 0.11 83.36 02:11:20 02:11:20 02:11:20