15:46:35 Started by upstream project "integration-distribution-test-scandium" build number 382
15:46:35 originally caused by:
15:46:35 Started by upstream project "autorelease-release-scandium-mvn39-openjdk21" build number 395
15:46:35 originally caused by:
15:46:35 Started by timer
15:46:35 Started by upstream project "autorelease-release-scandium-mvn39-openjdk21" build number 396
15:46:35 originally caused by:
15:46:35 Started by timer
15:46:35 Running as SYSTEM
15:46:35 [EnvInject] - Loading node environment variables.
15:46:35 Building remotely on prd-ubuntu2204-docker-4c-4g-25284 (ubuntu2204-docker-4c-4g) in workspace /w/workspace/distribution-csit-managed-scandium
15:46:35 [ssh-agent] Looking for ssh-agent implementation...
15:46:36 [ssh-agent] Exec ssh-agent (binary ssh-agent on a remote machine)
15:46:36 $ ssh-agent
15:46:36 SSH_AUTH_SOCK=/tmp/ssh-XXXXXXOBY7ax/agent.1566
15:46:36 SSH_AGENT_PID=1568
15:46:36 [ssh-agent] Started.
15:46:36 Running ssh-add (command line suppressed)
15:46:36 Identity added: /w/workspace/distribution-csit-managed-scandium@tmp/private_key_14390900121112775218.key (/w/workspace/distribution-csit-managed-scandium@tmp/private_key_14390900121112775218.key)
15:46:36 [ssh-agent] Using credentials jenkins (Release Engineering Jenkins Key)
15:46:36 No emails were triggered.
15:46:36 provisioning config files...
15:46:36 copy managed file [npmrc] to file:/home/jenkins/.npmrc
15:46:36 copy managed file [pipconf] to file:/home/jenkins/.config/pip/pip.conf
15:46:36 copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
15:46:36 [distribution-csit-managed-scandium] $ /bin/sh /tmp/jenkins15990607946580506836.sh
15:46:36 Karaf artifact is karaf
15:46:36 Karaf project is integration
15:46:36 Java home is
15:46:36 [EnvInject] - Injecting environment variables from a build step.
15:46:36 [EnvInject] - Injecting as environment variables the properties file path 'set_variables.env'
15:46:36 [EnvInject] - Variables injected successfully.
15:46:36 [distribution-csit-managed-scandium] $ /bin/bash /tmp/jenkins9465739144549619986.sh
15:46:36 2025-09-01 15:46:36 URL:https://raw.githubusercontent.com/opendaylight/integration-distribution/stable/scandium/pom.xml [2619/2619] -> "pom.xml" [1]
15:46:36 Bundle version is 0.21.3-SNAPSHOT
15:46:36 --2025-09-01 15:46:36-- https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.3-SNAPSHOT/maven-metadata.xml
15:46:36 Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6
15:46:36 Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected.
15:46:36 HTTP request sent, awaiting response... 200 OK
15:46:36 Length: 1410 (1.4K) [application/xml]
15:46:36 Saving to: ‘maven-metadata.xml’
15:46:36
15:46:36 0K . 100% 1.43G=0s
15:46:36
15:46:36 2025-09-01 15:46:36 (1.43 GB/s) - ‘maven-metadata.xml’ saved [1410/1410]
15:46:36
15:46:36
15:46:36
15:46:36 org.opendaylight.integration
15:46:36 karaf
15:46:36 0.21.3-SNAPSHOT
15:46:36
15:46:36
15:46:36 20250901.130015
15:46:36 176
15:46:36
15:46:36 20250901130015
15:46:36
15:46:36
15:46:36 pom
15:46:36 0.21.3-20250901.130015-176
15:46:36 20250901130015
15:46:36
15:46:36
15:46:36 tar.gz
15:46:36 0.21.3-20250901.130015-176
15:46:36 20250901130015
15:46:36
15:46:36
15:46:36 zip
15:46:36 0.21.3-20250901.130015-176
15:46:36 20250901130015
15:46:36
15:46:36
15:46:36 cyclonedx
15:46:36 xml
15:46:36 0.21.3-20250901.130015-176
15:46:36 20250901130015
15:46:36
15:46:36
15:46:36 cyclonedx
15:46:36 json
15:46:36 0.21.3-20250901.130015-176
15:46:36 20250901130015
15:46:36
15:46:36
15:46:36
15:46:36
15:46:37 Nexus timestamp is 0.21.3-20250901.130015-176
15:46:37 Distribution bundle URL is https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.3-SNAPSHOT/karaf-0.21.3-20250901.130015-176.zip
15:46:37 Distribution bundle is karaf-0.21.3-20250901.130015-176.zip
15:46:37 Distribution bundle version is 0.21.3-SNAPSHOT
15:46:37 Distribution folder is karaf-0.21.3-SNAPSHOT
15:46:37 Nexus prefix is https://nexus.opendaylight.org
15:46:37 [EnvInject] - Injecting environment variables from a build step.
15:46:37 [EnvInject] - Injecting as environment variables the properties file path 'detect_variables.env'
15:46:37 [EnvInject] - Variables injected successfully.
15:46:37 [EnvInject] - Injecting environment variables from a build step.
15:46:37 [EnvInject] - Injecting as environment variables the properties content
15:46:37 SET_JDK_VERSION=openjdk21
15:46:37 GIT_URL="git://devvexx.opendaylight.org/mirror"
15:46:37
15:46:37 [EnvInject] - Variables injected successfully.
15:46:37 [distribution-csit-managed-scandium] $ /bin/sh /tmp/jenkins14967355688268020916.sh
15:46:37 ---> update-java-alternatives.sh
15:46:37 ---> Updating Java version
15:46:37 ---> Ubuntu/Debian system detected
15:46:37 update-alternatives: using /usr/lib/jvm/java-21-openjdk-amd64/bin/java to provide /usr/bin/java (java) in manual mode
15:46:37 update-alternatives: using /usr/lib/jvm/java-21-openjdk-amd64/bin/javac to provide /usr/bin/javac (javac) in manual mode
15:46:37 update-alternatives: using /usr/lib/jvm/java-21-openjdk-amd64 to provide /usr/lib/jvm/java-openjdk (java_sdk_openjdk) in manual mode
15:46:37 openjdk version "21.0.5" 2024-10-15
15:46:37 OpenJDK Runtime Environment (build 21.0.5+11-Ubuntu-1ubuntu122.04)
15:46:37 OpenJDK 64-Bit Server VM (build 21.0.5+11-Ubuntu-1ubuntu122.04, mixed mode, sharing)
15:46:37 JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
15:46:37 [EnvInject] - Injecting environment variables from a build step.
15:46:37 [EnvInject] - Injecting as environment variables the properties file path '/tmp/java.env'
15:46:37 [EnvInject] - Variables injected successfully.
15:46:37 [distribution-csit-managed-scandium] $ /bin/sh /tmp/jenkins11542859406000507065.sh
15:46:37 0
15:46:37 Kill any controller running
15:46:37 Clean Existing distribution
15:46:37 Fetch the distribution...
15:46:37 --2025-09-01 15:46:37-- https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.3-SNAPSHOT/karaf-0.21.3-20250901.130015-176.zip
15:46:37 Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6
15:46:37 Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected.
15:46:37 HTTP request sent, awaiting response... 200 OK
15:46:37 Length: 187827154 (179M) [application/zip]
15:46:37 Saving to: ‘karaf-0.21.3-20250901.130015-176.zip’
15:46:37
15:46:37 0K ........ ........ ........ ........ ........ ........ 1% 97.5M 2s
15:46:37 3072K ........ ........ ........ ........ ........ ........ 3% 153M 1s
15:46:37 6144K ........ ........ ........ ........ ........ ........ 5% 129M 1s
15:46:37 9216K ........ ........ ........ ........ ........ ........ 6% 228M 1s
15:46:37 12288K ........ ........ ........ ........ ........ ........ 8% 194M 1s
15:46:37 15360K ........ ........ ........ ........ ........ ........ 10% 122M 1s
15:46:37 18432K ........ ........ ........ ........ ........ ........ 11% 183M 1s
15:46:37 21504K ........ ........ ........ ........ ........ ........ 13% 123M 1s
15:46:37 24576K ........ ........ ........ ........ ........ ........ 15% 211M 1s
15:46:37 27648K ........ ........ ........ ........ ........ ........ 16% 212M 1s
15:46:37 30720K ........ ........ ........ ........ ........ ........ 18% 266M 1s
15:46:37 33792K ........ ........ ........ ........ ........ ........ 20% 270M 1s
15:46:37 36864K ........ ........ ........ ........ ........ ........ 21% 283M 1s
15:46:37 39936K ........ ........ ........ ........ ........ ........ 23% 260M 1s
15:46:37 43008K ........ ........ ........ ........ ........ ........ 25% 282M 1s
15:46:37 46080K ........ ........ ........ ........ ........ ........ 26% 248M 1s
15:46:37 49152K ........ ........ ........ ........ ........ ........ 28% 245M 1s
15:46:37 52224K ........ ........ ........ ........ ........ ........ 30% 217M 1s
15:46:37 55296K ........ ........ ........ ........ ........ ........ 31% 275M 1s
15:46:37 58368K ........ ........ ........ ........ ........ ........ 33% 302M 1s
15:46:37 61440K ........ ........ ........ ........ ........ ........ 35% 325M 1s
15:46:37 64512K ........ ........ ........ ........ ........ ........ 36% 332M 1s
15:46:37 67584K ........ ........ ........ ........ ........ ........ 38% 329M 1s
15:46:37 70656K ........ ........ ........ ........ ........ ........ 40% 327M 1s
15:46:37 73728K ........ ........ ........ ........ ........ ........ 41% 230M 0s
15:46:37 76800K ........ ........ ........ ........ ........ ........ 43% 236M 0s
15:46:37 79872K ........ ........ ........ ........ ........ ........ 45% 392M 0s
15:46:37 82944K ........ ........ ........ ........ ........ ........ 46% 360M 0s
15:46:37 86016K ........ ........ ........ ........ ........ ........ 48% 333M 0s
15:46:37 89088K ........ ........ ........ ........ ........ ........ 50% 334M 0s
15:46:37 92160K ........ ........ ........ ........ ........ ........ 51% 332M 0s
15:46:38 95232K ........ ........ ........ ........ ........ ........ 53% 319M 0s
15:46:38 98304K ........ ........ ........ ........ ........ ........ 55% 327M 0s
15:46:38 101376K ........ ........ ........ ........ ........ ........ 56% 314M 0s
15:46:38 104448K ........ ........ ........ ........ ........ ........ 58% 308M 0s
15:46:38 107520K ........ ........ ........ ........ ........ ........ 60% 307M 0s
15:46:38 110592K ........ ........ ........ ........ ........ ........ 61% 249M 0s
15:46:38 113664K ........ ........ ........ ........ ........ ........ 63% 274M 0s
15:46:38 116736K ........ ........ ........ ........ ........ ........ 65% 318M 0s
15:46:38 119808K ........ ........ ........ ........ ........ ........ 66% 309M 0s
15:46:38 122880K ........ ........ ........ ........ ........ ........ 68% 264M 0s
15:46:38 125952K ........ ........ ........ ........ ........ ........ 70% 312M 0s
15:46:38 129024K ........ ........ ........ ........ ........ ........ 72% 322M 0s
15:46:38 132096K ........ ........ ........ ........ ........ ........ 73% 317M 0s
15:46:38 135168K ........ ........ ........ ........ ........ ........ 75% 340M 0s
15:46:38 138240K ........ ........ ........ ........ ........ ........ 77% 200M 0s
15:46:38 141312K ........ ........ ........ ........ ........ ........ 78% 210M 0s
15:46:38 144384K ........ ........ ........ ........ ........ ........ 80% 299M 0s
15:46:38 147456K ........ ........ ........ ........ ........ ........ 82% 343M 0s
15:46:38 150528K ........ ........ ........ ........ ........ ........ 83% 335M 0s
15:46:38 153600K ........ ........ ........ ........ ........ ........ 85% 299M 0s
15:46:38 156672K ........ ........ ........ ........ ........ ........ 87% 316M 0s
15:46:38 159744K ........ ........ ........ ........ ........ ........ 88% 330M 0s
15:46:38 162816K ........ ........ ........ ........ ........ ........ 90% 344M 0s
15:46:38 165888K ........ ........ ........ ........ ........ ........ 92% 347M 0s
15:46:38 168960K ........ ........ ........ ........ ........ ........ 93% 347M 0s
15:46:38 172032K ........ ........ ........ ........ ........ ........ 95% 351M 0s
15:46:38 175104K ........ ........ ........ ........ ........ ........ 97% 352M 0s
15:46:38 178176K ........ ........ ........ ........ ........ ........ 98% 342M 0s
15:46:38 181248K ........ ........ ........ ........ .. 100% 361M=0.7s
15:46:38
15:46:38 2025-09-01 15:46:38 (257 MB/s) - ‘karaf-0.21.3-20250901.130015-176.zip’ saved [187827154/187827154]
15:46:38
15:46:38 Extracting the new controller...
15:46:39 Configuring the startup features...
15:46:39 ################################################################################
15:46:39 #
15:46:39 # Licensed to the Apache Software Foundation (ASF) under one or more
15:46:39 # contributor license agreements. See the NOTICE file distributed with
15:46:39 # this work for additional information regarding copyright ownership.
15:46:39 # The ASF licenses this file to You under the Apache License, Version 2.0
15:46:39 # (the "License"); you may not use this file except in compliance with
15:46:39 # the License. You may obtain a copy of the License at
15:46:39 #
15:46:39 # http://www.apache.org/licenses/LICENSE-2.0
15:46:39 #
15:46:39 # Unless required by applicable law or agreed to in writing, software
15:46:39 # distributed under the License is distributed on an "AS IS" BASIS,
15:46:39 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15:46:39 # See the License for the specific language governing permissions and
15:46:39 # limitations under the License.
15:46:39 #
15:46:39 ################################################################################
15:46:39
15:46:39 #
15:46:39 # Comma separated list of features repositories to register by default
15:46:39 #
15:46:39 featuresRepositories = mvn:org.opendaylight.integration/features-test/0.21.3-SNAPSHOT/xml/features,file:${karaf.etc}/abbcfa12-f854-44ec-9098-33105aa1b176.xml
15:46:39
15:46:39 #
15:46:39 # Comma separated list of features to install at startup
15:46:39 #
15:46:39 featuresBoot = odl-integration-all,af20f0cb-1cf7-4444-97f6-33ed740a4b71
15:46:39
15:46:39 #
15:46:39 # Resource repositories (OBR) that the features resolver can use
15:46:39 # to resolve requirements/capabilities
15:46:39 #
15:46:39 # The format of the resourceRepositories is
15:46:39 # resourceRepositories=[xml:url|json:url],...
15:46:39 # for Instance:
15:46:39 #
15:46:39 #resourceRepositories=xml:http://host/path/to/index.xml
15:46:39 # or
15:46:39 #resourceRepositories=json:http://host/path/to/index.json
15:46:39 #
15:46:39
15:46:39 #
15:46:39 # Defines if the boot features are started in asynchronous mode (in a dedicated thread)
15:46:39 #
15:46:39 featuresBootAsynchronous=false
15:46:39
15:46:39 #
15:46:39 # Service requirements enforcement
15:46:39 #
15:46:39 # By default, the feature resolver checks the service requirements/capabilities of
15:46:39 # bundles for new features (xml schema >= 1.3.0) in order to automatically installs
15:46:39 # the required bundles.
15:46:39 # The following flag can have those values:
15:46:39 # - disable: service requirements are completely ignored
15:46:39 # - default: service requirements are ignored for old features
15:46:39 # - enforce: service requirements are always verified
15:46:39 #
15:46:39 #serviceRequirements=default
15:46:39
15:46:39 #
15:46:39 # Store cfg file for config element in feature
15:46:39 #
15:46:39 #configCfgStore=true
15:46:39
15:46:39 #
15:46:39 # Define if the feature service automatically refresh bundles
15:46:39 #
15:46:39 autoRefresh=true
15:46:39
15:46:39 #
15:46:39 # Configuration of features processing mechanism (overrides, blacklisting, modification of features)
15:46:39 # XML file defines instructions related to features processing
15:46:39 # versions.properties may declare properties to resolve placeholders in XML file
15:46:39 # both files are relative to ${karaf.etc}
15:46:39 #
15:46:39 #featureProcessing=org.apache.karaf.features.xml
15:46:39 #featureProcessingVersions=versions.properties
15:46:39 Configuring the log...
15:46:39 ################################################################################
15:46:39 #
15:46:39 # Licensed to the Apache Software Foundation (ASF) under one or more
15:46:39 # contributor license agreements. See the NOTICE file distributed with
15:46:39 # this work for additional information regarding copyright ownership.
15:46:39 # The ASF licenses this file to You under the Apache License, Version 2.0
15:46:39 # (the "License"); you may not use this file except in compliance with
15:46:39 # the License. You may obtain a copy of the License at
15:46:39 #
15:46:39 # http://www.apache.org/licenses/LICENSE-2.0
15:46:39 #
15:46:39 # Unless required by applicable law or agreed to in writing, software
15:46:39 # distributed under the License is distributed on an "AS IS" BASIS,
15:46:39 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15:46:39 # See the License for the specific language governing permissions and
15:46:39 # limitations under the License.
15:46:39 #
15:46:39 ################################################################################
15:46:39
15:46:39 # Common pattern layout for appenders
15:46:39 log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
15:46:39
15:46:39 # Root logger
15:46:39 log4j2.rootLogger.level = INFO
15:46:39 # uncomment to use asynchronous loggers, which require mvn:com.lmax/disruptor/3.3.2 library
15:46:39 #log4j2.rootLogger.type = asyncRoot
15:46:39 #log4j2.rootLogger.includeLocation = false
15:46:39 log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile
15:46:39 log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi
15:46:39 log4j2.rootLogger.appenderRef.Console.ref = Console
15:46:39 log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter
15:46:39 log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF}
15:46:39
15:46:39 # Filters for logs marked by org.opendaylight.odlparent.Markers
15:46:39 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter
15:46:39 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair
15:46:39 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker
15:46:39 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL
15:46:39 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or
15:46:39 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY
15:46:39 log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL
15:46:39
15:46:39 # Loggers configuration
15:46:39
15:46:39 # Spifly logger
15:46:39 log4j2.logger.spifly.name = org.apache.aries.spifly
15:46:39 log4j2.logger.spifly.level = WARN
15:46:39
15:46:39 # Security audit logger
15:46:39 log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit
15:46:39 log4j2.logger.audit.level = INFO
15:46:39 log4j2.logger.audit.additivity = false
15:46:39 log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile
15:46:39
15:46:39 # Appenders configuration
15:46:39
15:46:39 # Console appender not used by default (see log4j2.rootLogger.appenderRefs)
15:46:39 log4j2.appender.console.type = Console
15:46:39 log4j2.appender.console.name = Console
15:46:39 log4j2.appender.console.layout.type = PatternLayout
15:46:39 log4j2.appender.console.layout.pattern = ${log4j2.pattern}
15:46:39
15:46:39 # Rolling file appender
15:46:39 log4j2.appender.rolling.type = RollingRandomAccessFile
15:46:39 log4j2.appender.rolling.name = RollingFile
15:46:39 log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
15:46:39 log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i
15:46:39 # uncomment to not force a disk flush
15:46:39 #log4j2.appender.rolling.immediateFlush = false
15:46:39 log4j2.appender.rolling.append = true
15:46:39 log4j2.appender.rolling.layout.type = PatternLayout
15:46:39 log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
15:46:39 log4j2.appender.rolling.policies.type = Policies
15:46:39 log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
15:46:39 log4j2.appender.rolling.policies.size.size = 64MB
15:46:39 log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
15:46:39 log4j2.appender.rolling.strategy.max = 7
15:46:39
15:46:39 # Audit file appender
15:46:39 log4j2.appender.audit.type = RollingRandomAccessFile
15:46:39 log4j2.appender.audit.name = AuditRollingFile
15:46:39 log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log
15:46:39 log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i
15:46:39 log4j2.appender.audit.append = true
15:46:39 log4j2.appender.audit.layout.type = PatternLayout
15:46:39 log4j2.appender.audit.layout.pattern = ${log4j2.pattern}
15:46:39 log4j2.appender.audit.policies.type = Policies
15:46:39 log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy
15:46:39 log4j2.appender.audit.policies.size.size = 8MB
15:46:39 log4j2.appender.audit.strategy.type = DefaultRolloverStrategy
15:46:39 log4j2.appender.audit.strategy.max = 7
15:46:39
15:46:39 # OSGi appender
15:46:39 log4j2.appender.osgi.type = PaxOsgi
15:46:39 log4j2.appender.osgi.name = PaxOsgi
15:46:39 log4j2.appender.osgi.filter = *
15:46:39
15:46:39 # help with identification of maven-related problems with pax-url-aether
15:46:39 #log4j2.logger.aether.name = shaded.org.eclipse.aether
15:46:39 #log4j2.logger.aether.level = TRACE
15:46:39 #log4j2.logger.http-headers.name = shaded.org.apache.http.headers
15:46:39 #log4j2.logger.http-headers.level = DEBUG
15:46:39 #log4j2.logger.maven.name = org.ops4j.pax.url.mvn
15:46:39 #log4j2.logger.maven.level = TRACE
15:46:39 Configure max memory...
15:46:39 #!/bin/sh
15:46:39 #
15:46:39 # Licensed to the Apache Software Foundation (ASF) under one or more
15:46:39 # contributor license agreements. See the NOTICE file distributed with
15:46:39 # this work for additional information regarding copyright ownership.
15:46:39 # The ASF licenses this file to You under the Apache License, Version 2.0
15:46:39 # (the "License"); you may not use this file except in compliance with
15:46:39 # the License. You may obtain a copy of the License at
15:46:39 #
15:46:39 # http://www.apache.org/licenses/LICENSE-2.0
15:46:39 #
15:46:39 # Unless required by applicable law or agreed to in writing, software
15:46:39 # distributed under the License is distributed on an "AS IS" BASIS,
15:46:39 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
15:46:39 # See the License for the specific language governing permissions and
15:46:39 # limitations under the License.
15:46:39 #
15:46:39
15:46:39 #
15:46:39 # handle specific scripts; the SCRIPT_NAME is exactly the name of the Karaf
15:46:39 # script: client, instance, shell, start, status, stop, karaf
15:46:39 #
15:46:39 # if [ "${KARAF_SCRIPT}" == "SCRIPT_NAME" ]; then
15:46:39 # Actions go here...
15:46:39 # fi
15:46:39
15:46:39 #
15:46:39 # general settings which should be applied for all scripts go here; please keep
15:46:39 # in mind that it is possible that scripts might be executed more than once, e.g.
15:46:39 # in example of the start script where the start script is executed first and the
15:46:39 # karaf script afterwards.
15:46:39 #
15:46:39
15:46:39 #
15:46:39 # The following section shows the possible configuration options for the default
15:46:39 # karaf scripts
15:46:39 #
15:46:39 # export JAVA_HOME # Location of Java installation
15:46:39 # export JAVA_OPTS # Generic JVM options, for instance, where you can pass the memory configuration
15:46:39 # export JAVA_NON_DEBUG_OPTS # Additional non-debug JVM options
15:46:39 # export EXTRA_JAVA_OPTS # Additional JVM options
15:46:39 # export KARAF_HOME # Karaf home folder
15:46:39 # export KARAF_DATA # Karaf data folder
15:46:39 # export KARAF_BASE # Karaf base folder
15:46:39 # export KARAF_ETC # Karaf etc folder
15:46:39 # export KARAF_LOG # Karaf log folder
15:46:39 # export KARAF_SYSTEM_OPTS # First citizen Karaf options
15:46:39 # export KARAF_OPTS # Additional available Karaf options
15:46:39 # export KARAF_DEBUG # Enable debug mode
15:46:39 # export KARAF_REDIRECT # Enable/set the std/err redirection when using bin/start
15:46:39 # export KARAF_NOROOT # Prevent execution as root if set to true
15:46:39 Listing all open ports on controller system
15:46:39 /tmp/jenkins11542859406000507065.sh: 70: netstat: not found
15:46:39 Sourcing Java environment variable ..
15:46:39 openjdk version "21.0.5" 2024-10-15
15:46:39 OpenJDK Runtime Environment (build 21.0.5+11-Ubuntu-1ubuntu122.04)
15:46:39 OpenJDK 64-Bit Server VM (build 21.0.5+11-Ubuntu-1ubuntu122.04, mixed mode, sharing)
15:46:39 JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
15:46:39 Redirecting karaf console output to karaf_console.log
15:46:39 Starting controller...
15:46:39 start: Redirecting Karaf output to /w/workspace/distribution-csit-managed-scandium/karaf-0.21.3-SNAPSHOT/data/log/karaf_console.log
15:46:39 Waiting up to 6 minutes for controller to come up, checking every 5 seconds...
15:46:44 already waited 5 seconds...
15:46:49 already waited 10 seconds...
15:46:54 already waited 15 seconds...
15:46:59 already waited 20 seconds...
15:47:05 already waited 25 seconds...
15:47:05 Controller is UP
15:47:05 Listing all open ports on controller system
15:47:05 /tmp/jenkins11542859406000507065.sh: 116: netstat: not found
15:47:05 looking for "Error installing boot feature repository" in karaf.log file
15:47:05 looking for "Error installing boot feature repository" in karaf_console.log file
15:47:05 looking for "BindException: Address already in use" in karaf.log file
15:47:05 looking for "BindException: Address already in use" in karaf_console.log file
15:47:05 looking for "server is unhealthy" in karaf.log file
15:47:05 looking for "server is unhealthy" in karaf_console.log file
15:47:05 Fetching Karaf logs
15:47:05 Kill controller
15:47:05 Bug 4628: Detecting misplaced config files
15:47:05 Bug 4628 confirmed.
15:47:05 $ ssh-agent -k
15:47:05 unset SSH_AUTH_SOCK;
15:47:05 unset SSH_AGENT_PID;
15:47:05 echo Agent pid 1568 killed;
15:47:05 [ssh-agent] Stopped.
15:47:05 [PostBuildScript] - [INFO] Executing post build scripts.
15:47:05 [distribution-csit-managed-scandium] $ /bin/bash /tmp/jenkins15313632811033735639.sh
15:47:05 Archiving csit artifacts
15:47:05 seq: invalid floating point argument: ‘’
15:47:05 Try 'seq --help' for more information.
15:47:05 % Total % Received % Xferd Average Speed Time Time Time Current
15:47:05 Dload Upload Total Spent Left Speed
15:47:05
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
100 237 0 237 0 0 1686 0 --:--:-- --:--:-- --:--:-- 1692
15:47:05 Archive: robot-plugin.zip
15:47:05 End-of-central-directory signature not found. Either this file is not
15:47:05 a zipfile, or it constitutes one disk of a multi-part archive. In the
15:47:05 latter case the central directory and zipfile comment will be found on
15:47:05 the last disk(s) of this archive.
15:47:05 unzip: cannot find zipfile directory in one of robot-plugin.zip or
15:47:05 robot-plugin.zip.zip, and cannot find robot-plugin.zip.ZIP, period.
15:47:05 mv: cannot stat '*.log.gz': No such file or directory
15:47:05 mv: cannot stat '*.csv': No such file or directory
15:47:05 mv: cannot stat '*.png': No such file or directory
15:47:05 [PostBuildScript] - [INFO] Executing post build scripts.
15:47:05 [distribution-csit-managed-scandium] $ /bin/bash /tmp/jenkins15423314637402623638.sh
15:47:05 ---> sysstat.sh
15:47:05 [distribution-csit-managed-scandium] $ /bin/bash /tmp/jenkins1594081525978955593.sh
15:47:05 ---> package-listing.sh
15:47:05 ++ facter osfamily
15:47:05 ++ tr '[:upper:]' '[:lower:]'
15:47:06 + OS_FAMILY=debian
15:47:06 + workspace=/w/workspace/distribution-csit-managed-scandium
15:47:06 + START_PACKAGES=/tmp/packages_start.txt
15:47:06 + END_PACKAGES=/tmp/packages_end.txt
15:47:06 + DIFF_PACKAGES=/tmp/packages_diff.txt
15:47:06 + PACKAGES=/tmp/packages_start.txt
15:47:06 + '[' /w/workspace/distribution-csit-managed-scandium ']'
15:47:06 + PACKAGES=/tmp/packages_end.txt
15:47:06 + case "${OS_FAMILY}" in
15:47:06 + dpkg -l
15:47:06 + grep '^ii'
15:47:06 + '[' -f /tmp/packages_start.txt ']'
15:47:06 + '[' -f /tmp/packages_end.txt ']'
15:47:06 + diff /tmp/packages_start.txt /tmp/packages_end.txt
15:47:06 + '[' /w/workspace/distribution-csit-managed-scandium ']'
15:47:06 + mkdir -p /w/workspace/distribution-csit-managed-scandium/archives/
15:47:06 + cp -f /tmp/packages_diff.txt /tmp/packages_end.txt /tmp/packages_start.txt /w/workspace/distribution-csit-managed-scandium/archives/
15:47:06 [distribution-csit-managed-scandium] $ /bin/bash /tmp/jenkins12229106114289298160.sh
15:47:06 ---> capture-instance-metadata.sh
15:47:06 Setup pyenv:
15:47:06 * system (set by /opt/pyenv/version)
15:47:06 * 3.8.20 (set by /opt/pyenv/version)
15:47:06 * 3.9.20 (set by /opt/pyenv/version)
15:47:06 * 3.10.15 (set by /opt/pyenv/version)
15:47:06 * 3.11.10 (set by /opt/pyenv/version)
15:47:10 lf-activate-venv(): INFO: Creating python3 venv at /tmp/venv-ahSY
15:47:10 lf-activate-venv(): INFO: Save venv in file: /tmp/.os_lf_venv
15:47:15 lf-activate-venv(): INFO: Installing: lftools
15:47:42 lf-activate-venv(): INFO: Adding /tmp/venv-ahSY/bin to PATH
15:47:42 INFO: Running in OpenStack, capturing instance metadata
15:47:42 [distribution-csit-managed-scandium] $ /bin/bash /tmp/jenkins14992890907493941509.sh
15:47:42 provisioning config files...
15:47:42 Could not find credentials [logs] for distribution-csit-managed-scandium #375
15:47:42 copy managed file [jenkins-log-archives-settings] to file:/w/workspace/distribution-csit-managed-scandium@tmp/config3343201482110666087tmp
15:47:42 Regular expression run condition: Expression=[^.*logs-s3.*], Label=[odl-logs-s3-cloudfront-index]
15:47:42 Run condition [Regular expression match] enabling perform for step [Provide Configuration files]
15:47:42 provisioning config files...
15:47:43 copy managed file [jenkins-s3-log-ship] to file:/home/jenkins/.aws/credentials
15:47:43 [EnvInject] - Injecting environment variables from a build step.
15:47:43 [EnvInject] - Injecting as environment variables the properties content
15:47:43 SERVER_ID=logs
15:47:43
15:47:43 [EnvInject] - Variables injected successfully.
15:47:43 [distribution-csit-managed-scandium] $ /bin/bash /tmp/jenkins1353877564849048445.sh
15:47:43 ---> create-netrc.sh
15:47:43 WARN: Log server credential not found.
15:47:43 [distribution-csit-managed-scandium] $ /bin/bash /tmp/jenkins8755249300108869245.sh
15:47:43 ---> python-tools-install.sh
15:47:43 Setup pyenv:
15:47:43 system
15:47:43 3.8.20
15:47:43 3.9.20
15:47:43 3.10.15
15:47:43 * 3.11.10 (set by /w/workspace/distribution-csit-managed-scandium/.python-version)
15:47:43 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-ahSY from file:/tmp/.os_lf_venv
15:47:45 lf-activate-venv(): INFO: Installing: lftools
15:47:54 lf-activate-venv(): INFO: Adding /tmp/venv-ahSY/bin to PATH
15:47:54 Generating Requirements File
15:48:16 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
15:48:16 httplib2 0.30.0 requires pyparsing<4,>=3.0.4, but you have pyparsing 2.4.7 which is incompatible.
15:48:16 Python 3.11.10
15:48:16 pip 25.2 from /tmp/venv-ahSY/lib/python3.11/site-packages/pip (python 3.11)
15:48:17 appdirs==1.4.4
15:48:17 argcomplete==3.6.2
15:48:17 aspy.yaml==1.3.0
15:48:17 attrs==25.3.0
15:48:17 autopage==0.5.2
15:48:17 beautifulsoup4==4.13.5
15:48:17 boto3==1.40.21
15:48:17 botocore==1.40.21
15:48:17 bs4==0.0.2
15:48:17 cachetools==5.5.2
15:48:17 certifi==2025.8.3
15:48:17 cffi==1.17.1
15:48:17 cfgv==3.4.0
15:48:17 chardet==5.2.0
15:48:17 charset-normalizer==3.4.3
15:48:17 click==8.2.1
15:48:17 cliff==4.11.0
15:48:17 cmd2==2.7.0
15:48:17 cryptography==3.3.2
15:48:17 debtcollector==3.0.0
15:48:17 decorator==5.2.1
15:48:17 defusedxml==0.7.1
15:48:17 Deprecated==1.2.18
15:48:17 distlib==0.4.0
15:48:17 dnspython==2.7.0
15:48:17 docker==7.1.0
15:48:17 dogpile.cache==1.4.0
15:48:17 durationpy==0.10
15:48:17 email-validator==2.3.0
15:48:17 filelock==3.19.1
15:48:17 future==1.0.0
15:48:17 gitdb==4.0.12
15:48:17 GitPython==3.1.45
15:48:17 google-auth==2.40.3
15:48:17 httplib2==0.30.0
15:48:17 identify==2.6.13
15:48:17 idna==3.10
15:48:17 importlib-resources==1.5.0
15:48:17 iso8601==2.1.0
15:48:17 Jinja2==3.1.6
15:48:17 jmespath==1.0.1
15:48:17 jsonpatch==1.33
15:48:17 jsonpointer==3.0.0
15:48:17 jsonschema==4.25.1
15:48:17 jsonschema-specifications==2025.4.1
15:48:17 keystoneauth1==5.12.0
15:48:17 kubernetes==33.1.0
15:48:17 lftools==0.37.13
15:48:17 lxml==6.0.1
15:48:17 markdown-it-py==4.0.0
15:48:17 MarkupSafe==3.0.2
15:48:17 mdurl==0.1.2
15:48:17 msgpack==1.1.1
15:48:17 multi_key_dict==2.0.3
15:48:17 munch==4.0.0
15:48:17 netaddr==1.3.0
15:48:17 niet==1.4.2
15:48:17 nodeenv==1.9.1
15:48:17 oauth2client==4.1.3
15:48:17 oauthlib==3.3.1
15:48:17 openstacksdk==4.7.0
15:48:17 os-client-config==2.3.0
15:48:17 os-service-types==1.8.0
15:48:17 osc-lib==4.2.0
15:48:17 oslo.config==10.0.0
15:48:17 oslo.context==6.1.0
15:48:17 oslo.i18n==6.6.0
15:48:17 oslo.log==7.2.1
15:48:17 oslo.serialization==5.8.0
15:48:17 oslo.utils==9.1.0
15:48:17 packaging==25.0
15:48:17 pbr==7.0.1
15:48:17 platformdirs==4.4.0
15:48:17 prettytable==3.16.0
15:48:17 psutil==7.0.0
15:48:17 pyasn1==0.6.1
15:48:17 pyasn1_modules==0.4.2
15:48:17 pycparser==2.22
15:48:17 pygerrit2==2.0.15
15:48:17 PyGithub==2.7.0
15:48:17 Pygments==2.19.2
15:48:17 PyJWT==2.10.1
15:48:17 PyNaCl==1.5.0
15:48:17 pyparsing==2.4.7
15:48:17 pyperclip==1.9.0
15:48:17 pyrsistent==0.20.0
15:48:17 python-cinderclient==9.8.0
15:48:17 python-dateutil==2.9.0.post0
15:48:17 python-heatclient==4.3.0
15:48:17 python-jenkins==1.8.3
15:48:17 python-keystoneclient==5.7.0
15:48:17 python-magnumclient==4.8.1
15:48:17 python-openstackclient==8.2.0
15:48:17 python-swiftclient==4.8.0
15:48:17 PyYAML==6.0.2
15:48:17 referencing==0.36.2
15:48:17 requests==2.32.5
15:48:17 requests-oauthlib==2.0.0
15:48:17 requestsexceptions==1.4.0
15:48:17 rfc3986==2.0.0
15:48:17 rich==14.1.0
15:48:17 rich-argparse==1.7.1
15:48:17 rpds-py==0.27.1
15:48:17 rsa==4.9.1
15:48:17 ruamel.yaml==0.18.15
15:48:17 ruamel.yaml.clib==0.2.12
15:48:17 s3transfer==0.13.1
15:48:17 simplejson==3.20.1
15:48:17 six==1.17.0
15:48:17 smmap==5.0.2
15:48:17 soupsieve==2.8
15:48:17 stevedore==5.5.0
15:48:17 tabulate==0.9.0
15:48:17 toml==0.10.2
15:48:17 tomlkit==0.13.3
15:48:17 tqdm==4.67.1
15:48:17 typing_extensions==4.15.0
15:48:17 tzdata==2025.2
15:48:17 urllib3==1.26.20
15:48:17 virtualenv==20.34.0
15:48:17 wcwidth==0.2.13
15:48:17 websocket-client==1.8.0
15:48:17 wrapt==1.17.3
15:48:17 xdg==6.0.0
15:48:17 xmltodict==0.14.2
15:48:17 yq==3.4.3
15:48:17 [distribution-csit-managed-scandium] $ /bin/bash /tmp/jenkins11410185264407622989.sh
15:48:17 ---> sudo-logs.sh
15:48:17 Archiving 'sudo' log..
15:48:17 [distribution-csit-managed-scandium] $ /bin/bash /tmp/jenkins2633573022443306609.sh
15:48:17 ---> job-cost.sh
15:48:17 Setup pyenv:
15:48:17 system
15:48:17 3.8.20
15:48:17 3.9.20
15:48:17 3.10.15
15:48:17 * 3.11.10 (set by /w/workspace/distribution-csit-managed-scandium/.python-version)
15:48:17 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-ahSY from file:/tmp/.os_lf_venv
15:48:19 lf-activate-venv(): INFO: Installing: zipp==1.1.0 python-openstackclient urllib3~=1.26.15
15:48:25 lf-activate-venv(): INFO: Adding /tmp/venv-ahSY/bin to PATH
15:48:25 INFO: No Stack...
15:48:26 INFO: Retrieving Pricing Info for: v3-standard-4
15:48:26 INFO: Archiving Costs
15:48:26 [distribution-csit-managed-scandium] $ /bin/bash -l /tmp/jenkins11461743825278605992.sh
15:48:26 ---> logs-deploy.sh
15:48:26 Setup pyenv:
15:48:26 system
15:48:26 3.8.20
15:48:26 3.9.20
15:48:26 3.10.15
15:48:26 * 3.11.10 (set by /w/workspace/distribution-csit-managed-scandium/.python-version)
15:48:26 lf-activate-venv(): INFO: Reuse venv:/tmp/venv-ahSY from file:/tmp/.os_lf_venv
15:48:28 lf-activate-venv(): INFO: Installing: lftools
15:48:38 lf-activate-venv(): INFO: Adding /tmp/venv-ahSY/bin to PATH
15:48:38 WARNING: Nexus logging server not set
15:48:38 INFO: S3 path logs/releng/vex-yul-odl-jenkins-1/distribution-csit-managed-scandium/375/
15:48:38 INFO: archiving logs to S3
15:48:40 ---> uname -a:
15:48:40 Linux prd-ubuntu2204-docker-4c-4g-25284 5.15.0-131-generic #141-Ubuntu SMP Fri Jan 10 21:18:28 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
15:48:40
15:48:40
15:48:40 ---> lscpu:
15:48:40 Architecture: x86_64
15:48:40 CPU op-mode(s): 32-bit, 64-bit
15:48:40 Address sizes: 40 bits physical, 48 bits virtual
15:48:40 Byte Order: Little Endian
15:48:40 CPU(s): 4
15:48:40 On-line CPU(s) list: 0-3
15:48:40 Vendor ID: AuthenticAMD
15:48:40 Model name: AMD EPYC-Rome Processor
15:48:40 CPU family: 23
15:48:40 Model: 49
15:48:40 Thread(s) per core: 1
15:48:40 Core(s) per socket: 1
15:48:40 Socket(s): 4
15:48:40 Stepping: 0
15:48:40 BogoMIPS: 5600.00
15:48:40 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 clzero xsaveerptr wbnoinvd arat npt nrip_save umip rdpid arch_capabilities
15:48:40 Virtualization: AMD-V
15:48:40 Hypervisor vendor: KVM
15:48:40 Virtualization type: full
15:48:40 L1d cache: 128 KiB (4 instances)
15:48:40 L1i cache: 128 KiB (4 instances)
15:48:40 L2 cache: 2 MiB (4 instances)
15:48:40 L3 cache: 64 MiB (4 instances)
15:48:40 NUMA node(s): 1
15:48:40 NUMA node0 CPU(s): 0-3
15:48:40 Vulnerability Gather data sampling: Not affected
15:48:40 Vulnerability Itlb multihit: Not affected
15:48:40 Vulnerability L1tf: Not affected
15:48:40 Vulnerability Mds: Not affected
15:48:40 Vulnerability Meltdown: Not affected
15:48:40 Vulnerability Mmio stale data: Not affected
15:48:40 Vulnerability Reg file data sampling: Not affected
15:48:40 Vulnerability Retbleed: Mitigation; untrained return thunk; SMT disabled
15:48:40 Vulnerability Spec rstack overflow: Mitigation; SMT disabled
15:48:40 Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
15:48:40 Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
15:48:40 Vulnerability Spectre v2: Mitigation; Retpolines; IBPB conditional; STIBP disabled; RSB filling; PBRSB-eIBRS Not affected; BHI Not affected
15:48:40 Vulnerability Srbds: Not affected
15:48:40 Vulnerability Tsx async abort: Not affected
15:48:40
15:48:40
15:48:40 ---> nproc:
15:48:40 4
15:48:40
15:48:40
15:48:40 ---> df -h:
15:48:40 Filesystem Size Used Avail Use% Mounted on
15:48:40 tmpfs 1.6G 1.1M 1.6G 1% /run
15:48:40 /dev/vda1 78G 8.8G 69G 12% /
15:48:40 tmpfs 7.9G 0 7.9G 0% /dev/shm
15:48:40 tmpfs 5.0M 0 5.0M 0% /run/lock
15:48:40 /dev/vda15 105M 6.1M 99M 6% /boot/efi
15:48:40 tmpfs 1.6G 4.0K 1.6G 1% /run/user/1001
15:48:40
15:48:40
15:48:40 ---> free -m:
15:48:40 total used free shared buff/cache available
15:48:40 Mem: 15989 601 12436 4 2951 15053
15:48:40 Swap: 1023 0 1023
15:48:40
15:48:40
15:48:40 ---> ip addr:
15:48:40 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
15:48:40 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
15:48:40 inet 127.0.0.1/8 scope host lo
15:48:40 valid_lft forever preferred_lft forever
15:48:40 inet6 ::1/128 scope host
15:48:40 valid_lft forever preferred_lft forever
15:48:40 2: ens3: mtu 1458 qdisc mq state UP group default qlen 1000
15:48:40 link/ether fa:16:3e:43:46:90 brd ff:ff:ff:ff:ff:ff
15:48:40 altname enp0s3
15:48:40 inet 10.30.171.157/23 metric 100 brd 10.30.171.255 scope global dynamic ens3
15:48:40 valid_lft 86239sec preferred_lft 86239sec
15:48:40 inet6 fe80::f816:3eff:fe43:4690/64 scope link
15:48:40 valid_lft forever preferred_lft forever
15:48:40 3: docker0: mtu 1458 qdisc noqueue state DOWN group default
15:48:40 link/ether 02:42:97:74:87:da brd ff:ff:ff:ff:ff:ff
15:48:40 inet 10.250.0.254/24 brd 10.250.0.255 scope global docker0
15:48:40 valid_lft forever preferred_lft forever
15:48:40
15:48:40
15:48:40 ---> sar -b -r -n DEV:
15:48:40 Linux 5.15.0-131-generic (prd-ubuntu2204-docker-4c-4g-25284) 09/01/25 _x86_64_ (4 CPU)
15:48:40
15:48:40 15:46:02 LINUX RESTART (4 CPU)
15:48:40
15:48:40
15:48:40 ---> sar -P ALL:
15:48:40 Linux 5.15.0-131-generic (prd-ubuntu2204-docker-4c-4g-25284) 09/01/25 _x86_64_ (4 CPU)
15:48:40
15:48:40 15:46:02 LINUX RESTART (4 CPU)
15:48:40
15:48:40
15:48:40