23:09:29  Started by upstream project "integration-distribution-weekly-test-trigger-scandium" build number 70
23:09:29  originally caused by:
23:09:29   Started by timer
23:09:29  Running as SYSTEM
23:09:29  [EnvInject] - Loading node environment variables.
23:09:29  Building remotely on prd-centos8-robot-2c-8g-1901 (centos8-robot-2c-8g) in workspace /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium
23:09:31  [ssh-agent] Looking for ssh-agent implementation...
23:09:31  [ssh-agent]   Exec ssh-agent (binary ssh-agent on a remote machine)
23:09:31  $ ssh-agent
23:09:31  SSH_AUTH_SOCK=/tmp/ssh-JKnwUlZ3op4L/agent.5281
23:09:31  SSH_AGENT_PID=5282
23:09:31  [ssh-agent] Started.
23:09:31  Running ssh-add (command line suppressed)
23:09:31  Identity added: /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium@tmp/private_key_13236092914223982680.key (/w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium@tmp/private_key_13236092914223982680.key)
23:09:31  [ssh-agent] Using credentials jenkins (Release Engineering Jenkins Key)
23:09:31  The recommended git tool is: NONE
23:09:35  using credential opendaylight-jenkins-ssh
23:09:35  Wiping out workspace first.
23:09:35  Cloning the remote Git repository
23:09:35  Cloning repository git://devvexx.opendaylight.org/mirror/integration/test
23:09:35   > git init /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/test # timeout=10
23:09:35  Fetching upstream changes from git://devvexx.opendaylight.org/mirror/integration/test
23:09:35   > git --version # timeout=10
23:09:35   > git --version # 'git version 2.43.0'
23:09:35  using GIT_SSH to set credentials Release Engineering Jenkins Key
23:09:35  [INFO] Currently running in a labeled security context
23:09:35  [INFO] Currently SELinux is 'enforcing' on the host
23:09:35   > /usr/bin/chcon --type=ssh_home_t /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/test@tmp/jenkins-gitclient-ssh6618950430940489854.key
23:09:35  Verifying host key using known hosts file
23:09:35  You're using 'Known hosts file' strategy to verify ssh host keys, but your known_hosts file does not exist, please go to 'Manage Jenkins' -> 'Security' -> 'Git Host Key Verification Configuration' and configure host key verification.
23:09:35   > git fetch --tags --force --progress -- git://devvexx.opendaylight.org/mirror/integration/test +refs/heads/*:refs/remotes/origin/* # timeout=10
23:09:38   > git config remote.origin.url git://devvexx.opendaylight.org/mirror/integration/test # timeout=10
23:09:38   > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
23:09:38   > git config remote.origin.url git://devvexx.opendaylight.org/mirror/integration/test # timeout=10
23:09:38  Fetching upstream changes from git://devvexx.opendaylight.org/mirror/integration/test
23:09:38  using GIT_SSH to set credentials Release Engineering Jenkins Key
23:09:38  [INFO] Currently running in a labeled security context
23:09:38  [INFO] Currently SELinux is 'enforcing' on the host
23:09:38   > /usr/bin/chcon --type=ssh_home_t /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/test@tmp/jenkins-gitclient-ssh4258049995134636500.key
23:09:38  Verifying host key using known hosts file
23:09:38  You're using 'Known hosts file' strategy to verify ssh host keys, but your known_hosts file does not exist, please go to 'Manage Jenkins' -> 'Security' -> 'Git Host Key Verification Configuration' and configure host key verification.
23:09:38   > git fetch --tags --force --progress -- git://devvexx.opendaylight.org/mirror/integration/test master # timeout=10
23:09:38   > git rev-parse FETCH_HEAD^{commit} # timeout=10
23:09:38  Checking out Revision 9e7a2f1bec76f24ac7173c3a00f09ed1af208887 (origin/master)
23:09:38   > git config core.sparsecheckout # timeout=10
23:09:38   > git checkout -f 9e7a2f1bec76f24ac7173c3a00f09ed1af208887 # timeout=10
23:09:39  Commit message: "Add pekko templates"
23:09:39   > git rev-parse FETCH_HEAD^{commit} # timeout=10
23:09:39   > git rev-list --no-walk e12906d887353b3b6c7ca6e293959c75cf9a8409 # timeout=10
23:09:39  No emails were triggered.
23:09:39  provisioning config files...
23:09:39  copy managed file [npmrc] to file:/home/jenkins/.npmrc
23:09:39  copy managed file [pipconf] to file:/home/jenkins/.config/pip/pip.conf
23:09:41  copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
23:09:41  [controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium] $ /bin/bash /tmp/jenkins927717563744259285.sh
23:09:42  ---> python-tools-install.sh
23:09:42  Setup pyenv:
23:09:42    system
23:09:42  * 3.8.13 (set by /opt/pyenv/version)
23:09:42  * 3.9.13 (set by /opt/pyenv/version)
23:09:42  * 3.10.13 (set by /opt/pyenv/version)
23:09:42  * 3.11.7 (set by /opt/pyenv/version)
23:09:46  lf-activate-venv(): INFO: Creating python3 venv at /tmp/venv-8Z2X
23:09:46  lf-activate-venv(): INFO: Save venv in file: /tmp/.os_lf_venv
23:09:50  lf-activate-venv(): INFO: Installing: lftools 
23:10:14  lf-activate-venv(): INFO: Adding /tmp/venv-8Z2X/bin to PATH
23:10:14  Generating Requirements File
23:10:35  Python 3.11.7
23:10:36  pip 25.2 from /tmp/venv-8Z2X/lib/python3.11/site-packages/pip (python 3.11)
23:10:36  appdirs==1.4.4
23:10:36  argcomplete==3.6.2
23:10:36  aspy.yaml==1.3.0
23:10:36  attrs==25.3.0
23:10:36  autopage==0.5.2
23:10:36  beautifulsoup4==4.13.4
23:10:36  boto3==1.40.11
23:10:36  botocore==1.40.11
23:10:36  bs4==0.0.2
23:10:36  cachetools==5.5.2
23:10:36  certifi==2025.8.3
23:10:36  cffi==1.17.1
23:10:36  cfgv==3.4.0
23:10:36  chardet==5.2.0
23:10:36  charset-normalizer==3.4.3
23:10:36  click==8.2.1
23:10:36  cliff==4.10.0
23:10:36  cmd2==2.7.0
23:10:36  cryptography==3.3.2
23:10:36  debtcollector==3.0.0
23:10:36  decorator==5.2.1
23:10:36  defusedxml==0.7.1
23:10:36  Deprecated==1.2.18
23:10:36  distlib==0.4.0
23:10:36  dnspython==2.7.0
23:10:36  docker==7.1.0
23:10:36  dogpile.cache==1.4.0
23:10:36  durationpy==0.10
23:10:36  email_validator==2.2.0
23:10:36  filelock==3.19.1
23:10:36  future==1.0.0
23:10:36  gitdb==4.0.12
23:10:36  GitPython==3.1.45
23:10:36  google-auth==2.40.3
23:10:36  httplib2==0.22.0
23:10:36  identify==2.6.13
23:10:36  idna==3.10
23:10:36  importlib-resources==1.5.0
23:10:36  iso8601==2.1.0
23:10:36  Jinja2==3.1.6
23:10:36  jmespath==1.0.1
23:10:36  jsonpatch==1.33
23:10:36  jsonpointer==3.0.0
23:10:36  jsonschema==4.25.0
23:10:36  jsonschema-specifications==2025.4.1
23:10:36  keystoneauth1==5.11.1
23:10:36  kubernetes==33.1.0
23:10:36  lftools==0.37.13
23:10:36  lxml==6.0.0
23:10:36  markdown-it-py==4.0.0
23:10:36  MarkupSafe==3.0.2
23:10:36  mdurl==0.1.2
23:10:36  msgpack==1.1.1
23:10:36  multi_key_dict==2.0.3
23:10:36  munch==4.0.0
23:10:36  netaddr==1.3.0
23:10:36  niet==1.4.2
23:10:36  nodeenv==1.9.1
23:10:36  oauth2client==4.1.3
23:10:36  oauthlib==3.3.1
23:10:36  openstacksdk==4.7.0
23:10:36  os-client-config==2.3.0
23:10:36  os-service-types==1.8.0
23:10:36  osc-lib==4.1.0
23:10:36  oslo.config==10.0.0
23:10:36  oslo.context==6.0.0
23:10:36  oslo.i18n==6.5.1
23:10:36  oslo.log==7.2.0
23:10:36  oslo.serialization==5.7.0
23:10:36  oslo.utils==9.0.0
23:10:36  packaging==25.0
23:10:36  pbr==7.0.0
23:10:36  platformdirs==4.3.8
23:10:36  prettytable==3.16.0
23:10:36  psutil==7.0.0
23:10:36  pyasn1==0.6.1
23:10:36  pyasn1_modules==0.4.2
23:10:36  pycparser==2.22
23:10:36  pygerrit2==2.0.15
23:10:36  PyGithub==2.7.0
23:10:36  Pygments==2.19.2
23:10:36  PyJWT==2.10.1
23:10:36  PyNaCl==1.5.0
23:10:36  pyparsing==2.4.7
23:10:36  pyperclip==1.9.0
23:10:36  pyrsistent==0.20.0
23:10:36  python-cinderclient==9.7.0
23:10:36  python-dateutil==2.9.0.post0
23:10:36  python-heatclient==4.3.0
23:10:36  python-jenkins==1.8.3
23:10:36  python-keystoneclient==5.6.0
23:10:36  python-magnumclient==4.8.1
23:10:36  python-openstackclient==8.1.0
23:10:36  python-swiftclient==4.8.0
23:10:36  PyYAML==6.0.2
23:10:36  referencing==0.36.2
23:10:36  requests==2.32.4
23:10:36  requests-oauthlib==2.0.0
23:10:36  requestsexceptions==1.4.0
23:10:36  rfc3986==2.0.0
23:10:36  rich==14.1.0
23:10:36  rich-argparse==1.7.1
23:10:36  rpds-py==0.27.0
23:10:36  rsa==4.9.1
23:10:36  ruamel.yaml==0.18.14
23:10:36  ruamel.yaml.clib==0.2.12
23:10:36  s3transfer==0.13.1
23:10:36  simplejson==3.20.1
23:10:36  six==1.17.0
23:10:36  smmap==5.0.2
23:10:36  soupsieve==2.7
23:10:36  stevedore==5.4.1
23:10:36  tabulate==0.9.0
23:10:36  toml==0.10.2
23:10:36  tomlkit==0.13.3
23:10:36  tqdm==4.67.1
23:10:36  typing_extensions==4.14.1
23:10:36  tzdata==2025.2
23:10:36  urllib3==1.26.20
23:10:36  virtualenv==20.34.0
23:10:36  wcwidth==0.2.13
23:10:36  websocket-client==1.8.0
23:10:36  wrapt==1.17.3
23:10:36  xdg==6.0.0
23:10:36  xmltodict==0.14.2
23:10:36  yq==3.4.3
23:10:36  [EnvInject] - Injecting environment variables from a build step.
23:10:36  [EnvInject] - Injecting as environment variables the properties content 
23:10:36  OS_STACK_TEMPLATE=csit-2-instance-type.yaml
23:10:36  OS_CLOUD=vex
23:10:36  OS_STACK_NAME=releng-controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium-69
23:10:36  OS_STACK_TEMPLATE_DIR=openstack-hot
23:10:36  
23:10:36  [EnvInject] - Variables injected successfully.
23:10:36  provisioning config files...
23:10:36  copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
23:10:36  [controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium] $ /bin/bash /tmp/jenkins12700502780249334849.sh
23:10:36  ---> Create parameters file for OpenStack HOT
23:10:36  OpenStack Heat parameters generated
23:10:36  -----------------------------------
23:10:36  parameters:
23:10:36      vm_0_count: '3'
23:10:36      vm_0_flavor: 'v3-standard-4'
23:10:36      vm_0_image: 'ZZCI - Ubuntu 22.04 - builder - x86_64 - 20250201-010426.857'
23:10:36      vm_1_count: '1'
23:10:36      vm_1_flavor: 'v3-standard-2'
23:10:36      vm_1_image: 'ZZCI - Ubuntu 22.04 - mininet-ovs-217 - x86_64 - 20250201-060151.911'
23:10:36      
23:10:36      job_name: '05380-69'
23:10:36      silo: 'releng'
23:10:36  [controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium] $ /bin/bash -l /tmp/jenkins10325102496458650368.sh
23:10:36  ---> Create HEAT stack
23:10:36  + source /home/jenkins/lf-env.sh
23:10:36  + lf-activate-venv --python python3 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
23:10:36  ++ mktemp -d /tmp/venv-XXXX
23:10:36  + lf_venv=/tmp/venv-R224
23:10:36  + local venv_file=/tmp/.os_lf_venv
23:10:36  + local python=python3
23:10:36  + local options
23:10:36  + local set_path=true
23:10:36  + local install_args=
23:10:36  ++ getopt -o np:v: -l no-path,system-site-packages,python:,venv-file: -n lf-activate-venv -- --python python3 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
23:10:36  + options=' --python '\''python3'\'' -- '\''lftools[openstack]'\'' '\''kubernetes'\'' '\''niet'\'' '\''python-heatclient'\'' '\''python-openstackclient'\'' '\''python-magnumclient'\'' '\''yq'\'''
23:10:36  + eval set -- ' --python '\''python3'\'' -- '\''lftools[openstack]'\'' '\''kubernetes'\'' '\''niet'\'' '\''python-heatclient'\'' '\''python-openstackclient'\'' '\''python-magnumclient'\'' '\''yq'\'''
23:10:36  ++ set -- --python python3 -- 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
23:10:36  + true
23:10:36  + case $1 in
23:10:36  + python=python3
23:10:36  + shift 2
23:10:36  + true
23:10:36  + case $1 in
23:10:36  + shift
23:10:36  + break
23:10:36  + case $python in
23:10:36  + local pkg_list=
23:10:36  + [[ -d /opt/pyenv ]]
23:10:36  + echo 'Setup pyenv:'
23:10:36  Setup pyenv:
23:10:36  + export PYENV_ROOT=/opt/pyenv
23:10:36  + PYENV_ROOT=/opt/pyenv
23:10:36  + export PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
23:10:36  + PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
23:10:36  + pyenv versions
23:10:36    system
23:10:36    3.8.13
23:10:36    3.9.13
23:10:36    3.10.13
23:10:36  * 3.11.7 (set by /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/.python-version)
23:10:36  + command -v pyenv
23:10:36  ++ pyenv init - --no-rehash
23:10:36  + eval 'PATH="$(bash --norc -ec '\''IFS=:; paths=($PATH); 
23:10:36  for i in ${!paths[@]}; do 
23:10:36  if [[ ${paths[i]} == "'\'''\''/opt/pyenv/shims'\'''\''" ]]; then unset '\''\'\'''\''paths[i]'\''\'\'''\''; 
23:10:36  fi; done; 
23:10:36  echo "${paths[*]}"'\'')"
23:10:36  export PATH="/opt/pyenv/shims:${PATH}"
23:10:36  export PYENV_SHELL=bash
23:10:36  source '\''/opt/pyenv/libexec/../completions/pyenv.bash'\''
23:10:36  pyenv() {
23:10:36    local command
23:10:36    command="${1:-}"
23:10:36    if [ "$#" -gt 0 ]; then
23:10:36      shift
23:10:36    fi
23:10:36  
23:10:36    case "$command" in
23:10:36    rehash|shell)
23:10:36      eval "$(pyenv "sh-$command" "$@")"
23:10:36      ;;
23:10:36    *)
23:10:36      command pyenv "$command" "$@"
23:10:36      ;;
23:10:36    esac
23:10:36  }'
23:10:36  +++ bash --norc -ec 'IFS=:; paths=($PATH); 
23:10:36  for i in ${!paths[@]}; do 
23:10:36  if [[ ${paths[i]} == "/opt/pyenv/shims" ]]; then unset '\''paths[i]'\''; 
23:10:36  fi; done; 
23:10:36  echo "${paths[*]}"'
23:10:36  ++ PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
23:10:36  ++ export PATH=/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
23:10:36  ++ PATH=/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
23:10:36  ++ export PYENV_SHELL=bash
23:10:36  ++ PYENV_SHELL=bash
23:10:36  ++ source /opt/pyenv/libexec/../completions/pyenv.bash
23:10:36  +++ complete -F _pyenv pyenv
23:10:36  ++ lf-pyver python3
23:10:36  ++ local py_version_xy=python3
23:10:36  ++ local py_version_xyz=
23:10:36  ++ awk '{ print $1 }'
23:10:36  ++ pyenv versions
23:10:36  ++ local command
23:10:36  ++ command=versions
23:10:36  ++ '[' 1 -gt 0 ']'
23:10:36  ++ shift
23:10:36  ++ case "$command" in
23:10:36  ++ command pyenv versions
23:10:36  ++ pyenv versions
23:10:36  ++ sed 's/^[ *]* //'
23:10:36  ++ grep -E '^[0-9.]*[0-9]$'
23:10:37  ++ [[ ! -s /tmp/.pyenv_versions ]]
23:10:37  +++ grep '^3' /tmp/.pyenv_versions
23:10:37  +++ tail -n 1
23:10:37  +++ sort -V
23:10:37  ++ py_version_xyz=3.11.7
23:10:37  ++ [[ -z 3.11.7 ]]
23:10:37  ++ echo 3.11.7
23:10:37  ++ return 0
23:10:37  + pyenv local 3.11.7
23:10:37  + local command
23:10:37  + command=local
23:10:37  + '[' 2 -gt 0 ']'
23:10:37  + shift
23:10:37  + case "$command" in
23:10:37  + command pyenv local 3.11.7
23:10:37  + pyenv local 3.11.7
23:10:37  + for arg in "$@"
23:10:37  + case $arg in
23:10:37  + pkg_list+='lftools[openstack] '
23:10:37  + for arg in "$@"
23:10:37  + case $arg in
23:10:37  + pkg_list+='kubernetes '
23:10:37  + for arg in "$@"
23:10:37  + case $arg in
23:10:37  + pkg_list+='niet '
23:10:37  + for arg in "$@"
23:10:37  + case $arg in
23:10:37  + pkg_list+='python-heatclient '
23:10:37  + for arg in "$@"
23:10:37  + case $arg in
23:10:37  + pkg_list+='python-openstackclient '
23:10:37  + for arg in "$@"
23:10:37  + case $arg in
23:10:37  + pkg_list+='python-magnumclient '
23:10:37  + for arg in "$@"
23:10:37  + case $arg in
23:10:37  + pkg_list+='yq '
23:10:37  + [[ -f /tmp/.os_lf_venv ]]
23:10:37  ++ cat /tmp/.os_lf_venv
23:10:37  + lf_venv=/tmp/venv-8Z2X
23:10:37  + echo 'lf-activate-venv(): INFO: Reuse venv:/tmp/venv-8Z2X from' file:/tmp/.os_lf_venv
23:10:37  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-8Z2X from file:/tmp/.os_lf_venv
23:10:37  + /tmp/venv-8Z2X/bin/python3 -m pip install --upgrade --quiet pip 'setuptools<66' virtualenv
23:10:39  + [[ -z lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq  ]]
23:10:39  + echo 'lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq '
23:10:39  lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq 
23:10:39  + /tmp/venv-8Z2X/bin/python3 -m pip install --upgrade --quiet --upgrade-strategy eager 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
23:10:55  + type python3
23:10:55  + true
23:10:55  + echo 'lf-activate-venv(): INFO: Adding /tmp/venv-8Z2X/bin to PATH'
23:10:55  lf-activate-venv(): INFO: Adding /tmp/venv-8Z2X/bin to PATH
23:10:55  + PATH=/tmp/venv-8Z2X/bin:/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
23:10:55  + return 0
23:10:55  + openstack --os-cloud vex limits show --absolute
23:10:57  +--------------------------+---------+
23:10:57  | Name                     |   Value |
23:10:57  +--------------------------+---------+
23:10:57  | maxTotalInstances        |      -1 |
23:10:57  | maxTotalCores            |     450 |
23:10:57  | maxTotalRAMSize          | 1000000 |
23:10:57  | maxServerMeta            |     128 |
23:10:57  | maxImageMeta             |     128 |
23:10:57  | maxPersonality           |       5 |
23:10:57  | maxPersonalitySize       |   10240 |
23:10:57  | maxTotalKeypairs         |     100 |
23:10:57  | maxServerGroups          |      10 |
23:10:57  | maxServerGroupMembers    |      10 |
23:10:57  | maxTotalFloatingIps      |      -1 |
23:10:57  | maxSecurityGroups        |      -1 |
23:10:57  | maxSecurityGroupRules    |      -1 |
23:10:57  | totalRAMUsed             |  483328 |
23:10:57  | totalCoresUsed           |     118 |
23:10:57  | totalInstancesUsed       |      43 |
23:10:57  | totalFloatingIpsUsed     |       0 |
23:10:57  | totalSecurityGroupsUsed  |       0 |
23:10:57  | totalServerGroupsUsed    |       0 |
23:10:57  | maxTotalVolumes          |      -1 |
23:10:57  | maxTotalSnapshots        |      10 |
23:10:57  | maxTotalVolumeGigabytes  |    4096 |
23:10:57  | maxTotalBackups          |      10 |
23:10:57  | maxTotalBackupGigabytes  |    1000 |
23:10:57  | totalVolumesUsed         |       3 |
23:10:57  | totalGigabytesUsed       |      60 |
23:10:57  | totalSnapshotsUsed       |       0 |
23:10:57  | totalBackupsUsed         |       0 |
23:10:57  | totalBackupGigabytesUsed |       0 |
23:10:57  +--------------------------+---------+
23:10:57  + pushd /opt/ciman/openstack-hot
23:10:57  /opt/ciman/openstack-hot /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium
23:10:57  + lftools openstack --os-cloud vex stack create releng-controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium-69 csit-2-instance-type.yaml /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/stack-parameters.yaml
23:11:34  Creating stack releng-controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium-69
23:11:34  Waiting to initialize infrastructure...
23:11:34  Waiting to initialize infrastructure...
23:11:34  Stack initialization successful.
23:11:34  ------------------------------------
23:11:34  Stack Details
23:11:34  ------------------------------------
23:11:34  {'added': None,
23:11:34   'capabilities': [],
23:11:34   'created_at': '2025-08-16T23:11:00Z',
23:11:34   'deleted': None,
23:11:34   'deleted_at': None,
23:11:34   'description': 'No description',
23:11:34   'environment': None,
23:11:34   'environment_files': None,
23:11:34   'files': None,
23:11:34   'files_container': None,
23:11:34   'id': 'f4b38fdd-7204-40f8-83ad-5dee1fe5d0af',
23:11:34   'is_rollback_disabled': True,
23:11:34   'links': [{'href': 'https://orchestration.public.mtl1.vexxhost.net/v1/12c36e260d8e4bb2913965203b1b491f/stacks/releng-controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium-69/f4b38fdd-7204-40f8-83ad-5dee1fe5d0af',
23:11:34              'rel': 'self'}],
23:11:34   'location': Munch({'cloud': 'vex', 'region_name': 'ca-ymq-1', 'zone': None, 'project': Munch({'id': '12c36e260d8e4bb2913965203b1b491f', 'name': '61975f2c-7c17-4d69-82fa-c3ae420ad6fd', 'domain_id': None, 'domain_name': 'Default'})}),
23:11:34   'name': 'releng-controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium-69',
23:11:34   'notification_topics': [],
23:11:34   'outputs': [{'description': 'IP addresses of the 1st vm types',
23:11:34                'output_key': 'vm_0_ips',
23:11:34                'output_value': ['10.30.170.232',
23:11:34                                 '10.30.170.47',
23:11:34                                 '10.30.170.110']},
23:11:34               {'description': 'IP addresses of the 2nd vm types',
23:11:34                'output_key': 'vm_1_ips',
23:11:34                'output_value': ['10.30.170.56']}],
23:11:34   'owner_id': ****,
23:11:34   'parameters': {'OS::project_id': '12c36e260d8e4bb2913965203b1b491f',
23:11:34                  'OS::stack_id': 'f4b38fdd-7204-40f8-83ad-5dee1fe5d0af',
23:11:34                  'OS::stack_name': 'releng-controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium-69',
23:11:34                  'job_name': '05380-69',
23:11:34                  'silo': 'releng',
23:11:34                  'vm_0_count': '3',
23:11:34                  'vm_0_flavor': 'v3-standard-4',
23:11:34                  'vm_0_image': 'ZZCI - Ubuntu 22.04 - builder - x86_64 - '
23:11:34                                '20250201-010426.857',
23:11:34                  'vm_1_count': '1',
23:11:34                  'vm_1_flavor': 'v3-standard-2',
23:11:34                  'vm_1_image': 'ZZCI - Ubuntu 22.04 - mininet-ovs-217 - x86_64 '
23:11:34                                '- 20250201-060151.911'},
23:11:34   'parent_id': None,
23:11:34   'replaced': None,
23:11:34   'status': 'CREATE_COMPLETE',
23:11:34   'status_reason': 'Stack CREATE completed successfully',
23:11:34   'tags': [],
23:11:34   'template': None,
23:11:34   'template_description': 'No description',
23:11:34   'template_url': None,
23:11:34   'timeout_mins': 15,
23:11:34   'unchanged': None,
23:11:34   'updated': None,
23:11:34   'updated_at': None,
23:11:34   'user_project_id': '9fc799139b3e456ebebf6c8795c470f4'}
23:11:34  ------------------------------------
23:11:34  + popd
23:11:34  /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium
23:11:34  [controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium] $ /bin/bash -l /tmp/jenkins8454538152231473683.sh
23:11:34  ---> Copy SSH public keys to CSIT lab
23:11:34  Setup pyenv:
23:11:34    system
23:11:34    3.8.13
23:11:34    3.9.13
23:11:34    3.10.13
23:11:34  * 3.11.7 (set by /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/.python-version)
23:11:35  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-8Z2X from file:/tmp/.os_lf_venv
23:11:37  lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes python-heatclient python-openstackclient 
23:11:51  lf-activate-venv(): INFO: Adding /tmp/venv-8Z2X/bin to PATH
23:11:53  SSH not responding on 10.30.170.47. Retrying in 10 seconds...
23:11:53  SSH not responding on 10.30.170.110. Retrying in 10 seconds...
23:11:53  SSH not responding on 10.30.170.232. Retrying in 10 seconds...
23:11:53  Warning: Permanently added '10.30.170.56' (ECDSA) to the list of known hosts.
23:11:54   releng-05380-69-1-mininet-ovs-217-0
23:11:54  Successfully copied public keys to slave 10.30.170.56
23:12:03  Ping to 10.30.170.232 successful.
23:12:03  Ping to 10.30.170.47 successful.
23:12:03  Ping to 10.30.170.110 successful.
23:12:03  SSH not responding on 10.30.170.232. Retrying in 10 seconds...
23:12:03  SSH not responding on 10.30.170.47. Retrying in 10 seconds...
23:12:03  SSH not responding on 10.30.170.110. Retrying in 10 seconds...
23:12:13  Ping to 10.30.170.232 successful.
23:12:13  Ping to 10.30.170.47 successful.
23:12:13  Ping to 10.30.170.110 successful.
23:12:13  SSH not responding on 10.30.170.47. Retrying in 10 seconds...
23:12:13  SSH not responding on 10.30.170.232. Retrying in 10 seconds...
23:12:18  Warning: Permanently added '10.30.170.110' (ECDSA) to the list of known hosts.
23:12:19   releng-05380-69-0-builder-2
23:12:19  Successfully copied public keys to slave 10.30.170.110
23:12:23  Ping to 10.30.170.47 successful.
23:12:23  Ping to 10.30.170.232 successful.
23:12:24  SSH not responding on 10.30.170.47. Retrying in 10 seconds...
23:12:25  Warning: Permanently added '10.30.170.232' (ECDSA) to the list of known hosts.
23:12:26   releng-05380-69-0-builder-0
23:12:26  Successfully copied public keys to slave 10.30.170.232
23:12:26  Process 6505 ready.
23:12:34  Ping to 10.30.170.47 successful.
23:12:35  Warning: Permanently added '10.30.170.47' (ECDSA) to the list of known hosts.
23:12:35   releng-05380-69-0-builder-1
23:12:35  Successfully copied public keys to slave 10.30.170.47
23:12:35  Process 6506 ready.
23:12:35  Process 6507 ready.
23:12:35  Process 6509 ready.
23:12:35  SSH ready on all stack servers.
23:12:35  [controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium] $ /bin/bash -l /tmp/jenkins13789049074007869939.sh
23:12:36  Setup pyenv:
23:12:36    system
23:12:36    3.8.13
23:12:36    3.9.13
23:12:36    3.10.13
23:12:36  * 3.11.7 (set by /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/.python-version)
23:12:40  lf-activate-venv(): INFO: Creating python3 venv at /tmp/venv-siTl
23:12:40  lf-activate-venv(): INFO: Save venv in file: /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/.robot_venv
23:12:44  lf-activate-venv(): INFO: Installing: setuptools wheel 
23:12:45  lf-activate-venv(): INFO: Adding /tmp/venv-siTl/bin to PATH
23:12:45  + echo 'Installing Python Requirements'
23:12:45  Installing Python Requirements
23:12:45  + cat
23:12:45  + python -m pip install -r requirements.txt
23:12:46  Looking in indexes: https://nexus3.opendaylight.org/repository/PyPi/simple
23:12:46  Collecting docker-py (from -r requirements.txt (line 1))
23:12:46    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/docker-py/1.10.6/docker_py-1.10.6-py2.py3-none-any.whl (50 kB)
23:12:46  Collecting ipaddr (from -r requirements.txt (line 2))
23:12:46    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/ipaddr/2.2.0/ipaddr-2.2.0.tar.gz (26 kB)
23:12:46    Preparing metadata (setup.py): started
23:12:46    Preparing metadata (setup.py): finished with status 'done'
23:12:46  Collecting netaddr (from -r requirements.txt (line 3))
23:12:46    Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/netaddr/1.3.0/netaddr-1.3.0-py3-none-any.whl (2.3 MB)
23:12:46  Collecting netifaces (from -r requirements.txt (line 4))
23:12:46    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/netifaces/0.11.0/netifaces-0.11.0.tar.gz (30 kB)
23:12:46    Preparing metadata (setup.py): started
23:12:46    Preparing metadata (setup.py): finished with status 'done'
23:12:46  Collecting pyhocon (from -r requirements.txt (line 5))
23:12:46    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/pyhocon/0.3.61/pyhocon-0.3.61-py3-none-any.whl (25 kB)
23:12:46  Collecting requests (from -r requirements.txt (line 6))
23:12:46    Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/requests/2.32.4/requests-2.32.4-py3-none-any.whl (64 kB)
23:12:47  Collecting robotframework (from -r requirements.txt (line 7))
23:12:47    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework/7.3.2/robotframework-7.3.2-py3-none-any.whl (795 kB)
23:12:47       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 795.1/795.1 kB 31.6 MB/s  0:00:00
23:12:47  Collecting robotframework-httplibrary (from -r requirements.txt (line 8))
23:12:47    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-httplibrary/0.4.2/robotframework-httplibrary-0.4.2.tar.gz (9.1 kB)
23:12:47    Preparing metadata (setup.py): started
23:12:47    Preparing metadata (setup.py): finished with status 'done'
23:12:47  Collecting robotframework-requests==0.9.7 (from -r requirements.txt (line 9))
23:12:47    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-requests/0.9.7/robotframework_requests-0.9.7-py3-none-any.whl (21 kB)
23:12:47  Collecting robotframework-selenium2library (from -r requirements.txt (line 10))
23:12:47    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-selenium2library/3.0.0/robotframework_selenium2library-3.0.0-py2.py3-none-any.whl (6.2 kB)
23:12:47  Collecting robotframework-sshlibrary==3.8.0 (from -r requirements.txt (line 11))
23:12:47    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-sshlibrary/3.8.0/robotframework-sshlibrary-3.8.0.tar.gz (51 kB)
23:12:47    Preparing metadata (setup.py): started
23:12:47    Preparing metadata (setup.py): finished with status 'done'
23:12:47  Collecting scapy (from -r requirements.txt (line 12))
23:12:47    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/scapy/2.6.1/scapy-2.6.1-py3-none-any.whl (2.4 MB)
23:12:47       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.4/2.4 MB 38.9 MB/s  0:00:00
23:12:47  Collecting jsonpath-rw (from -r requirements.txt (line 15))
23:12:47    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/jsonpath-rw/1.4.0/jsonpath-rw-1.4.0.tar.gz (13 kB)
23:12:47    Preparing metadata (setup.py): started
23:12:47    Preparing metadata (setup.py): finished with status 'done'
23:12:47  Collecting elasticsearch (from -r requirements.txt (line 18))
23:12:47    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elasticsearch/9.1.0/elasticsearch-9.1.0-py3-none-any.whl (929 kB)
23:12:47       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 929.5/929.5 kB 35.3 MB/s  0:00:00
23:12:48  Collecting elasticsearch-dsl (from -r requirements.txt (line 19))
23:12:48    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elasticsearch-dsl/8.18.0/elasticsearch_dsl-8.18.0-py3-none-any.whl (10 kB)
23:12:48  Collecting pyangbind (from -r requirements.txt (line 22))
23:12:48    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/pyangbind/0.8.6/pyangbind-0.8.6-py3-none-any.whl (52 kB)
23:12:48  Collecting isodate (from -r requirements.txt (line 25))
23:12:48    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/isodate/0.7.2/isodate-0.7.2-py3-none-any.whl (22 kB)
23:12:48  Collecting jmespath (from -r requirements.txt (line 28))
23:12:48    Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/jmespath/1.0.1/jmespath-1.0.1-py3-none-any.whl (20 kB)
23:12:48  Collecting jsonpatch (from -r requirements.txt (line 31))
23:12:48    Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/jsonpatch/1.33/jsonpatch-1.33-py2.py3-none-any.whl (12 kB)
23:12:48  Collecting paramiko>=1.15.3 (from robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
23:12:48    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/paramiko/4.0.0/paramiko-4.0.0-py3-none-any.whl (223 kB)
23:12:48  Collecting scp>=0.13.0 (from robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
23:12:48    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/scp/0.15.0/scp-0.15.0-py2.py3-none-any.whl (8.8 kB)
23:12:48  Collecting docker-pycreds>=0.2.1 (from docker-py->-r requirements.txt (line 1))
23:12:48    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/docker-pycreds/0.4.0/docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)
23:12:48  Collecting six>=1.4.0 (from docker-py->-r requirements.txt (line 1))
23:12:48    Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/six/1.17.0/six-1.17.0-py2.py3-none-any.whl (11 kB)
23:12:48  Collecting websocket-client>=0.32.0 (from docker-py->-r requirements.txt (line 1))
23:12:48    Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/websocket-client/1.8.0/websocket_client-1.8.0-py3-none-any.whl (58 kB)
23:12:48  Collecting pyparsing<4,>=2 (from pyhocon->-r requirements.txt (line 5))
23:12:48    Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/pyparsing/3.2.3/pyparsing-3.2.3-py3-none-any.whl (111 kB)
23:12:48  Collecting charset_normalizer<4,>=2 (from requests->-r requirements.txt (line 6))
23:12:48    Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/charset-normalizer/3.4.3/charset_normalizer-3.4.3-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (150 kB)
23:12:48  Collecting idna<4,>=2.5 (from requests->-r requirements.txt (line 6))
23:12:48    Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/idna/3.10/idna-3.10-py3-none-any.whl (70 kB)
23:12:49  Collecting urllib3<3,>=1.21.1 (from requests->-r requirements.txt (line 6))
23:12:49    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/urllib3/2.5.0/urllib3-2.5.0-py3-none-any.whl (129 kB)
23:12:49  Collecting certifi>=2017.4.17 (from requests->-r requirements.txt (line 6))
23:12:49    Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/certifi/2025.8.3/certifi-2025.8.3-py3-none-any.whl (161 kB)
23:12:49  Collecting webtest>=2.0 (from robotframework-httplibrary->-r requirements.txt (line 8))
23:12:49    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/webtest/3.0.6/webtest-3.0.6-py3-none-any.whl (32 kB)
23:12:49  Collecting jsonpointer (from robotframework-httplibrary->-r requirements.txt (line 8))
23:12:49    Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/jsonpointer/3.0.0/jsonpointer-3.0.0-py2.py3-none-any.whl (7.6 kB)
23:12:49  Collecting robotframework-seleniumlibrary>=3.0.0 (from robotframework-selenium2library->-r requirements.txt (line 10))
23:12:49    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-seleniumlibrary/6.7.1/robotframework_seleniumlibrary-6.7.1-py2.py3-none-any.whl (104 kB)
23:12:49  Collecting ply (from jsonpath-rw->-r requirements.txt (line 15))
23:12:49    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/ply/3.11/ply-3.11-py2.py3-none-any.whl (49 kB)
23:12:49  Collecting decorator (from jsonpath-rw->-r requirements.txt (line 15))
23:12:50    Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/decorator/5.2.1/decorator-5.2.1-py3-none-any.whl (9.2 kB)
23:12:50  Collecting elastic-transport<10,>=9.1.0 (from elasticsearch->-r requirements.txt (line 18))
23:12:50    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elastic-transport/9.1.0/elastic_transport-9.1.0-py3-none-any.whl (65 kB)
23:12:50  Collecting python-dateutil (from elasticsearch->-r requirements.txt (line 18))
23:12:50    Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/python-dateutil/2.9.0.post0/python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
23:12:50  Collecting typing-extensions (from elasticsearch->-r requirements.txt (line 18))
23:12:50    Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/typing-extensions/4.14.1/typing_extensions-4.14.1-py3-none-any.whl (43 kB)
23:12:50  Collecting elasticsearch (from -r requirements.txt (line 18))
23:12:50    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elasticsearch/8.19.0/elasticsearch-8.19.0-py3-none-any.whl (926 kB)
23:12:50       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 926.9/926.9 kB 46.1 MB/s  0:00:00
23:12:50  INFO: pip is looking at multiple versions of elasticsearch-dsl to determine which version is compatible with other requirements. This could take a while.
23:12:50  Collecting elasticsearch-dsl (from -r requirements.txt (line 19))
23:12:50    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elasticsearch-dsl/8.17.1/elasticsearch_dsl-8.17.1-py3-none-any.whl (158 kB)
23:12:50    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elasticsearch-dsl/8.17.0/elasticsearch_dsl-8.17.0-py3-none-any.whl (158 kB)
23:12:50    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elasticsearch-dsl/8.16.0/elasticsearch_dsl-8.16.0-py3-none-any.whl (158 kB)
23:12:50    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elasticsearch-dsl/8.15.4/elasticsearch_dsl-8.15.4-py3-none-any.whl (104 kB)
23:12:50  Collecting elastic-transport<9,>=8.15.1 (from elasticsearch->-r requirements.txt (line 18))
23:12:50    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elastic-transport/8.17.1/elastic_transport-8.17.1-py3-none-any.whl (64 kB)
23:12:50  Collecting pyang (from pyangbind->-r requirements.txt (line 22))
23:12:50    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/pyang/2.6.1/pyang-2.6.1-py2.py3-none-any.whl (594 kB)
23:12:50       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 594.7/594.7 kB 27.1 MB/s  0:00:00
23:12:50  Collecting lxml (from pyangbind->-r requirements.txt (line 22))
23:12:50    Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/lxml/6.0.0/lxml-6.0.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (5.2 MB)
23:12:51  Collecting regex (from pyangbind->-r requirements.txt (line 22))
23:12:51    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/regex/2025.7.34/regex-2025.7.34-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (798 kB)
23:12:51       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 798.9/798.9 kB 29.0 MB/s  0:00:00
23:12:51  Collecting enum34 (from pyangbind->-r requirements.txt (line 22))
23:12:51    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/enum34/1.1.10/enum34-1.1.10-py3-none-any.whl (11 kB)
23:12:51  Collecting bcrypt>=3.2 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
23:12:51    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/bcrypt/4.3.0/bcrypt-4.3.0-cp39-abi3-manylinux_2_28_x86_64.whl (284 kB)
23:12:52  Collecting cryptography>=3.3 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
23:12:52    Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/cryptography/45.0.6/cryptography-45.0.6-cp311-abi3-manylinux_2_28_x86_64.whl (4.5 MB)
23:12:52  Collecting invoke>=2.0 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
23:12:52    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/invoke/2.2.0/invoke-2.2.0-py3-none-any.whl (160 kB)
23:12:52  Collecting pynacl>=1.5 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
23:12:52    Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/pynacl/1.5.0/PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (856 kB)
23:12:52  Collecting cffi>=1.14 (from cryptography>=3.3->paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
23:12:52    Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/cffi/1.17.1/cffi-1.17.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (467 kB)
23:12:52  Collecting pycparser (from cffi>=1.14->cryptography>=3.3->paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
23:12:52    Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/pycparser/2.22/pycparser-2.22-py3-none-any.whl (117 kB)
23:12:52  Collecting selenium>=4.3.0 (from robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
23:12:52    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/selenium/4.35.0/selenium-4.35.0-py3-none-any.whl (9.6 MB)
23:12:53       ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.6/9.6 MB 72.1 MB/s  0:00:00
23:12:53  Collecting robotframework-pythonlibcore>=4.4.1 (from robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
23:12:53    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-pythonlibcore/4.4.1/robotframework_pythonlibcore-4.4.1-py2.py3-none-any.whl (12 kB)
23:12:53  Collecting click>=8.0 (from robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
23:12:53    Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/click/8.2.1/click-8.2.1-py3-none-any.whl (102 kB)
23:12:53  Collecting trio~=0.30.0 (from selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
23:12:53    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/trio/0.30.0/trio-0.30.0-py3-none-any.whl (499 kB)
23:12:53  Collecting trio-websocket~=0.12.2 (from selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
23:12:53    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/trio-websocket/0.12.2/trio_websocket-0.12.2-py3-none-any.whl (21 kB)
23:12:53  Collecting attrs>=23.2.0 (from trio~=0.30.0->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
23:12:53    Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/attrs/25.3.0/attrs-25.3.0-py3-none-any.whl (63 kB)
23:12:53  Collecting sortedcontainers (from trio~=0.30.0->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
23:12:53    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/sortedcontainers/2.4.0/sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
23:12:53  Collecting outcome (from trio~=0.30.0->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
23:12:53    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/outcome/1.3.0.post0/outcome-1.3.0.post0-py2.py3-none-any.whl (10 kB)
23:12:53  Collecting sniffio>=1.3.0 (from trio~=0.30.0->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
23:12:53    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/sniffio/1.3.1/sniffio-1.3.1-py3-none-any.whl (10 kB)
23:12:53  Collecting wsproto>=0.14 (from trio-websocket~=0.12.2->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
23:12:53    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/wsproto/1.2.0/wsproto-1.2.0-py3-none-any.whl (24 kB)
23:12:53  Collecting pysocks!=1.5.7,<2.0,>=1.5.6 (from urllib3[socks]<3.0,>=2.5.0->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
23:12:53    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/pysocks/1.7.1/PySocks-1.7.1-py3-none-any.whl (16 kB)
23:12:53  Collecting WebOb>=1.2 (from webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8))
23:12:53    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/webob/1.8.9/WebOb-1.8.9-py2.py3-none-any.whl (115 kB)
23:12:53  Collecting waitress>=3.0.2 (from webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8))
23:12:53    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/waitress/3.0.2/waitress-3.0.2-py3-none-any.whl (56 kB)
23:12:53  Collecting beautifulsoup4 (from webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8))
23:12:53    Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/beautifulsoup4/4.13.4/beautifulsoup4-4.13.4-py3-none-any.whl (187 kB)
23:12:53  Collecting h11<1,>=0.9.0 (from wsproto>=0.14->trio-websocket~=0.12.2->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
23:12:53    Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/h11/0.16.0/h11-0.16.0-py3-none-any.whl (37 kB)
23:12:53  Collecting soupsieve>1.2 (from beautifulsoup4->webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8))
23:12:53    Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/soupsieve/2.7/soupsieve-2.7-py3-none-any.whl (36 kB)
23:12:53  Building wheels for collected packages: robotframework-sshlibrary, ipaddr, netifaces, robotframework-httplibrary, jsonpath-rw
23:12:53    DEPRECATION: Building 'robotframework-sshlibrary' using the legacy setup.py bdist_wheel mechanism, which will be removed in a future version. pip 25.3 will enforce this behaviour change. A possible replacement is to use the standardized build interface by setting the `--use-pep517` option, (possibly combined with `--no-build-isolation`), or adding a `pyproject.toml` file to the source tree of 'robotframework-sshlibrary'. Discussion can be found at https://github.com/pypa/pip/issues/6334
23:12:53    Building wheel for robotframework-sshlibrary (setup.py): started
23:12:54    Building wheel for robotframework-sshlibrary (setup.py): finished with status 'done'
23:12:54    Created wheel for robotframework-sshlibrary: filename=robotframework_sshlibrary-3.8.0-py3-none-any.whl size=55205 sha256=3e72a09d3020ab42a610761fd964359d0e8da1538efaa9606ebc623917427e5d
23:12:54    Stored in directory: /home/jenkins/.cache/pip/wheels/f7/c9/b3/a977b7bcc410d45ae27d240df3d00a12585509180e373ecccc
23:12:54    DEPRECATION: Building 'ipaddr' using the legacy setup.py bdist_wheel mechanism, which will be removed in a future version. pip 25.3 will enforce this behaviour change. A possible replacement is to use the standardized build interface by setting the `--use-pep517` option, (possibly combined with `--no-build-isolation`), or adding a `pyproject.toml` file to the source tree of 'ipaddr'. Discussion can be found at https://github.com/pypa/pip/issues/6334
23:12:54    Building wheel for ipaddr (setup.py): started
23:12:54    Building wheel for ipaddr (setup.py): finished with status 'done'
23:12:54    Created wheel for ipaddr: filename=ipaddr-2.2.0-py3-none-any.whl size=18353 sha256=69757e829770e8fd03de3b97650cce7468cc6c5f8965446096e1dbce50f527bc
23:12:54    Stored in directory: /home/jenkins/.cache/pip/wheels/dc/6c/04/da2d847fa8d45c59af3e1d83e2acc29cb8adcbaf04c0898dbf
23:12:54    DEPRECATION: Building 'netifaces' using the legacy setup.py bdist_wheel mechanism, which will be removed in a future version. pip 25.3 will enforce this behaviour change. A possible replacement is to use the standardized build interface by setting the `--use-pep517` option, (possibly combined with `--no-build-isolation`), or adding a `pyproject.toml` file to the source tree of 'netifaces'. Discussion can be found at https://github.com/pypa/pip/issues/6334
23:12:54    Building wheel for netifaces (setup.py): started
23:12:56    Building wheel for netifaces (setup.py): finished with status 'done'
23:12:56    Created wheel for netifaces: filename=netifaces-0.11.0-cp311-cp311-linux_x86_64.whl size=41073 sha256=38f216eff1692416fb781ab008225663c1981fc1b306bee30c484e3f52709ce4
23:12:56    Stored in directory: /home/jenkins/.cache/pip/wheels/f8/18/88/e61d54b995bea304bdb1d040a92b72228a1bf72ca2a3eba7c9
23:12:56    DEPRECATION: Building 'robotframework-httplibrary' using the legacy setup.py bdist_wheel mechanism, which will be removed in a future version. pip 25.3 will enforce this behaviour change. A possible replacement is to use the standardized build interface by setting the `--use-pep517` option, (possibly combined with `--no-build-isolation`), or adding a `pyproject.toml` file to the source tree of 'robotframework-httplibrary'. Discussion can be found at https://github.com/pypa/pip/issues/6334
23:12:56    Building wheel for robotframework-httplibrary (setup.py): started
23:12:56    Building wheel for robotframework-httplibrary (setup.py): finished with status 'done'
23:12:56    Created wheel for robotframework-httplibrary: filename=robotframework_httplibrary-0.4.2-py3-none-any.whl size=10014 sha256=231afbf5b281c6f72c7a73000d4339717c0c678926dd352fe842b1457be87557
23:12:56    Stored in directory: /home/jenkins/.cache/pip/wheels/aa/bc/0d/9a20dd51effef392aae2733cb4c7b66c6fa29fca33d88b57ed
23:12:56    DEPRECATION: Building 'jsonpath-rw' using the legacy setup.py bdist_wheel mechanism, which will be removed in a future version. pip 25.3 will enforce this behaviour change. A possible replacement is to use the standardized build interface by setting the `--use-pep517` option, (possibly combined with `--no-build-isolation`), or adding a `pyproject.toml` file to the source tree of 'jsonpath-rw'. Discussion can be found at https://github.com/pypa/pip/issues/6334
23:12:56    Building wheel for jsonpath-rw (setup.py): started
23:12:57    Building wheel for jsonpath-rw (setup.py): finished with status 'done'
23:12:57    Created wheel for jsonpath-rw: filename=jsonpath_rw-1.4.0-py3-none-any.whl size=15176 sha256=10b79db7724c04404a8cb145036a5a4b28c4bb7bdc10aec76ff01fbf64704423
23:12:57    Stored in directory: /home/jenkins/.cache/pip/wheels/f1/54/63/9a8da38cefae13755097b36cc852decc25d8ef69c37d58d4eb
23:12:57  Successfully built robotframework-sshlibrary ipaddr netifaces robotframework-httplibrary jsonpath-rw
23:12:57  Installing collected packages: sortedcontainers, ply, netifaces, ipaddr, enum34, websocket-client, WebOb, waitress, urllib3, typing-extensions, soupsieve, sniffio, six, scapy, robotframework-pythonlibcore, robotframework, regex, pysocks, pyparsing, pycparser, netaddr, lxml, jsonpointer, jmespath, isodate, invoke, idna, h11, decorator, click, charset_normalizer, certifi, bcrypt, attrs, wsproto, requests, python-dateutil, pyhocon, pyang, outcome, jsonpath-rw, jsonpatch, elastic-transport, docker-pycreds, cffi, beautifulsoup4, webtest, trio, robotframework-requests, pynacl, pyangbind, elasticsearch, docker-py, cryptography, trio-websocket, robotframework-httplibrary, paramiko, elasticsearch-dsl, selenium, scp, robotframework-sshlibrary, robotframework-seleniumlibrary, robotframework-selenium2library
23:13:04  
23:13:04  Successfully installed WebOb-1.8.9 attrs-25.3.0 bcrypt-4.3.0 beautifulsoup4-4.13.4 certifi-2025.8.3 cffi-1.17.1 charset_normalizer-3.4.3 click-8.2.1 cryptography-45.0.6 decorator-5.2.1 docker-py-1.10.6 docker-pycreds-0.4.0 elastic-transport-8.17.1 elasticsearch-8.19.0 elasticsearch-dsl-8.15.4 enum34-1.1.10 h11-0.16.0 idna-3.10 invoke-2.2.0 ipaddr-2.2.0 isodate-0.7.2 jmespath-1.0.1 jsonpatch-1.33 jsonpath-rw-1.4.0 jsonpointer-3.0.0 lxml-6.0.0 netaddr-1.3.0 netifaces-0.11.0 outcome-1.3.0.post0 paramiko-4.0.0 ply-3.11 pyang-2.6.1 pyangbind-0.8.6 pycparser-2.22 pyhocon-0.3.61 pynacl-1.5.0 pyparsing-3.2.3 pysocks-1.7.1 python-dateutil-2.9.0.post0 regex-2025.7.34 requests-2.32.4 robotframework-7.3.2 robotframework-httplibrary-0.4.2 robotframework-pythonlibcore-4.4.1 robotframework-requests-0.9.7 robotframework-selenium2library-3.0.0 robotframework-seleniumlibrary-6.7.1 robotframework-sshlibrary-3.8.0 scapy-2.6.1 scp-0.15.0 selenium-4.35.0 six-1.17.0 sniffio-1.3.1 sortedcontainers-2.4.0 soupsieve-2.7 trio-0.30.0 trio-websocket-0.12.2 typing-extensions-4.14.1 urllib3-2.5.0 waitress-3.0.2 websocket-client-1.8.0 webtest-3.0.6 wsproto-1.2.0
23:13:04  + pip freeze
23:13:04  attrs==25.3.0
23:13:04  bcrypt==4.3.0
23:13:04  beautifulsoup4==4.13.4
23:13:04  certifi==2025.8.3
23:13:04  cffi==1.17.1
23:13:04  charset-normalizer==3.4.3
23:13:04  click==8.2.1
23:13:04  cryptography==45.0.6
23:13:04  decorator==5.2.1
23:13:04  distlib==0.4.0
23:13:04  docker-py==1.10.6
23:13:04  docker-pycreds==0.4.0
23:13:04  elastic-transport==8.17.1
23:13:04  elasticsearch==8.19.0
23:13:04  elasticsearch-dsl==8.15.4
23:13:04  enum34==1.1.10
23:13:04  filelock==3.19.1
23:13:04  h11==0.16.0
23:13:04  idna==3.10
23:13:04  invoke==2.2.0
23:13:04  ipaddr==2.2.0
23:13:04  isodate==0.7.2
23:13:04  jmespath==1.0.1
23:13:04  jsonpatch==1.33
23:13:04  jsonpath-rw==1.4.0
23:13:04  jsonpointer==3.0.0
23:13:04  lxml==6.0.0
23:13:04  netaddr==1.3.0
23:13:04  netifaces==0.11.0
23:13:04  outcome==1.3.0.post0
23:13:04  paramiko==4.0.0
23:13:04  platformdirs==4.3.8
23:13:04  ply==3.11
23:13:04  pyang==2.6.1
23:13:04  pyangbind==0.8.6
23:13:04  pycparser==2.22
23:13:04  pyhocon==0.3.61
23:13:04  PyNaCl==1.5.0
23:13:04  pyparsing==3.2.3
23:13:04  PySocks==1.7.1
23:13:04  python-dateutil==2.9.0.post0
23:13:04  regex==2025.7.34
23:13:04  requests==2.32.4
23:13:04  robotframework==7.3.2
23:13:04  robotframework-httplibrary==0.4.2
23:13:04  robotframework-pythonlibcore==4.4.1
23:13:04  robotframework-requests==0.9.7
23:13:04  robotframework-selenium2library==3.0.0
23:13:04  robotframework-seleniumlibrary==6.7.1
23:13:04  robotframework-sshlibrary==3.8.0
23:13:04  scapy==2.6.1
23:13:04  scp==0.15.0
23:13:04  selenium==4.35.0
23:13:04  six==1.17.0
23:13:04  sniffio==1.3.1
23:13:04  sortedcontainers==2.4.0
23:13:04  soupsieve==2.7
23:13:04  trio==0.30.0
23:13:04  trio-websocket==0.12.2
23:13:04  typing_extensions==4.14.1
23:13:04  urllib3==2.5.0
23:13:04  virtualenv==20.34.0
23:13:04  waitress==3.0.2
23:13:04  WebOb==1.8.9
23:13:04  websocket-client==1.8.0
23:13:04  WebTest==3.0.6
23:13:04  wsproto==1.2.0
23:13:04  [EnvInject] - Injecting environment variables from a build step.
23:13:04  [EnvInject] - Injecting as environment variables the properties file path 'env.properties'
23:13:04  [EnvInject] - Variables injected successfully.
23:13:04  [controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium] $ /bin/bash -l /tmp/jenkins16889177354384204290.sh
23:13:04  Setup pyenv:
23:13:04    system
23:13:04    3.8.13
23:13:04    3.9.13
23:13:04    3.10.13
23:13:04  * 3.11.7 (set by /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/.python-version)
23:13:05  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-8Z2X from file:/tmp/.os_lf_venv
23:13:07  lf-activate-venv(): INFO: Installing: python-heatclient python-openstackclient yq 
23:13:13  ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
23:13:13  lftools 0.37.13 requires urllib3<2.1.0, but you have urllib3 2.5.0 which is incompatible.
23:13:14  lf-activate-venv(): INFO: Adding /tmp/venv-8Z2X/bin to PATH
23:13:14  + ODL_SYSTEM=()
23:13:14  + TOOLS_SYSTEM=()
23:13:14  + OPENSTACK_SYSTEM=()
23:13:14  + OPENSTACK_CONTROLLERS=()
23:13:14  + mapfile -t ADDR
23:13:14  ++ openstack stack show -f json -c outputs releng-controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium-69
23:13:14  ++ jq -r '.outputs[] | select(.output_key | match("^vm_[0-9]+_ips$")) | .output_value | .[]'
23:13:15  + for i in "${ADDR[@]}"
23:13:15  ++ ssh 10.30.170.232 hostname -s
23:13:15  Warning: Permanently added '10.30.170.232' (ECDSA) to the list of known hosts.
23:13:15  + REMHOST=releng-05380-69-0-builder-0
23:13:15  + case ${REMHOST} in
23:13:15  + ODL_SYSTEM=("${ODL_SYSTEM[@]}" "${i}")
23:13:15  + for i in "${ADDR[@]}"
23:13:15  ++ ssh 10.30.170.47 hostname -s
23:13:16  Warning: Permanently added '10.30.170.47' (ECDSA) to the list of known hosts.
23:13:16  + REMHOST=releng-05380-69-0-builder-1
23:13:16  + case ${REMHOST} in
23:13:16  + ODL_SYSTEM=("${ODL_SYSTEM[@]}" "${i}")
23:13:16  + for i in "${ADDR[@]}"
23:13:16  ++ ssh 10.30.170.110 hostname -s
23:13:16  Warning: Permanently added '10.30.170.110' (ECDSA) to the list of known hosts.
23:13:16  + REMHOST=releng-05380-69-0-builder-2
23:13:16  + case ${REMHOST} in
23:13:16  + ODL_SYSTEM=("${ODL_SYSTEM[@]}" "${i}")
23:13:16  + for i in "${ADDR[@]}"
23:13:16  ++ ssh 10.30.170.56 hostname -s
23:13:16  Warning: Permanently added '10.30.170.56' (ECDSA) to the list of known hosts.
23:13:17  + REMHOST=releng-05380-69-1-mininet-ovs-217-0
23:13:17  + case ${REMHOST} in
23:13:17  + TOOLS_SYSTEM=("${TOOLS_SYSTEM[@]}" "${i}")
23:13:17  + echo NUM_ODL_SYSTEM=3
23:13:17  + echo NUM_TOOLS_SYSTEM=1
23:13:17  + '[' '' == yes ']'
23:13:17  + NUM_OPENSTACK_SYSTEM=0
23:13:17  + echo NUM_OPENSTACK_SYSTEM=0
23:13:17  + '[' 0 -eq 2 ']'
23:13:17  + echo ODL_SYSTEM_IP=10.30.170.232
23:13:17  ++ seq 0 2
23:13:17  + for i in $(seq 0 $(( ${#ODL_SYSTEM[@]} - 1 )))
23:13:17  + echo ODL_SYSTEM_1_IP=10.30.170.232
23:13:17  + for i in $(seq 0 $(( ${#ODL_SYSTEM[@]} - 1 )))
23:13:17  + echo ODL_SYSTEM_2_IP=10.30.170.47
23:13:17  + for i in $(seq 0 $(( ${#ODL_SYSTEM[@]} - 1 )))
23:13:17  + echo ODL_SYSTEM_3_IP=10.30.170.110
23:13:17  + echo TOOLS_SYSTEM_IP=10.30.170.56
23:13:17  ++ seq 0 0
23:13:17  + for i in $(seq 0 $(( ${#TOOLS_SYSTEM[@]} - 1 )))
23:13:17  + echo TOOLS_SYSTEM_1_IP=10.30.170.56
23:13:17  + openstack_index=0
23:13:17  + NUM_OPENSTACK_CONTROL_NODES=1
23:13:17  + echo NUM_OPENSTACK_CONTROL_NODES=1
23:13:17  ++ seq 0 0
23:13:17  + for i in $(seq 0 $((NUM_OPENSTACK_CONTROL_NODES - 1)))
23:13:17  + echo OPENSTACK_CONTROL_NODE_1_IP=
23:13:17  + NUM_OPENSTACK_COMPUTE_NODES=-1
23:13:17  + echo NUM_OPENSTACK_COMPUTE_NODES=-1
23:13:17  + '[' -1 -ge 2 ']'
23:13:17  ++ seq 0 -2
23:13:17  + NUM_OPENSTACK_HAPROXY_NODES=0
23:13:17  + echo NUM_OPENSTACK_HAPROXY_NODES=0
23:13:17  ++ seq 0 -1
23:13:17  + echo 'Contents of slave_addresses.txt:'
23:13:17  Contents of slave_addresses.txt:
23:13:17  + cat slave_addresses.txt
23:13:17  NUM_ODL_SYSTEM=3
23:13:17  NUM_TOOLS_SYSTEM=1
23:13:17  NUM_OPENSTACK_SYSTEM=0
23:13:17  ODL_SYSTEM_IP=10.30.170.232
23:13:17  ODL_SYSTEM_1_IP=10.30.170.232
23:13:17  ODL_SYSTEM_2_IP=10.30.170.47
23:13:17  ODL_SYSTEM_3_IP=10.30.170.110
23:13:17  TOOLS_SYSTEM_IP=10.30.170.56
23:13:17  TOOLS_SYSTEM_1_IP=10.30.170.56
23:13:17  NUM_OPENSTACK_CONTROL_NODES=1
23:13:17  OPENSTACK_CONTROL_NODE_1_IP=
23:13:17  NUM_OPENSTACK_COMPUTE_NODES=-1
23:13:17  NUM_OPENSTACK_HAPROXY_NODES=0
23:13:17  [EnvInject] - Injecting environment variables from a build step.
23:13:17  [EnvInject] - Injecting as environment variables the properties file path 'slave_addresses.txt'
23:13:17  [EnvInject] - Variables injected successfully.
23:13:17  [controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium] $ /bin/sh /tmp/jenkins16701921285274372967.sh
23:13:17  Preparing for JRE Version 21
23:13:17  Karaf artifact is karaf
23:13:17  Karaf project is integration
23:13:17  Java home is /usr/lib/jvm/java-21-openjdk-amd64
23:13:17  [EnvInject] - Injecting environment variables from a build step.
23:13:17  [EnvInject] - Injecting as environment variables the properties file path 'set_variables.env'
23:13:17  [EnvInject] - Variables injected successfully.
23:13:17  [controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium] $ /bin/bash /tmp/jenkins14301465218598020259.sh
23:13:17  2025-08-16 23:13:17 URL:https://raw.githubusercontent.com/opendaylight/integration-distribution/stable/scandium/pom.xml [2619/2619] -> "pom.xml" [1]
23:13:17  Bundle version is 0.21.3-SNAPSHOT
23:13:17  --2025-08-16 23:13:17--  https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.3-SNAPSHOT/maven-metadata.xml
23:13:17  Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6
23:13:17  Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected.
23:13:17  HTTP request sent, awaiting response... 200 OK
23:13:17  Length: 1410 (1.4K) [application/xml]
23:13:17  Saving to: ‘maven-metadata.xml’
23:13:17  
23:13:17       0K .                                                     100% 45.8M=0s
23:13:17  
23:13:17  2025-08-16 23:13:17 (45.8 MB/s) - ‘maven-metadata.xml’ saved [1410/1410]
23:13:17  
23:13:17  <?xml version="1.0" encoding="UTF-8"?>
23:13:17  <metadata modelVersion="1.1.0">
23:13:17    <groupId>org.opendaylight.integration</groupId>
23:13:17    <artifactId>karaf</artifactId>
23:13:17    <version>0.21.3-SNAPSHOT</version>
23:13:17    <versioning>
23:13:17      <snapshot>
23:13:17        <timestamp>20250816.212524</timestamp>
23:13:17        <buildNumber>110</buildNumber>
23:13:17      </snapshot>
23:13:17      <lastUpdated>20250816212524</lastUpdated>
23:13:17      <snapshotVersions>
23:13:17        <snapshotVersion>
23:13:17          <extension>pom</extension>
23:13:17          <value>0.21.3-20250816.212524-110</value>
23:13:17          <updated>20250816212524</updated>
23:13:17        </snapshotVersion>
23:13:17        <snapshotVersion>
23:13:17          <extension>tar.gz</extension>
23:13:17          <value>0.21.3-20250816.212524-110</value>
23:13:17          <updated>20250816212524</updated>
23:13:17        </snapshotVersion>
23:13:17        <snapshotVersion>
23:13:17          <extension>zip</extension>
23:13:17          <value>0.21.3-20250816.212524-110</value>
23:13:17          <updated>20250816212524</updated>
23:13:17        </snapshotVersion>
23:13:17        <snapshotVersion>
23:13:17          <classifier>cyclonedx</classifier>
23:13:17          <extension>xml</extension>
23:13:17          <value>0.21.3-20250816.212524-110</value>
23:13:17          <updated>20250816212524</updated>
23:13:17        </snapshotVersion>
23:13:17        <snapshotVersion>
23:13:17          <classifier>cyclonedx</classifier>
23:13:17          <extension>json</extension>
23:13:17          <value>0.21.3-20250816.212524-110</value>
23:13:17          <updated>20250816212524</updated>
23:13:17        </snapshotVersion>
23:13:17      </snapshotVersions>
23:13:17    </versioning>
23:13:17  </metadata>
23:13:17  Nexus timestamp is 0.21.3-20250816.212524-110
23:13:17  Distribution bundle URL is https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.3-SNAPSHOT/karaf-0.21.3-20250816.212524-110.zip
23:13:17  Distribution bundle is karaf-0.21.3-20250816.212524-110.zip
23:13:17  Distribution bundle version is 0.21.3-SNAPSHOT
23:13:17  Distribution folder is karaf-0.21.3-SNAPSHOT
23:13:17  Nexus prefix is https://nexus.opendaylight.org
23:13:17  [EnvInject] - Injecting environment variables from a build step.
23:13:17  [EnvInject] - Injecting as environment variables the properties file path 'detect_variables.env'
23:13:17  [EnvInject] - Variables injected successfully.
23:13:17  [controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium] $ /bin/bash -l /tmp/jenkins10043267735190407953.sh
23:13:17  Setup pyenv:
23:13:17    system
23:13:17    3.8.13
23:13:17    3.9.13
23:13:17    3.10.13
23:13:17  * 3.11.7 (set by /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/.python-version)
23:13:18  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-8Z2X from file:/tmp/.os_lf_venv
23:13:19  ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
23:13:19  lftools 0.37.13 requires urllib3<2.1.0, but you have urllib3 2.5.0 which is incompatible.
23:13:20  lf-activate-venv(): INFO: Installing: python-heatclient python-openstackclient 
23:13:25  ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
23:13:25  lftools 0.37.13 requires urllib3<2.1.0, but you have urllib3 2.5.0 which is incompatible.
23:13:25  lf-activate-venv(): INFO: Adding /tmp/venv-8Z2X/bin to PATH
23:13:25  Copying common-functions.sh to /tmp
23:13:27  Copying common-functions.sh to 10.30.170.232:/tmp
23:13:27  Warning: Permanently added '10.30.170.232' (ECDSA) to the list of known hosts.
23:13:28  Copying common-functions.sh to 10.30.170.47:/tmp
23:13:28  Warning: Permanently added '10.30.170.47' (ECDSA) to the list of known hosts.
23:13:28  Copying common-functions.sh to 10.30.170.110:/tmp
23:13:28  Warning: Permanently added '10.30.170.110' (ECDSA) to the list of known hosts.
23:13:29  Copying common-functions.sh to 10.30.170.56:/tmp
23:13:29  Warning: Permanently added '10.30.170.56' (ECDSA) to the list of known hosts.
23:13:29  [controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium] $ /bin/bash /tmp/jenkins18154650112346330874.sh
23:13:29  common-functions.sh is being sourced
23:13:29  common-functions environment:
23:13:29  MAVENCONF: /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:13:29  ACTUALFEATURES: 
23:13:29  FEATURESCONF: /tmp/karaf-0.21.3-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:13:29  CUSTOMPROP: /tmp/karaf-0.21.3-SNAPSHOT/etc/custom.properties
23:13:29  LOGCONF: /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:13:29  MEMCONF: /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:29  CONTROLLERMEM: 2048m
23:13:29  AKKACONF: /tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/akka.conf
23:13:29  MODULESCONF: /tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/modules.conf
23:13:29  MODULESHARDSCONF: /tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/module-shards.conf
23:13:29  SUITES: 
23:13:29  
23:13:29  #################################################
23:13:29  ##         Configure Cluster and Start         ##
23:13:29  #################################################
23:13:30  ACTUALFEATURES: odl-infrautils-ready,odl-jolokia,odl-restconf-noauth,odl-clustering-test-app
23:13:30  SPACE_SEPARATED_FEATURES: odl-infrautils-ready odl-jolokia odl-restconf-noauth odl-clustering-test-app
23:13:30  Locating script plan to use...
23:13:30  Finished running script plans
23:13:30  Configuring member-1 with IP address 10.30.170.232
23:13:30  Warning: Permanently added '10.30.170.232' (ECDSA) to the list of known hosts.
23:13:31  Warning: Permanently added '10.30.170.232' (ECDSA) to the list of known hosts.
23:13:31  + source /tmp/common-functions.sh karaf-0.21.3-SNAPSHOT scandium
23:13:31  common-functions.sh is being sourced
23:13:31  ++ [[ /tmp/common-functions.sh == \/\t\m\p\/\c\o\n\f\i\g\u\r\a\t\i\o\n\-\s\c\r\i\p\t\.\s\h ]]
23:13:31  ++ echo 'common-functions.sh is being sourced'
23:13:31  ++ BUNDLEFOLDER=karaf-0.21.3-SNAPSHOT
23:13:31  ++ DISTROSTREAM=scandium
23:13:31  ++ export MAVENCONF=/tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:13:31  ++ MAVENCONF=/tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:13:31  ++ export FEATURESCONF=/tmp/karaf-0.21.3-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:13:31  ++ FEATURESCONF=/tmp/karaf-0.21.3-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:13:31  ++ export CUSTOMPROP=/tmp/karaf-0.21.3-SNAPSHOT/etc/custom.properties
23:13:31  ++ CUSTOMPROP=/tmp/karaf-0.21.3-SNAPSHOT/etc/custom.properties
23:13:31  ++ export LOGCONF=/tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:13:31  ++ LOGCONF=/tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:13:31  ++ export MEMCONF=/tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:31  ++ MEMCONF=/tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:31  ++ export CONTROLLERMEM=
23:13:31  ++ CONTROLLERMEM=
23:13:31  ++ case "${DISTROSTREAM}" in
23:13:31  ++ CLUSTER_SYSTEM=akka
23:13:31  ++ export AKKACONF=/tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/akka.conf
23:13:31  ++ AKKACONF=/tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/akka.conf
23:13:31  ++ export MODULESCONF=/tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/modules.conf
23:13:31  ++ MODULESCONF=/tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/modules.conf
23:13:31  ++ export MODULESHARDSCONF=/tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/module-shards.conf
23:13:31  ++ MODULESHARDSCONF=/tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/module-shards.conf
23:13:31  ++ print_common_env
23:13:31  ++ cat
23:13:31  common-functions environment:
23:13:31  MAVENCONF: /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:13:31  ACTUALFEATURES: 
23:13:31  FEATURESCONF: /tmp/karaf-0.21.3-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:13:31  CUSTOMPROP: /tmp/karaf-0.21.3-SNAPSHOT/etc/custom.properties
23:13:31  LOGCONF: /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:13:31  MEMCONF: /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:31  CONTROLLERMEM: 
23:13:31  AKKACONF: /tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/akka.conf
23:13:31  MODULESCONF: /tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/modules.conf
23:13:31  MODULESHARDSCONF: /tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/module-shards.conf
23:13:31  SUITES: 
23:13:31  
23:13:31  ++ SSH='ssh -t -t'
23:13:31  ++ extra_services_cntl='     dnsmasq.service     httpd.service     libvirtd.service     openvswitch.service     ovs-vswitchd.service     ovsdb-server.service     rabbitmq-server.service '
23:13:31  ++ extra_services_cmp='     libvirtd.service     openvswitch.service     ovs-vswitchd.service     ovsdb-server.service '
23:13:31  Changing to /tmp
23:13:31  Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.3-SNAPSHOT/karaf-0.21.3-20250816.212524-110.zip
23:13:31  + echo 'Changing to /tmp'
23:13:31  + cd /tmp
23:13:31  + echo 'Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.3-SNAPSHOT/karaf-0.21.3-20250816.212524-110.zip'
23:13:31  + wget --progress=dot:mega https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.3-SNAPSHOT/karaf-0.21.3-20250816.212524-110.zip
23:13:31  --2025-08-16 23:13:30--  https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.3-SNAPSHOT/karaf-0.21.3-20250816.212524-110.zip
23:13:31  Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6
23:13:31  Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected.
23:13:31  HTTP request sent, awaiting response... 200 OK
23:13:31  Length: 187828019 (179M) [application/zip]
23:13:31  Saving to: ‘karaf-0.21.3-20250816.212524-110.zip’
23:13:31  
23:13:31       0K ........ ........ ........ ........ ........ ........  1% 83.4M 2s
23:13:31    3072K ........ ........ ........ ........ ........ ........  3%  133M 2s
23:13:31    6144K ........ ........ ........ ........ ........ ........  5%  151M 1s
23:13:31    9216K ........ ........ ........ ........ ........ ........  6%  187M 1s
23:13:31   12288K ........ ........ ........ ........ ........ ........  8%  171M 1s
23:13:31   15360K ........ ........ ........ ........ ........ ........ 10%  161M 1s
23:13:31   18432K ........ ........ ........ ........ ........ ........ 11%  183M 1s
23:13:31   21504K ........ ........ ........ ........ ........ ........ 13%  194M 1s
23:13:31   24576K ........ ........ ........ ........ ........ ........ 15%  203M 1s
23:13:31   27648K ........ ........ ........ ........ ........ ........ 16%  191M 1s
23:13:31   30720K ........ ........ ........ ........ ........ ........ 18%  183M 1s
23:13:31   33792K ........ ........ ........ ........ ........ ........ 20%  186M 1s
23:13:31   36864K ........ ........ ........ ........ ........ ........ 21% 69.6M 1s
23:13:31   39936K ........ ........ ........ ........ ........ ........ 23% 78.7M 1s
23:13:31   43008K ........ ........ ........ ........ ........ ........ 25% 80.3M 1s
23:13:31   46080K ........ ........ ........ ........ ........ ........ 26% 84.1M 1s
23:13:31   49152K ........ ........ ........ ........ ........ ........ 28% 84.7M 1s
23:13:31   52224K ........ ........ ........ ........ ........ ........ 30%  129M 1s
23:13:31   55296K ........ ........ ........ ........ ........ ........ 31%  107M 1s
23:13:31   58368K ........ ........ ........ ........ ........ ........ 33%  112M 1s
23:13:31   61440K ........ ........ ........ ........ ........ ........ 35%  125M 1s
23:13:31   64512K ........ ........ ........ ........ ........ ........ 36%  137M 1s
23:13:31   67584K ........ ........ ........ ........ ........ ........ 38%  119M 1s
23:13:31   70656K ........ ........ ........ ........ ........ ........ 40% 90.7M 1s
23:13:31   73728K ........ ........ ........ ........ ........ ........ 41%  126M 1s
23:13:31   76800K ........ ........ ........ ........ ........ ........ 43% 93.3M 1s
23:13:31   79872K ........ ........ ........ ........ ........ ........ 45% 81.0M 1s
23:13:31   82944K ........ ........ ........ ........ ........ ........ 46% 82.4M 1s
23:13:31   86016K ........ ........ ........ ........ ........ ........ 48% 91.7M 1s
23:13:31   89088K ........ ........ ........ ........ ........ ........ 50%  159M 1s
23:13:31   92160K ........ ........ ........ ........ ........ ........ 51%  112M 1s
23:13:31   95232K ........ ........ ........ ........ ........ ........ 53%  102M 1s
23:13:31   98304K ........ ........ ........ ........ ........ ........ 55%  144M 1s
23:13:31  101376K ........ ........ ........ ........ ........ ........ 56%  123M 1s
23:13:31  104448K ........ ........ ........ ........ ........ ........ 58%  121M 1s
23:13:31  107520K ........ ........ ........ ........ ........ ........ 60%  216M 1s
23:13:31  110592K ........ ........ ........ ........ ........ ........ 61%  140M 1s
23:13:31  113664K ........ ........ ........ ........ ........ ........ 63%  119M 1s
23:13:31  116736K ........ ........ ........ ........ ........ ........ 65%  115M 1s
23:13:31  119808K ........ ........ ........ ........ ........ ........ 66%  100M 1s
23:13:31  122880K ........ ........ ........ ........ ........ ........ 68%  100M 0s
23:13:31  125952K ........ ........ ........ ........ ........ ........ 70%  160M 0s
23:13:31  129024K ........ ........ ........ ........ ........ ........ 72%  121M 0s
23:13:31  132096K ........ ........ ........ ........ ........ ........ 73%  137M 0s
23:13:31  135168K ........ ........ ........ ........ ........ ........ 75%  165M 0s
23:13:31  138240K ........ ........ ........ ........ ........ ........ 77%  146M 0s
23:13:31  141312K ........ ........ ........ ........ ........ ........ 78% 61.9M 0s
23:13:31  144384K ........ ........ ........ ........ ........ ........ 80%  166M 0s
23:13:31  147456K ........ ........ ........ ........ ........ ........ 82%  148M 0s
23:13:31  150528K ........ ........ ........ ........ ........ ........ 83%  196M 0s
23:13:31  153600K ........ ........ ........ ........ ........ ........ 85%  162M 0s
23:13:31  156672K ........ ........ ........ ........ ........ ........ 87%  177M 0s
23:13:31  159744K ........ ........ ........ ........ ........ ........ 88%  155M 0s
23:13:31  162816K ........ ........ ........ ........ ........ ........ 90%  189M 0s
23:13:31  165888K ........ ........ ........ ........ ........ ........ 92%  189M 0s
23:13:31  168960K ........ ........ ........ ........ ........ ........ 93%  135M 0s
23:13:31  172032K ........ ........ ........ ........ ........ ........ 95%  213M 0s
23:13:31  175104K ........ ........ ........ ........ ........ ........ 97%  165M 0s
23:13:31  178176K ........ ........ ........ ........ ........ ........ 98%  195M 0s
23:13:31  181248K ........ ........ ........ ........ ..               100%  191M=1.4s
23:13:31  
23:13:31  2025-08-16 23:13:31 (126 MB/s) - ‘karaf-0.21.3-20250816.212524-110.zip’ saved [187828019/187828019]
23:13:31  
23:13:31  Extracting the new controller...
23:13:31  + echo 'Extracting the new controller...'
23:13:31  + unzip -q karaf-0.21.3-20250816.212524-110.zip
23:13:33  Adding external repositories...
23:13:33  + echo 'Adding external repositories...'
23:13:33  + sed -ie 's%org.ops4j.pax.url.mvn.repositories=%org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases%g' /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:13:33  + cat /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:13:33  ################################################################################
23:13:33  #
23:13:33  #    Licensed to the Apache Software Foundation (ASF) under one or more
23:13:33  #    contributor license agreements.  See the NOTICE file distributed with
23:13:33  #    this work for additional information regarding copyright ownership.
23:13:33  #    The ASF licenses this file to You under the Apache License, Version 2.0
23:13:33  #    (the "License"); you may not use this file except in compliance with
23:13:33  #    the License.  You may obtain a copy of the License at
23:13:33  #
23:13:33  #       http://www.apache.org/licenses/LICENSE-2.0
23:13:33  #
23:13:33  #    Unless required by applicable law or agreed to in writing, software
23:13:33  #    distributed under the License is distributed on an "AS IS" BASIS,
23:13:33  #    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23:13:33  #    See the License for the specific language governing permissions and
23:13:33  #    limitations under the License.
23:13:33  #
23:13:33  ################################################################################
23:13:33  
23:13:33  #
23:13:33  # If set to true, the following property will not allow any certificate to be used
23:13:33  # when accessing Maven repositories through SSL
23:13:33  #
23:13:33  #org.ops4j.pax.url.mvn.certificateCheck=
23:13:33  
23:13:33  #
23:13:33  # Path to the local Maven settings file.
23:13:33  # The repositories defined in this file will be automatically added to the list
23:13:33  # of default repositories if the 'org.ops4j.pax.url.mvn.repositories' property
23:13:33  # below is not set.
23:13:33  # The following locations are checked for the existence of the settings.xml file
23:13:33  #   * 1. looks for the specified url
23:13:33  #   * 2. if not found looks for ${user.home}/.m2/settings.xml
23:13:33  #   * 3. if not found looks for ${maven.home}/conf/settings.xml
23:13:33  #   * 4. if not found looks for ${M2_HOME}/conf/settings.xml
23:13:33  #
23:13:33  #org.ops4j.pax.url.mvn.settings=
23:13:33  
23:13:33  #
23:13:33  # Path to the local Maven repository which is used to avoid downloading
23:13:33  # artifacts when they already exist locally.
23:13:33  # The value of this property will be extracted from the settings.xml file
23:13:33  # above, or defaulted to:
23:13:33  #     System.getProperty( "user.home" ) + "/.m2/repository"
23:13:33  #
23:13:33  org.ops4j.pax.url.mvn.localRepository=${karaf.home}/${karaf.default.repository}
23:13:33  
23:13:33  #
23:13:33  # Default this to false. It's just weird to use undocumented repos
23:13:33  #
23:13:33  org.ops4j.pax.url.mvn.useFallbackRepositories=false
23:13:33  
23:13:33  #
23:13:33  # Uncomment if you don't wanna use the proxy settings
23:13:33  # from the Maven conf/settings.xml file
23:13:33  #
23:13:33  # org.ops4j.pax.url.mvn.proxySupport=false
23:13:33  
23:13:33  #
23:13:33  # Comma separated list of repositories scanned when resolving an artifact.
23:13:33  # Those repositories will be checked before iterating through the
23:13:33  #    below list of repositories and even before the local repository
23:13:33  # A repository url can be appended with zero or more of the following flags:
23:13:33  #    @snapshots  : the repository contains snaphots
23:13:33  #    @noreleases : the repository does not contain any released artifacts
23:13:33  #
23:13:33  # The following property value will add the system folder as a repo.
23:13:33  #
23:13:33  org.ops4j.pax.url.mvn.defaultRepositories=\
23:13:33      file:${karaf.home}/${karaf.default.repository}@id=system.repository@snapshots,\
23:13:33      file:${karaf.data}/kar@id=kar.repository@multi@snapshots,\
23:13:33      file:${karaf.base}/${karaf.default.repository}@id=child.system.repository@snapshots
23:13:33  
23:13:33  # Use the default local repo (e.g.~/.m2/repository) as a "remote" repo
23:13:33  #org.ops4j.pax.url.mvn.defaultLocalRepoAsRemote=false
23:13:33  
23:13:33  #
23:13:33  # Comma separated list of repositories scanned when resolving an artifact.
23:13:33  # The default list includes the following repositories:
23:13:33  #    http://repo1.maven.org/maven2@id=central
23:13:33  #    http://repository.springsource.com/maven/bundles/release@id=spring.ebr
23:13:33  #    http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external
23:13:33  #    http://zodiac.springsource.com/maven/bundles/release@id=gemini
23:13:33  #    http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases
23:13:33  #    https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases
23:13:33  #    https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases
23:13:33  # To add repositories to the default ones, prepend '+' to the list of repositories
23:13:33  # to add.
23:13:33  # A repository url can be appended with zero or more of the following flags:
23:13:33  #    @snapshots  : the repository contains snapshots
23:13:33  #    @noreleases : the repository does not contain any released artifacts
23:13:33  #    @id=repository.id : the id for the repository, just like in the settings.xml this is optional but recommended
23:13:33  #
23:13:33  org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases
23:13:33  
23:13:33  ### ^^^ No remote repositories. This is the only ODL change compared to Karaf defaults.Configuring the startup features...
23:13:33  + [[ True == \T\r\u\e ]]
23:13:33  + echo 'Configuring the startup features...'
23:13:33  + sed -ie 's/\(featuresBoot=\|featuresBoot =\)/featuresBoot = odl-infrautils-ready,odl-jolokia,odl-restconf-noauth,odl-clustering-test-app,/g' /tmp/karaf-0.21.3-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:13:33  + FEATURE_TEST_STRING=features-test
23:13:33  + FEATURE_TEST_VERSION=0.21.3-SNAPSHOT
23:13:33  + KARAF_VERSION=karaf4
23:13:33  + [[ integration == \i\n\t\e\g\r\a\t\i\o\n ]]
23:13:33  + sed -ie 's%\(featuresRepositories=\|featuresRepositories =\)%featuresRepositories = mvn:org.opendaylight.integration/features-test/0.21.3-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features,%g' /tmp/karaf-0.21.3-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:13:33  + [[ ! -z '' ]]
23:13:33  + cat /tmp/karaf-0.21.3-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:13:34  ################################################################################
23:13:34  #
23:13:34  #    Licensed to the Apache Software Foundation (ASF) under one or more
23:13:34  #    contributor license agreements.  See the NOTICE file distributed with
23:13:34  #    this work for additional information regarding copyright ownership.
23:13:34  #    The ASF licenses this file to You under the Apache License, Version 2.0
23:13:34  #    (the "License"); you may not use this file except in compliance with
23:13:34  #    the License.  You may obtain a copy of the License at
23:13:34  #
23:13:34  #       http://www.apache.org/licenses/LICENSE-2.0
23:13:34  #
23:13:34  #    Unless required by applicable law or agreed to in writing, software
23:13:34  #    distributed under the License is distributed on an "AS IS" BASIS,
23:13:34  #    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23:13:34  #    See the License for the specific language governing permissions and
23:13:34  #    limitations under the License.
23:13:34  #
23:13:34  ################################################################################
23:13:34  
23:13:34  #
23:13:34  # Comma separated list of features repositories to register by default
23:13:34  #
23:13:34  featuresRepositories = mvn:org.opendaylight.integration/features-test/0.21.3-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features, file:${karaf.etc}/09f88248-c0ed-4aad-82be-fc3ff480fbea.xml
23:13:34  
23:13:34  #
23:13:34  # Comma separated list of features to install at startup
23:13:34  #
23:13:34  featuresBoot = odl-infrautils-ready,odl-jolokia,odl-restconf-noauth,odl-clustering-test-app, 6edfc6d1-cf52-4045-8ea5-0bba3a9055c2
23:13:34  
23:13:34  #
23:13:34  # Resource repositories (OBR) that the features resolver can use
23:13:34  # to resolve requirements/capabilities
23:13:34  #
23:13:34  # The format of the resourceRepositories is 
23:13:34  # resourceRepositories=[xml:url|json:url],...
23:13:34  # for Instance:
23:13:34  #
23:13:34  #resourceRepositories=xml:http://host/path/to/index.xml
23:13:34  # or
23:13:34  #resourceRepositories=json:http://host/path/to/index.json
23:13:34  #
23:13:34  
23:13:34  #
23:13:34  # Defines if the boot features are started in asynchronous mode (in a dedicated thread)
23:13:34  #
23:13:34  featuresBootAsynchronous=false
23:13:34  
23:13:34  #
23:13:34  # Service requirements enforcement
23:13:34  #
23:13:34  # By default, the feature resolver checks the service requirements/capabilities of
23:13:34  # bundles for new features (xml schema >= 1.3.0) in order to automatically installs
23:13:34  # the required bundles.
23:13:34  # The following flag can have those values:
23:13:34  #   - disable: service requirements are completely ignored
23:13:34  #   - default: service requirements are ignored for old features
23:13:34  #   - enforce: service requirements are always verified
23:13:34  #
23:13:34  #serviceRequirements=default
23:13:34  
23:13:34  #
23:13:34  # Store cfg file for config element in feature
23:13:34  #
23:13:34  #configCfgStore=true
23:13:34  
23:13:34  #
23:13:34  # Define if the feature service automatically refresh bundles
23:13:34  #
23:13:34  autoRefresh=true
23:13:34  
23:13:34  #
23:13:34  # Configuration of features processing mechanism (overrides, blacklisting, modification of features)
23:13:34  # XML file defines instructions related to features processing
23:13:34  # versions.properties may declare properties to resolve placeholders in XML file
23:13:34  # both files are relative to ${karaf.etc}
23:13:34  #
23:13:34  #featureProcessing=org.apache.karaf.features.xml
23:13:34  #featureProcessingVersions=versions.properties
23:13:34  + configure_karaf_log karaf4 ''
23:13:34  + local -r karaf_version=karaf4
23:13:34  + local -r controllerdebugmap=
23:13:34  + local logapi=log4j
23:13:34  + grep log4j2 /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:13:34  log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
23:13:34  log4j2.rootLogger.level = INFO
23:13:34  #log4j2.rootLogger.type = asyncRoot
23:13:34  #log4j2.rootLogger.includeLocation = false
23:13:34  log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile
23:13:34  log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi
23:13:34  log4j2.rootLogger.appenderRef.Console.ref = Console
23:13:34  log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter
23:13:34  log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF}
23:13:34  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter
23:13:34  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair
23:13:34  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker
23:13:34  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL
23:13:34  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or
23:13:34  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY
23:13:34  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL
23:13:34  log4j2.logger.spifly.name = org.apache.aries.spifly
23:13:34  log4j2.logger.spifly.level = WARN
23:13:34  log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit
23:13:34  log4j2.logger.audit.level = INFO
23:13:34  log4j2.logger.audit.additivity = false
23:13:34  log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile
23:13:34  # Console appender not used by default (see log4j2.rootLogger.appenderRefs)
23:13:34  log4j2.appender.console.type = Console
23:13:34  log4j2.appender.console.name = Console
23:13:34  log4j2.appender.console.layout.type = PatternLayout
23:13:34  log4j2.appender.console.layout.pattern = ${log4j2.pattern}
23:13:34  log4j2.appender.rolling.type = RollingRandomAccessFile
23:13:34  log4j2.appender.rolling.name = RollingFile
23:13:34  log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
23:13:34  log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i
23:13:34  #log4j2.appender.rolling.immediateFlush = false
23:13:34  log4j2.appender.rolling.append = true
23:13:34  log4j2.appender.rolling.layout.type = PatternLayout
23:13:34  log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
23:13:34  log4j2.appender.rolling.policies.type = Policies
23:13:34  log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
23:13:34  log4j2.appender.rolling.policies.size.size = 64MB
23:13:34  log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
23:13:34  log4j2.appender.rolling.strategy.max = 7
23:13:34  log4j2.appender.audit.type = RollingRandomAccessFile
23:13:34  log4j2.appender.audit.name = AuditRollingFile
23:13:34  log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log
23:13:34  log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i
23:13:34  log4j2.appender.audit.append = true
23:13:34  log4j2.appender.audit.layout.type = PatternLayout
23:13:34  log4j2.appender.audit.layout.pattern = ${log4j2.pattern}
23:13:34  log4j2.appender.audit.policies.type = Policies
23:13:34  log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy
23:13:34  log4j2.appender.audit.policies.size.size = 8MB
23:13:34  log4j2.appender.audit.strategy.type = DefaultRolloverStrategy
23:13:34  log4j2.appender.audit.strategy.max = 7
23:13:34  log4j2.appender.osgi.type = PaxOsgi
23:13:34  log4j2.appender.osgi.name = PaxOsgi
23:13:34  log4j2.appender.osgi.filter = *
23:13:34  #log4j2.logger.aether.name = shaded.org.eclipse.aether
23:13:34  #log4j2.logger.aether.level = TRACE
23:13:34  #log4j2.logger.http-headers.name = shaded.org.apache.http.headers
23:13:34  #log4j2.logger.http-headers.level = DEBUG
23:13:34  #log4j2.logger.maven.name = org.ops4j.pax.url.mvn
23:13:34  #log4j2.logger.maven.level = TRACE
23:13:34  + logapi=log4j2
23:13:34  + echo 'Configuring the karaf log... karaf_version: karaf4, logapi: log4j2'
23:13:34  Configuring the karaf log... karaf_version: karaf4, logapi: log4j2
23:13:34  + '[' log4j2 == log4j2 ']'
23:13:34  + sed -ie 's/log4j2.appender.rolling.policies.size.size = 64MB/log4j2.appender.rolling.policies.size.size = 1GB/g' /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:13:34  + orgmodule=org.opendaylight.yangtools.yang.parser.repo.YangTextSchemaContextResolver
23:13:34  + orgmodule_=org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver
23:13:34  + echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN'
23:13:34  + echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN'
23:13:34  controllerdebugmap: 
23:13:34  cat /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:13:34  + unset IFS
23:13:34  + echo 'controllerdebugmap: '
23:13:34  + '[' -n '' ']'
23:13:34  + echo 'cat /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg'
23:13:34  + cat /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:13:34  ################################################################################
23:13:34  #
23:13:34  #    Licensed to the Apache Software Foundation (ASF) under one or more
23:13:34  #    contributor license agreements.  See the NOTICE file distributed with
23:13:34  #    this work for additional information regarding copyright ownership.
23:13:34  #    The ASF licenses this file to You under the Apache License, Version 2.0
23:13:34  #    (the "License"); you may not use this file except in compliance with
23:13:34  #    the License.  You may obtain a copy of the License at
23:13:34  #
23:13:34  #       http://www.apache.org/licenses/LICENSE-2.0
23:13:34  #
23:13:34  #    Unless required by applicable law or agreed to in writing, software
23:13:34  #    distributed under the License is distributed on an "AS IS" BASIS,
23:13:34  #    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23:13:34  #    See the License for the specific language governing permissions and
23:13:34  #    limitations under the License.
23:13:34  #
23:13:34  ################################################################################
23:13:34  
23:13:34  # Common pattern layout for appenders
23:13:34  log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
23:13:34  
23:13:34  # Root logger
23:13:34  log4j2.rootLogger.level = INFO
23:13:34  # uncomment to use asynchronous loggers, which require mvn:com.lmax/disruptor/3.3.2 library
23:13:34  #log4j2.rootLogger.type = asyncRoot
23:13:34  #log4j2.rootLogger.includeLocation = false
23:13:34  log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile
23:13:34  log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi
23:13:34  log4j2.rootLogger.appenderRef.Console.ref = Console
23:13:34  log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter
23:13:34  log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF}
23:13:34  
23:13:34  # Filters for logs marked by org.opendaylight.odlparent.Markers
23:13:34  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter
23:13:34  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair
23:13:34  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker
23:13:34  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL
23:13:34  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or
23:13:34  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY
23:13:34  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL
23:13:34  
23:13:34  # Loggers configuration
23:13:34  
23:13:34  # Spifly logger
23:13:34  log4j2.logger.spifly.name = org.apache.aries.spifly
23:13:34  log4j2.logger.spifly.level = WARN
23:13:34  
23:13:34  # Security audit logger
23:13:34  log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit
23:13:34  log4j2.logger.audit.level = INFO
23:13:34  log4j2.logger.audit.additivity = false
23:13:34  log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile
23:13:34  
23:13:34  # Appenders configuration
23:13:34  
23:13:34  # Console appender not used by default (see log4j2.rootLogger.appenderRefs)
23:13:34  log4j2.appender.console.type = Console
23:13:34  log4j2.appender.console.name = Console
23:13:34  log4j2.appender.console.layout.type = PatternLayout
23:13:34  log4j2.appender.console.layout.pattern = ${log4j2.pattern}
23:13:34  
23:13:34  # Rolling file appender
23:13:34  log4j2.appender.rolling.type = RollingRandomAccessFile
23:13:34  log4j2.appender.rolling.name = RollingFile
23:13:34  log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
23:13:34  log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i
23:13:34  # uncomment to not force a disk flush
23:13:34  #log4j2.appender.rolling.immediateFlush = false
23:13:34  log4j2.appender.rolling.append = true
23:13:34  log4j2.appender.rolling.layout.type = PatternLayout
23:13:34  log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
23:13:34  log4j2.appender.rolling.policies.type = Policies
23:13:34  log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
23:13:34  log4j2.appender.rolling.policies.size.size = 1GB
23:13:34  log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
23:13:34  log4j2.appender.rolling.strategy.max = 7
23:13:34  
23:13:34  # Audit file appender
23:13:34  log4j2.appender.audit.type = RollingRandomAccessFile
23:13:34  log4j2.appender.audit.name = AuditRollingFile
23:13:34  log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log
23:13:34  log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i
23:13:34  log4j2.appender.audit.append = true
23:13:34  log4j2.appender.audit.layout.type = PatternLayout
23:13:34  log4j2.appender.audit.layout.pattern = ${log4j2.pattern}
23:13:34  log4j2.appender.audit.policies.type = Policies
23:13:34  log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy
23:13:34  log4j2.appender.audit.policies.size.size = 8MB
23:13:34  log4j2.appender.audit.strategy.type = DefaultRolloverStrategy
23:13:34  log4j2.appender.audit.strategy.max = 7
23:13:34  
23:13:34  # OSGi appender
23:13:34  log4j2.appender.osgi.type = PaxOsgi
23:13:34  log4j2.appender.osgi.name = PaxOsgi
23:13:34  log4j2.appender.osgi.filter = *
23:13:34  
23:13:34  # help with identification of maven-related problems with pax-url-aether
23:13:34  #log4j2.logger.aether.name = shaded.org.eclipse.aether
23:13:34  #log4j2.logger.aether.level = TRACE
23:13:34  #log4j2.logger.http-headers.name = shaded.org.apache.http.headers
23:13:34  #log4j2.logger.http-headers.level = DEBUG
23:13:34  #log4j2.logger.maven.name = org.ops4j.pax.url.mvn
23:13:34  #log4j2.logger.maven.level = TRACE
23:13:34  log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN
23:13:34  log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN
23:13:34  + set_java_vars /usr/lib/jvm/java-21-openjdk-amd64 2048m /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:34  + local -r java_home=/usr/lib/jvm/java-21-openjdk-amd64
23:13:34  + local -r controllermem=2048m
23:13:34  + local -r memconf=/tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:34  Configure
23:13:34      java home: /usr/lib/jvm/java-21-openjdk-amd64
23:13:34      max memory: 2048m
23:13:34      memconf: /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:34  + echo Configure
23:13:34  + echo '    java home: /usr/lib/jvm/java-21-openjdk-amd64'
23:13:34  + echo '    max memory: 2048m'
23:13:34  + echo '    memconf: /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv'
23:13:34  + sed -ie 's%^# export JAVA_HOME%export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-21-openjdk-amd64}%g' /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:34  + sed -ie 's/JAVA_MAX_MEM="2048m"/JAVA_MAX_MEM=2048m/g' /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:34  cat /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:34  + echo 'cat /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv'
23:13:34  + cat /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:34  #!/bin/sh
23:13:34  #
23:13:34  #    Licensed to the Apache Software Foundation (ASF) under one or more
23:13:34  #    contributor license agreements.  See the NOTICE file distributed with
23:13:34  #    this work for additional information regarding copyright ownership.
23:13:34  #    The ASF licenses this file to You under the Apache License, Version 2.0
23:13:34  #    (the "License"); you may not use this file except in compliance with
23:13:34  #    the License.  You may obtain a copy of the License at
23:13:34  #
23:13:34  #       http://www.apache.org/licenses/LICENSE-2.0
23:13:34  #
23:13:34  #    Unless required by applicable law or agreed to in writing, software
23:13:34  #    distributed under the License is distributed on an "AS IS" BASIS,
23:13:34  #    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23:13:34  #    See the License for the specific language governing permissions and
23:13:34  #    limitations under the License.
23:13:34  #
23:13:34  
23:13:34  #
23:13:34  # handle specific scripts; the SCRIPT_NAME is exactly the name of the Karaf
23:13:34  # script: client, instance, shell, start, status, stop, karaf
23:13:34  #
23:13:34  # if [ "${KARAF_SCRIPT}" == "SCRIPT_NAME" ]; then
23:13:34  #   Actions go here...
23:13:34  # fi
23:13:34  
23:13:34  #
23:13:34  # general settings which should be applied for all scripts go here; please keep
23:13:34  # in mind that it is possible that scripts might be executed more than once, e.g.
23:13:34  # in example of the start script where the start script is executed first and the
23:13:34  # karaf script afterwards.
23:13:34  #
23:13:34  
23:13:34  #
23:13:34  # The following section shows the possible configuration options for the default 
23:13:34  # karaf scripts
23:13:34  #
23:13:34  export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-21-openjdk-amd64} # Location of Java installation
23:13:34  # export JAVA_OPTS # Generic JVM options, for instance, where you can pass the memory configuration
23:13:34  # export JAVA_NON_DEBUG_OPTS # Additional non-debug JVM options
23:13:34  # export EXTRA_JAVA_OPTS # Additional JVM options
23:13:34  # export KARAF_HOME # Karaf home folder
23:13:34  # export KARAF_DATA # Karaf data folder
23:13:34  # export KARAF_BASE # Karaf base folder
23:13:34  # export KARAF_ETC  # Karaf etc  folder
23:13:34  # export KARAF_LOG  # Karaf log  folder
23:13:34  # export KARAF_SYSTEM_OPTS # First citizen Karaf options
23:13:34  # export KARAF_OPTS # Additional available Karaf options
23:13:34  # export KARAF_DEBUG # Enable debug mode
23:13:34  # export KARAF_REDIRECT # Enable/set the std/err redirection when using bin/start
23:13:34  # export KARAF_NOROOT # Prevent execution as root if set to true
23:13:34  Set Java version
23:13:34  + echo 'Set Java version'
23:13:34  + sudo /usr/sbin/alternatives --install /usr/bin/java java /usr/lib/jvm/java-21-openjdk-amd64/bin/java 1
23:13:34  sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper
23:13:34  sudo: a password is required
23:13:34  + sudo /usr/sbin/alternatives --set java /usr/lib/jvm/java-21-openjdk-amd64/bin/java
23:13:34  sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper
23:13:34  sudo: a password is required
23:13:34  JDK default version ...
23:13:34  + echo 'JDK default version ...'
23:13:34  + java -version
23:13:35  openjdk version "21.0.5" 2024-10-15
23:13:35  OpenJDK Runtime Environment (build 21.0.5+11-Ubuntu-1ubuntu122.04)
23:13:35  OpenJDK 64-Bit Server VM (build 21.0.5+11-Ubuntu-1ubuntu122.04, mixed mode, sharing)
23:13:35  Set JAVA_HOME
23:13:35  + echo 'Set JAVA_HOME'
23:13:35  + export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
23:13:35  + JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
23:13:35  ++ readlink -e /usr/lib/jvm/java-21-openjdk-amd64/bin/java
23:13:35  Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-21-openjdk-amd64/bin/java
23:13:35  + JAVA_RESOLVED=/usr/lib/jvm/java-21-openjdk-amd64/bin/java
23:13:35  + echo 'Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-21-openjdk-amd64/bin/java'
23:13:35  + echo 'Listing all open ports on controller system...'
23:13:35  Listing all open ports on controller system...
23:13:35  + netstat -pnatu
23:13:35  /tmp/configuration-script.sh: line 40: netstat: command not found
23:13:35  Configuring cluster
23:13:35  + '[' -f /tmp/custom_shard_config.txt ']'
23:13:35  + echo 'Configuring cluster'
23:13:35  + /tmp/karaf-0.21.3-SNAPSHOT/bin/configure_cluster.sh 1 10.30.170.232 10.30.170.47 10.30.170.110
23:13:35  ################################################
23:13:35  ##             Configure Cluster              ##
23:13:35  ################################################
23:13:35   NOTE: Cluster configuration files not found. Copying from
23:13:35   /tmp/karaf-0.21.3-SNAPSHOT/system/org/opendaylight/controller/sal-clustering-config/10.0.11
23:13:35  Configuring unique name in akka.conf
23:13:35  Configuring hostname in akka.conf
23:13:35  Configuring data and rpc seed nodes in akka.conf
23:13:35  modules = [
23:13:35  	
23:13:35  	{
23:13:35  		name = "inventory"
23:13:35  		namespace = "urn:opendaylight:inventory"
23:13:35  		shard-strategy = "module"
23:13:35  	},
23:13:35  	{
23:13:35  		name = "topology"
23:13:35  		namespace = "urn:TBD:params:xml:ns:yang:network-topology"
23:13:35  		shard-strategy = "module"
23:13:35  	},
23:13:35  	{
23:13:35  		name = "toaster"
23:13:35  		namespace = "http://netconfcentral.org/ns/toaster"
23:13:35  		shard-strategy = "module"
23:13:35  	}
23:13:35  ]
23:13:35  Configuring replication type in module-shards.conf
23:13:35  ################################################
23:13:35  ##   NOTE: Manually restart controller to     ##
23:13:35  ##         apply configuration.               ##
23:13:35  ################################################
23:13:35  Dump akka.conf
23:13:35  + echo 'Dump akka.conf'
23:13:35  + cat /tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/akka.conf
23:13:35  
23:13:35  odl-cluster-data {
23:13:35    akka {
23:13:35      remote {
23:13:35        artery {
23:13:35          enabled = on
23:13:35          transport = tcp
23:13:35          canonical.hostname = "10.30.170.232"
23:13:35          canonical.port = 2550
23:13:35        }
23:13:35      }
23:13:35  
23:13:35      cluster {
23:13:35        # Using artery.
23:13:35        seed-nodes = ["akka://opendaylight-cluster-data@10.30.170.232:2550",
23:13:35  				"akka://opendaylight-cluster-data@10.30.170.47:2550",
23:13:35  				"akka://opendaylight-cluster-data@10.30.170.110:2550"]
23:13:35  
23:13:35        roles = ["member-1"]
23:13:35  
23:13:35        # when under load we might trip a false positive on the failure detector
23:13:35        # failure-detector {
23:13:35          # heartbeat-interval = 4 s
23:13:35          # acceptable-heartbeat-pause = 16s
23:13:35        # }
23:13:35      }
23:13:35  
23:13:35      persistence {
23:13:35        # By default the snapshots/journal directories live in KARAF_HOME. You can choose to put it somewhere else by
23:13:35        # modifying the following two properties. The directory location specified may be a relative or absolute path. 
23:13:35        # The relative path is always relative to KARAF_HOME.
23:13:35  
23:13:35        # snapshot-store.local.dir = "target/snapshots"
23:13:35  
23:13:35        # Use lz4 compression for LocalSnapshotStore snapshots
23:13:35        snapshot-store.local.use-lz4-compression = false
23:13:35        # Size of blocks for lz4 compression: 64KB, 256KB, 1MB or 4MB
23:13:35        snapshot-store.local.lz4-blocksize = 256KB
23:13:35      }
23:13:35      disable-default-actor-system-quarantined-event-handling = "false"
23:13:35    }
23:13:35  }
23:13:35  Dump modules.conf
23:13:35  + echo 'Dump modules.conf'
23:13:35  + cat /tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/modules.conf
23:13:35  modules = [
23:13:35  	
23:13:35  	{
23:13:35  		name = "inventory"
23:13:35  		namespace = "urn:opendaylight:inventory"
23:13:35  		shard-strategy = "module"
23:13:35  	},
23:13:35  	{
23:13:35  		name = "topology"
23:13:35  		namespace = "urn:TBD:params:xml:ns:yang:network-topology"
23:13:35  		shard-strategy = "module"
23:13:35  	},
23:13:35  	{
23:13:35  		name = "toaster"
23:13:35  		namespace = "http://netconfcentral.org/ns/toaster"
23:13:35  		shard-strategy = "module"
23:13:35  	}
23:13:35  ]
23:13:35  Dump module-shards.conf
23:13:35  + echo 'Dump module-shards.conf'
23:13:35  + cat /tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/module-shards.conf
23:13:35  module-shards = [
23:13:35  	{
23:13:35  		name = "default"
23:13:35  		shards = [
23:13:35  			{
23:13:35  				name = "default"
23:13:35  				replicas = ["member-1",
23:13:35  				"member-2",
23:13:35  				"member-3"]
23:13:35  			}
23:13:35  		]
23:13:35  	},
23:13:35  	{
23:13:35  		name = "inventory"
23:13:35  		shards = [
23:13:35  			{
23:13:35  				name="inventory"
23:13:35  				replicas = ["member-1",
23:13:35  				"member-2",
23:13:35  				"member-3"]
23:13:35  			}
23:13:35  		]
23:13:35  	},
23:13:35  	{
23:13:35  		name = "topology"
23:13:35  		shards = [
23:13:35  			{
23:13:35  				name="topology"
23:13:35  				replicas = ["member-1",
23:13:35  				"member-2",
23:13:35  				"member-3"]
23:13:35  			}
23:13:35  		]
23:13:35  	},
23:13:35  	{
23:13:35  		name = "toaster"
23:13:35  		shards = [
23:13:35  			{
23:13:35  				name="toaster"
23:13:35  				replicas = ["member-1",
23:13:35  				"member-2",
23:13:35  				"member-3"]
23:13:35  			}
23:13:35  		]
23:13:35  	}
23:13:35  ]
23:13:35  Configuring member-2 with IP address 10.30.170.47
23:13:35  Warning: Permanently added '10.30.170.47' (ECDSA) to the list of known hosts.
23:13:35  Warning: Permanently added '10.30.170.47' (ECDSA) to the list of known hosts.
23:13:35  + source /tmp/common-functions.sh karaf-0.21.3-SNAPSHOT scandium
23:13:35  ++ [[ /tmp/common-functions.sh == \/\t\m\p\/\c\o\n\f\i\g\u\r\a\t\i\o\n\-\s\c\r\i\p\t\.\s\h ]]
23:13:35  common-functions.sh is being sourced
23:13:35  ++ echo 'common-functions.sh is being sourced'
23:13:35  ++ BUNDLEFOLDER=karaf-0.21.3-SNAPSHOT
23:13:35  ++ DISTROSTREAM=scandium
23:13:35  ++ export MAVENCONF=/tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:13:35  ++ MAVENCONF=/tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:13:35  ++ export FEATURESCONF=/tmp/karaf-0.21.3-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:13:35  ++ FEATURESCONF=/tmp/karaf-0.21.3-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:13:35  ++ export CUSTOMPROP=/tmp/karaf-0.21.3-SNAPSHOT/etc/custom.properties
23:13:35  ++ CUSTOMPROP=/tmp/karaf-0.21.3-SNAPSHOT/etc/custom.properties
23:13:35  ++ export LOGCONF=/tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:13:35  ++ LOGCONF=/tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:13:35  ++ export MEMCONF=/tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:35  ++ MEMCONF=/tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:35  ++ export CONTROLLERMEM=
23:13:35  ++ CONTROLLERMEM=
23:13:35  ++ case "${DISTROSTREAM}" in
23:13:35  ++ CLUSTER_SYSTEM=akka
23:13:35  ++ export AKKACONF=/tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/akka.conf
23:13:35  ++ AKKACONF=/tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/akka.conf
23:13:35  ++ export MODULESCONF=/tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/modules.conf
23:13:35  ++ MODULESCONF=/tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/modules.conf
23:13:35  ++ export MODULESHARDSCONF=/tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/module-shards.conf
23:13:35  ++ MODULESHARDSCONF=/tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/module-shards.conf
23:13:35  ++ print_common_env
23:13:35  ++ cat
23:13:35  common-functions environment:
23:13:35  MAVENCONF: /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:13:35  ACTUALFEATURES: 
23:13:35  FEATURESCONF: /tmp/karaf-0.21.3-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:13:35  CUSTOMPROP: /tmp/karaf-0.21.3-SNAPSHOT/etc/custom.properties
23:13:35  LOGCONF: /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:13:35  MEMCONF: /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:35  CONTROLLERMEM: 
23:13:35  AKKACONF: /tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/akka.conf
23:13:35  MODULESCONF: /tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/modules.conf
23:13:35  MODULESHARDSCONF: /tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/module-shards.conf
23:13:35  SUITES: 
23:13:35  
23:13:35  ++ SSH='ssh -t -t'
23:13:35  ++ extra_services_cntl='     dnsmasq.service     httpd.service     libvirtd.service     openvswitch.service     ovs-vswitchd.service     ovsdb-server.service     rabbitmq-server.service '
23:13:35  ++ extra_services_cmp='     libvirtd.service     openvswitch.service     ovs-vswitchd.service     ovsdb-server.service '
23:13:35  Changing to /tmp
23:13:35  Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.3-SNAPSHOT/karaf-0.21.3-20250816.212524-110.zip
23:13:35  + echo 'Changing to /tmp'
23:13:35  + cd /tmp
23:13:35  + echo 'Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.3-SNAPSHOT/karaf-0.21.3-20250816.212524-110.zip'
23:13:35  + wget --progress=dot:mega https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.3-SNAPSHOT/karaf-0.21.3-20250816.212524-110.zip
23:13:35  --2025-08-16 23:13:34--  https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.3-SNAPSHOT/karaf-0.21.3-20250816.212524-110.zip
23:13:35  Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6
23:13:35  Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected.
23:13:35  HTTP request sent, awaiting response... 200 OK
23:13:35  Length: 187828019 (179M) [application/zip]
23:13:35  Saving to: ‘karaf-0.21.3-20250816.212524-110.zip’
23:13:35  
23:13:35       0K ........ ........ ........ ........ ........ ........  1% 60.7M 3s
23:13:35    3072K ........ ........ ........ ........ ........ ........  3%  124M 2s
23:13:35    6144K ........ ........ ........ ........ ........ ........  5%  140M 2s
23:13:35    9216K ........ ........ ........ ........ ........ ........  6%  189M 2s
23:13:35   12288K ........ ........ ........ ........ ........ ........  8%  160M 1s
23:13:35   15360K ........ ........ ........ ........ ........ ........ 10%  109M 1s
23:13:35   18432K ........ ........ ........ ........ ........ ........ 11% 77.3M 1s
23:13:35   21504K ........ ........ ........ ........ ........ ........ 13%  201M 1s
23:13:35   24576K ........ ........ ........ ........ ........ ........ 15%  180M 1s
23:13:35   27648K ........ ........ ........ ........ ........ ........ 16%  182M 1s
23:13:35   30720K ........ ........ ........ ........ ........ ........ 18%  200M 1s
23:13:35   33792K ........ ........ ........ ........ ........ ........ 20%  208M 1s
23:13:35   36864K ........ ........ ........ ........ ........ ........ 21%  193M 1s
23:13:35   39936K ........ ........ ........ ........ ........ ........ 23%  219M 1s
23:13:35   43008K ........ ........ ........ ........ ........ ........ 25%  211M 1s
23:13:35   46080K ........ ........ ........ ........ ........ ........ 26%  222M 1s
23:13:35   49152K ........ ........ ........ ........ ........ ........ 28%  225M 1s
23:13:35   52224K ........ ........ ........ ........ ........ ........ 30%  218M 1s
23:13:35   55296K ........ ........ ........ ........ ........ ........ 31%  242M 1s
23:13:35   58368K ........ ........ ........ ........ ........ ........ 33%  218M 1s
23:13:35   61440K ........ ........ ........ ........ ........ ........ 35%  224M 1s
23:13:35   64512K ........ ........ ........ ........ ........ ........ 36%  201M 1s
23:13:35   67584K ........ ........ ........ ........ ........ ........ 38%  248M 1s
23:13:35   70656K ........ ........ ........ ........ ........ ........ 40%  243M 1s
23:13:35   73728K ........ ........ ........ ........ ........ ........ 41%  222M 1s
23:13:35   76800K ........ ........ ........ ........ ........ ........ 43%  196M 1s
23:13:35   79872K ........ ........ ........ ........ ........ ........ 45%  157M 1s
23:13:35   82944K ........ ........ ........ ........ ........ ........ 46%  124M 1s
23:13:35   86016K ........ ........ ........ ........ ........ ........ 48%  114M 1s
23:13:35   89088K ........ ........ ........ ........ ........ ........ 50%  142M 1s
23:13:35   92160K ........ ........ ........ ........ ........ ........ 51%  158M 1s
23:13:35   95232K ........ ........ ........ ........ ........ ........ 53%  122M 1s
23:13:35   98304K ........ ........ ........ ........ ........ ........ 55%  169M 0s
23:13:35  101376K ........ ........ ........ ........ ........ ........ 56%  203M 0s
23:13:35  104448K ........ ........ ........ ........ ........ ........ 58%  218M 0s
23:13:35  107520K ........ ........ ........ ........ ........ ........ 60%  209M 0s
23:13:35  110592K ........ ........ ........ ........ ........ ........ 61%  210M 0s
23:13:35  113664K ........ ........ ........ ........ ........ ........ 63%  219M 0s
23:13:35  116736K ........ ........ ........ ........ ........ ........ 65%  183M 0s
23:13:35  119808K ........ ........ ........ ........ ........ ........ 66%  148M 0s
23:13:35  122880K ........ ........ ........ ........ ........ ........ 68%  158M 0s
23:13:35  125952K ........ ........ ........ ........ ........ ........ 70%  203M 0s
23:13:35  129024K ........ ........ ........ ........ ........ ........ 72%  245M 0s
23:13:35  132096K ........ ........ ........ ........ ........ ........ 73%  170M 0s
23:13:35  135168K ........ ........ ........ ........ ........ ........ 75%  169M 0s
23:13:35  138240K ........ ........ ........ ........ ........ ........ 77%  152M 0s
23:13:35  141312K ........ ........ ........ ........ ........ ........ 78%  178M 0s
23:13:35  144384K ........ ........ ........ ........ ........ ........ 80%  163M 0s
23:13:35  147456K ........ ........ ........ ........ ........ ........ 82%  152M 0s
23:13:35  150528K ........ ........ ........ ........ ........ ........ 83%  143M 0s
23:13:35  153600K ........ ........ ........ ........ ........ ........ 85%  149M 0s
23:13:35  156672K ........ ........ ........ ........ ........ ........ 87%  152M 0s
23:13:35  159744K ........ ........ ........ ........ ........ ........ 88%  147M 0s
23:13:35  162816K ........ ........ ........ ........ ........ ........ 90%  174M 0s
23:13:35  165888K ........ ........ ........ ........ ........ ........ 92%  185M 0s
23:13:35  168960K ........ ........ ........ ........ ........ ........ 93%  196M 0s
23:13:35  172032K ........ ........ ........ ........ ........ ........ 95%  170M 0s
23:13:37  175104K ........ ........ ........ ........ ........ ........ 97%  181M 0s
23:13:37  178176K ........ ........ ........ ........ ........ ........ 98%  189M 0s
23:13:37  181248K ........ ........ ........ ........ ..               100%  162M=1.1s
23:13:37  
23:13:37  2025-08-16 23:13:35 (167 MB/s) - ‘karaf-0.21.3-20250816.212524-110.zip’ saved [187828019/187828019]
23:13:37  
23:13:37  Extracting the new controller...
23:13:37  + echo 'Extracting the new controller...'
23:13:37  + unzip -q karaf-0.21.3-20250816.212524-110.zip
23:13:37  Adding external repositories...
23:13:37  + echo 'Adding external repositories...'
23:13:37  + sed -ie 's%org.ops4j.pax.url.mvn.repositories=%org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases%g' /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:13:37  + cat /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:13:37  ################################################################################
23:13:37  #
23:13:37  #    Licensed to the Apache Software Foundation (ASF) under one or more
23:13:37  #    contributor license agreements.  See the NOTICE file distributed with
23:13:37  #    this work for additional information regarding copyright ownership.
23:13:37  #    The ASF licenses this file to You under the Apache License, Version 2.0
23:13:37  #    (the "License"); you may not use this file except in compliance with
23:13:37  #    the License.  You may obtain a copy of the License at
23:13:37  #
23:13:37  #       http://www.apache.org/licenses/LICENSE-2.0
23:13:37  #
23:13:37  #    Unless required by applicable law or agreed to in writing, software
23:13:37  #    distributed under the License is distributed on an "AS IS" BASIS,
23:13:37  #    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23:13:37  #    See the License for the specific language governing permissions and
23:13:37  #    limitations under the License.
23:13:37  #
23:13:37  ################################################################################
23:13:37  
23:13:37  #
23:13:37  # If set to true, the following property will not allow any certificate to be used
23:13:37  # when accessing Maven repositories through SSL
23:13:37  #
23:13:37  #org.ops4j.pax.url.mvn.certificateCheck=
23:13:37  
23:13:37  #
23:13:37  # Path to the local Maven settings file.
23:13:37  # The repositories defined in this file will be automatically added to the list
23:13:37  # of default repositories if the 'org.ops4j.pax.url.mvn.repositories' property
23:13:37  # below is not set.
23:13:37  # The following locations are checked for the existence of the settings.xml file
23:13:37  #   * 1. looks for the specified url
23:13:37  #   * 2. if not found looks for ${user.home}/.m2/settings.xml
23:13:37  #   * 3. if not found looks for ${maven.home}/conf/settings.xml
23:13:37  #   * 4. if not found looks for ${M2_HOME}/conf/settings.xml
23:13:37  #
23:13:37  #org.ops4j.pax.url.mvn.settings=
23:13:37  
23:13:37  #
23:13:37  # Path to the local Maven repository which is used to avoid downloading
23:13:37  # artifacts when they already exist locally.
23:13:37  # The value of this property will be extracted from the settings.xml file
23:13:37  # above, or defaulted to:
23:13:37  #     System.getProperty( "user.home" ) + "/.m2/repository"
23:13:37  #
23:13:37  org.ops4j.pax.url.mvn.localRepository=${karaf.home}/${karaf.default.repository}
23:13:37  
23:13:37  #
23:13:37  # Default this to false. It's just weird to use undocumented repos
23:13:37  #
23:13:37  org.ops4j.pax.url.mvn.useFallbackRepositories=false
23:13:37  
23:13:37  #
23:13:37  # Uncomment if you don't wanna use the proxy settings
23:13:37  # from the Maven conf/settings.xml file
23:13:37  #
23:13:37  # org.ops4j.pax.url.mvn.proxySupport=false
23:13:37  
23:13:37  #
23:13:37  # Comma separated list of repositories scanned when resolving an artifact.
23:13:37  # Those repositories will be checked before iterating through the
23:13:37  #    below list of repositories and even before the local repository
23:13:37  # A repository url can be appended with zero or more of the following flags:
23:13:37  #    @snapshots  : the repository contains snaphots
23:13:37  #    @noreleases : the repository does not contain any released artifacts
23:13:37  #
23:13:37  # The following property value will add the system folder as a repo.
23:13:37  #
23:13:37  org.ops4j.pax.url.mvn.defaultRepositories=\
23:13:37      file:${karaf.home}/${karaf.default.repository}@id=system.repository@snapshots,\
23:13:37      file:${karaf.data}/kar@id=kar.repository@multi@snapshots,\
23:13:37      file:${karaf.base}/${karaf.default.repository}@id=child.system.repository@snapshots
23:13:37  
23:13:37  # Use the default local repo (e.g.~/.m2/repository) as a "remote" repo
23:13:37  #org.ops4j.pax.url.mvn.defaultLocalRepoAsRemote=false
23:13:37  
23:13:37  #
23:13:37  # Comma separated list of repositories scanned when resolving an artifact.
23:13:37  # The default list includes the following repositories:
23:13:37  #    http://repo1.maven.org/maven2@id=central
23:13:37  #    http://repository.springsource.com/maven/bundles/release@id=spring.ebr
23:13:37  #    http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external
23:13:37  #    http://zodiac.springsource.com/maven/bundles/release@id=gemini
23:13:37  #    http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases
23:13:37  #    https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases
23:13:37  #    https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases
23:13:37  # To add repositories to the default ones, prepend '+' to the list of repositories
23:13:37  # to add.
23:13:37  # A repository url can be appended with zero or more of the following flags:
23:13:37  #    @snapshots  : the repository contains snapshots
23:13:37  #    @noreleases : the repository does not contain any released artifacts
23:13:37  #    @id=repository.id : the id for the repository, just like in the settings.xml this is optional but recommended
23:13:37  #
23:13:37  org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases
23:13:37  
23:13:37  ### ^^^ No remote repositories. This is the only ODL change compared to Karaf defaults.+ [[ True == \T\r\u\e ]]
23:13:37  Configuring the startup features...
23:13:37  + echo 'Configuring the startup features...'
23:13:37  + sed -ie 's/\(featuresBoot=\|featuresBoot =\)/featuresBoot = odl-infrautils-ready,odl-jolokia,odl-restconf-noauth,odl-clustering-test-app,/g' /tmp/karaf-0.21.3-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:13:37  + FEATURE_TEST_STRING=features-test
23:13:37  + FEATURE_TEST_VERSION=0.21.3-SNAPSHOT
23:13:37  + KARAF_VERSION=karaf4
23:13:37  + [[ integration == \i\n\t\e\g\r\a\t\i\o\n ]]
23:13:37  + sed -ie 's%\(featuresRepositories=\|featuresRepositories =\)%featuresRepositories = mvn:org.opendaylight.integration/features-test/0.21.3-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features,%g' /tmp/karaf-0.21.3-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:13:37  + [[ ! -z '' ]]
23:13:37  + cat /tmp/karaf-0.21.3-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:13:37  ################################################################################
23:13:37  #
23:13:37  #    Licensed to the Apache Software Foundation (ASF) under one or more
23:13:37  #    contributor license agreements.  See the NOTICE file distributed with
23:13:37  #    this work for additional information regarding copyright ownership.
23:13:37  #    The ASF licenses this file to You under the Apache License, Version 2.0
23:13:37  #    (the "License"); you may not use this file except in compliance with
23:13:37  #    the License.  You may obtain a copy of the License at
23:13:37  #
23:13:37  #       http://www.apache.org/licenses/LICENSE-2.0
23:13:37  #
23:13:37  #    Unless required by applicable law or agreed to in writing, software
23:13:37  #    distributed under the License is distributed on an "AS IS" BASIS,
23:13:37  #    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23:13:37  #    See the License for the specific language governing permissions and
23:13:37  #    limitations under the License.
23:13:37  #
23:13:37  ################################################################################
23:13:37  
23:13:37  #
23:13:37  # Comma separated list of features repositories to register by default
23:13:37  #
23:13:37  featuresRepositories = mvn:org.opendaylight.integration/features-test/0.21.3-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features, file:${karaf.etc}/09f88248-c0ed-4aad-82be-fc3ff480fbea.xml
23:13:37  
23:13:37  #
23:13:37  # Comma separated list of features to install at startup
23:13:37  #
23:13:37  featuresBoot = odl-infrautils-ready,odl-jolokia,odl-restconf-noauth,odl-clustering-test-app, 6edfc6d1-cf52-4045-8ea5-0bba3a9055c2
23:13:37  
23:13:37  #
23:13:37  # Resource repositories (OBR) that the features resolver can use
23:13:37  # to resolve requirements/capabilities
23:13:37  #
23:13:37  # The format of the resourceRepositories is 
23:13:37  # resourceRepositories=[xml:url|json:url],...
23:13:37  # for Instance:
23:13:37  #
23:13:37  #resourceRepositories=xml:http://host/path/to/index.xml
23:13:37  # or
23:13:37  #resourceRepositories=json:http://host/path/to/index.json
23:13:37  #
23:13:37  
23:13:37  #
23:13:37  # Defines if the boot features are started in asynchronous mode (in a dedicated thread)
23:13:37  #
23:13:37  featuresBootAsynchronous=false
23:13:37  
23:13:37  #
23:13:37  # Service requirements enforcement
23:13:37  #
23:13:37  # By default, the feature resolver checks the service requirements/capabilities of
23:13:37  # bundles for new features (xml schema >= 1.3.0) in order to automatically installs
23:13:37  # the required bundles.
23:13:37  # The following flag can have those values:
23:13:37  #   - disable: service requirements are completely ignored
23:13:37  #   - default: service requirements are ignored for old features
23:13:37  #   - enforce: service requirements are always verified
23:13:37  #
23:13:37  #serviceRequirements=default
23:13:37  
23:13:37  #
23:13:37  # Store cfg file for config element in feature
23:13:37  #
23:13:37  #configCfgStore=true
23:13:37  
23:13:37  #
23:13:37  # Define if the feature service automatically refresh bundles
23:13:37  #
23:13:37  autoRefresh=true
23:13:37  
23:13:37  #
23:13:37  # Configuration of features processing mechanism (overrides, blacklisting, modification of features)
23:13:37  # XML file defines instructions related to features processing
23:13:37  # versions.properties may declare properties to resolve placeholders in XML file
23:13:37  # both files are relative to ${karaf.etc}
23:13:37  #
23:13:37  #featureProcessing=org.apache.karaf.features.xml
23:13:37  #featureProcessingVersions=versions.properties
23:13:37  + configure_karaf_log karaf4 ''
23:13:37  + local -r karaf_version=karaf4
23:13:37  + local -r controllerdebugmap=
23:13:37  + local logapi=log4j
23:13:37  + grep log4j2 /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:13:37  log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
23:13:37  log4j2.rootLogger.level = INFO
23:13:37  #log4j2.rootLogger.type = asyncRoot
23:13:37  #log4j2.rootLogger.includeLocation = false
23:13:37  log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile
23:13:37  log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi
23:13:37  log4j2.rootLogger.appenderRef.Console.ref = Console
23:13:37  log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter
23:13:37  log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF}
23:13:37  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter
23:13:37  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair
23:13:37  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker
23:13:37  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL
23:13:37  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or
23:13:37  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY
23:13:37  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL
23:13:37  log4j2.logger.spifly.name = org.apache.aries.spifly
23:13:37  log4j2.logger.spifly.level = WARN
23:13:37  log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit
23:13:37  log4j2.logger.audit.level = INFO
23:13:37  log4j2.logger.audit.additivity = false
23:13:37  log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile
23:13:37  # Console appender not used by default (see log4j2.rootLogger.appenderRefs)
23:13:37  log4j2.appender.console.type = Console
23:13:37  log4j2.appender.console.name = Console
23:13:37  log4j2.appender.console.layout.type = PatternLayout
23:13:37  log4j2.appender.console.layout.pattern = ${log4j2.pattern}
23:13:37  log4j2.appender.rolling.type = RollingRandomAccessFile
23:13:37  log4j2.appender.rolling.name = RollingFile
23:13:37  log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
23:13:37  log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i
23:13:37  #log4j2.appender.rolling.immediateFlush = false
23:13:37  log4j2.appender.rolling.append = true
23:13:37  log4j2.appender.rolling.layout.type = PatternLayout
23:13:37  log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
23:13:37  log4j2.appender.rolling.policies.type = Policies
23:13:37  log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
23:13:37  log4j2.appender.rolling.policies.size.size = 64MB
23:13:37  log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
23:13:37  log4j2.appender.rolling.strategy.max = 7
23:13:37  log4j2.appender.audit.type = RollingRandomAccessFile
23:13:37  log4j2.appender.audit.name = AuditRollingFile
23:13:37  log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log
23:13:37  log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i
23:13:37  log4j2.appender.audit.append = true
23:13:37  log4j2.appender.audit.layout.type = PatternLayout
23:13:37  log4j2.appender.audit.layout.pattern = ${log4j2.pattern}
23:13:37  log4j2.appender.audit.policies.type = Policies
23:13:37  log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy
23:13:37  log4j2.appender.audit.policies.size.size = 8MB
23:13:37  log4j2.appender.audit.strategy.type = DefaultRolloverStrategy
23:13:37  log4j2.appender.audit.strategy.max = 7
23:13:37  log4j2.appender.osgi.type = PaxOsgi
23:13:37  log4j2.appender.osgi.name = PaxOsgi
23:13:37  log4j2.appender.osgi.filter = *
23:13:37  #log4j2.logger.aether.name = shaded.org.eclipse.aether
23:13:37  #log4j2.logger.aether.level = TRACE
23:13:37  #log4j2.logger.http-headers.name = shaded.org.apache.http.headers
23:13:37  #log4j2.logger.http-headers.level = DEBUG
23:13:37  #log4j2.logger.maven.name = org.ops4j.pax.url.mvn
23:13:37  #log4j2.logger.maven.level = TRACE
23:13:37  Configuring the karaf log... karaf_version: karaf4, logapi: log4j2
23:13:37  + logapi=log4j2
23:13:37  + echo 'Configuring the karaf log... karaf_version: karaf4, logapi: log4j2'
23:13:37  + '[' log4j2 == log4j2 ']'
23:13:37  + sed -ie 's/log4j2.appender.rolling.policies.size.size = 64MB/log4j2.appender.rolling.policies.size.size = 1GB/g' /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:13:37  + orgmodule=org.opendaylight.yangtools.yang.parser.repo.YangTextSchemaContextResolver
23:13:37  + orgmodule_=org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver
23:13:37  + echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN'
23:13:37  + echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN'
23:13:37  + unset IFS
23:13:37  + echo 'controllerdebugmap: '
23:13:37  controllerdebugmap: 
23:13:37  cat /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:13:37  + '[' -n '' ']'
23:13:37  + echo 'cat /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg'
23:13:37  + cat /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:13:37  ################################################################################
23:13:37  #
23:13:37  #    Licensed to the Apache Software Foundation (ASF) under one or more
23:13:37  #    contributor license agreements.  See the NOTICE file distributed with
23:13:37  #    this work for additional information regarding copyright ownership.
23:13:37  #    The ASF licenses this file to You under the Apache License, Version 2.0
23:13:37  #    (the "License"); you may not use this file except in compliance with
23:13:37  #    the License.  You may obtain a copy of the License at
23:13:37  #
23:13:37  #       http://www.apache.org/licenses/LICENSE-2.0
23:13:37  #
23:13:37  #    Unless required by applicable law or agreed to in writing, software
23:13:37  #    distributed under the License is distributed on an "AS IS" BASIS,
23:13:37  #    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23:13:37  #    See the License for the specific language governing permissions and
23:13:37  #    limitations under the License.
23:13:37  #
23:13:37  ################################################################################
23:13:37  
23:13:37  # Common pattern layout for appenders
23:13:37  log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
23:13:37  
23:13:37  # Root logger
23:13:37  log4j2.rootLogger.level = INFO
23:13:37  # uncomment to use asynchronous loggers, which require mvn:com.lmax/disruptor/3.3.2 library
23:13:37  #log4j2.rootLogger.type = asyncRoot
23:13:37  #log4j2.rootLogger.includeLocation = false
23:13:37  log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile
23:13:37  log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi
23:13:37  log4j2.rootLogger.appenderRef.Console.ref = Console
23:13:37  log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter
23:13:37  log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF}
23:13:37  
23:13:37  # Filters for logs marked by org.opendaylight.odlparent.Markers
23:13:37  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter
23:13:37  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair
23:13:37  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker
23:13:37  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL
23:13:37  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or
23:13:37  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY
23:13:37  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL
23:13:37  
23:13:37  # Loggers configuration
23:13:37  
23:13:37  # Spifly logger
23:13:37  log4j2.logger.spifly.name = org.apache.aries.spifly
23:13:37  log4j2.logger.spifly.level = WARN
23:13:37  
23:13:37  # Security audit logger
23:13:37  log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit
23:13:37  log4j2.logger.audit.level = INFO
23:13:37  log4j2.logger.audit.additivity = false
23:13:37  log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile
23:13:37  
23:13:37  # Appenders configuration
23:13:37  
23:13:37  # Console appender not used by default (see log4j2.rootLogger.appenderRefs)
23:13:37  log4j2.appender.console.type = Console
23:13:37  log4j2.appender.console.name = Console
23:13:37  log4j2.appender.console.layout.type = PatternLayout
23:13:37  log4j2.appender.console.layout.pattern = ${log4j2.pattern}
23:13:37  
23:13:37  # Rolling file appender
23:13:37  log4j2.appender.rolling.type = RollingRandomAccessFile
23:13:37  log4j2.appender.rolling.name = RollingFile
23:13:37  log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
23:13:37  log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i
23:13:37  # uncomment to not force a disk flush
23:13:37  #log4j2.appender.rolling.immediateFlush = false
23:13:37  log4j2.appender.rolling.append = true
23:13:37  log4j2.appender.rolling.layout.type = PatternLayout
23:13:37  log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
23:13:37  log4j2.appender.rolling.policies.type = Policies
23:13:37  log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
23:13:37  log4j2.appender.rolling.policies.size.size = 1GB
23:13:37  log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
23:13:37  log4j2.appender.rolling.strategy.max = 7
23:13:37  
23:13:37  # Audit file appender
23:13:37  log4j2.appender.audit.type = RollingRandomAccessFile
23:13:37  log4j2.appender.audit.name = AuditRollingFile
23:13:37  log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log
23:13:37  log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i
23:13:37  log4j2.appender.audit.append = true
23:13:37  log4j2.appender.audit.layout.type = PatternLayout
23:13:37  log4j2.appender.audit.layout.pattern = ${log4j2.pattern}
23:13:37  log4j2.appender.audit.policies.type = Policies
23:13:37  log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy
23:13:37  log4j2.appender.audit.policies.size.size = 8MB
23:13:37  log4j2.appender.audit.strategy.type = DefaultRolloverStrategy
23:13:37  log4j2.appender.audit.strategy.max = 7
23:13:37  
23:13:37  # OSGi appender
23:13:37  log4j2.appender.osgi.type = PaxOsgi
23:13:37  log4j2.appender.osgi.name = PaxOsgi
23:13:37  log4j2.appender.osgi.filter = *
23:13:37  
23:13:37  # help with identification of maven-related problems with pax-url-aether
23:13:37  #log4j2.logger.aether.name = shaded.org.eclipse.aether
23:13:37  #log4j2.logger.aether.level = TRACE
23:13:37  #log4j2.logger.http-headers.name = shaded.org.apache.http.headers
23:13:37  #log4j2.logger.http-headers.level = DEBUG
23:13:37  #log4j2.logger.maven.name = org.ops4j.pax.url.mvn
23:13:37  #log4j2.logger.maven.level = TRACE
23:13:37  log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN
23:13:37  log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN
23:13:37  Configure
23:13:37      java home: /usr/lib/jvm/java-21-openjdk-amd64
23:13:37  + set_java_vars /usr/lib/jvm/java-21-openjdk-amd64 2048m /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:37  + local -r java_home=/usr/lib/jvm/java-21-openjdk-amd64
23:13:37  + local -r controllermem=2048m
23:13:37  + local -r memconf=/tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:37  + echo Configure
23:13:37  + echo '    java home: /usr/lib/jvm/java-21-openjdk-amd64'
23:13:37      max memory: 2048m
23:13:37      memconf: /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:37  + echo '    max memory: 2048m'
23:13:37  + echo '    memconf: /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv'
23:13:37  + sed -ie 's%^# export JAVA_HOME%export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-21-openjdk-amd64}%g' /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:37  + sed -ie 's/JAVA_MAX_MEM="2048m"/JAVA_MAX_MEM=2048m/g' /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:37  cat /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:37  + echo 'cat /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv'
23:13:37  + cat /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:37  #!/bin/sh
23:13:37  #
23:13:37  #    Licensed to the Apache Software Foundation (ASF) under one or more
23:13:37  #    contributor license agreements.  See the NOTICE file distributed with
23:13:37  #    this work for additional information regarding copyright ownership.
23:13:37  #    The ASF licenses this file to You under the Apache License, Version 2.0
23:13:37  #    (the "License"); you may not use this file except in compliance with
23:13:37  #    the License.  You may obtain a copy of the License at
23:13:37  #
23:13:37  #       http://www.apache.org/licenses/LICENSE-2.0
23:13:37  #
23:13:37  #    Unless required by applicable law or agreed to in writing, software
23:13:37  #    distributed under the License is distributed on an "AS IS" BASIS,
23:13:37  #    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23:13:37  #    See the License for the specific language governing permissions and
23:13:37  #    limitations under the License.
23:13:37  #
23:13:37  
23:13:37  #
23:13:37  # handle specific scripts; the SCRIPT_NAME is exactly the name of the Karaf
23:13:37  # script: client, instance, shell, start, status, stop, karaf
23:13:37  #
23:13:37  # if [ "${KARAF_SCRIPT}" == "SCRIPT_NAME" ]; then
23:13:37  #   Actions go here...
23:13:37  # fi
23:13:37  
23:13:37  #
23:13:37  # general settings which should be applied for all scripts go here; please keep
23:13:37  # in mind that it is possible that scripts might be executed more than once, e.g.
23:13:37  # in example of the start script where the start script is executed first and the
23:13:37  # karaf script afterwards.
23:13:37  #
23:13:37  
23:13:37  #
23:13:37  # The following section shows the possible configuration options for the default 
23:13:37  # karaf scripts
23:13:37  #
23:13:37  export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-21-openjdk-amd64} # Location of Java installation
23:13:37  # export JAVA_OPTS # Generic JVM options, for instance, where you can pass the memory configuration
23:13:37  # export JAVA_NON_DEBUG_OPTS # Additional non-debug JVM options
23:13:37  # export EXTRA_JAVA_OPTS # Additional JVM options
23:13:37  # export KARAF_HOME # Karaf home folder
23:13:37  # export KARAF_DATA # Karaf data folder
23:13:37  # export KARAF_BASE # Karaf base folder
23:13:37  # export KARAF_ETC  # Karaf etc  folder
23:13:37  # export KARAF_LOG  # Karaf log  folder
23:13:37  # export KARAF_SYSTEM_OPTS # First citizen Karaf options
23:13:37  # export KARAF_OPTS # Additional available Karaf options
23:13:37  # export KARAF_DEBUG # Enable debug mode
23:13:37  # export KARAF_REDIRECT # Enable/set the std/err redirection when using bin/start
23:13:37  # export KARAF_NOROOT # Prevent execution as root if set to true
23:13:37  Set Java version
23:13:37  + echo 'Set Java version'
23:13:37  + sudo /usr/sbin/alternatives --install /usr/bin/java java /usr/lib/jvm/java-21-openjdk-amd64/bin/java 1
23:13:37  sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper
23:13:37  sudo: a password is required
23:13:37  + sudo /usr/sbin/alternatives --set java /usr/lib/jvm/java-21-openjdk-amd64/bin/java
23:13:37  sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper
23:13:37  sudo: a password is required
23:13:37  JDK default version ...
23:13:37  + echo 'JDK default version ...'
23:13:37  + java -version
23:13:37  openjdk version "21.0.5" 2024-10-15
23:13:37  OpenJDK Runtime Environment (build 21.0.5+11-Ubuntu-1ubuntu122.04)
23:13:37  OpenJDK 64-Bit Server VM (build 21.0.5+11-Ubuntu-1ubuntu122.04, mixed mode, sharing)
23:13:37  Set JAVA_HOME
23:13:37  + echo 'Set JAVA_HOME'
23:13:37  + export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
23:13:37  + JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
23:13:37  ++ readlink -e /usr/lib/jvm/java-21-openjdk-amd64/bin/java
23:13:37  Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-21-openjdk-amd64/bin/java
23:13:37  + JAVA_RESOLVED=/usr/lib/jvm/java-21-openjdk-amd64/bin/java
23:13:37  + echo 'Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-21-openjdk-amd64/bin/java'
23:13:37  Listing all open ports on controller system...
23:13:37  + echo 'Listing all open ports on controller system...'
23:13:37  + netstat -pnatu
23:13:37  /tmp/configuration-script.sh: line 40: netstat: command not found
23:13:37  + '[' -f /tmp/custom_shard_config.txt ']'
23:13:37  Configuring cluster
23:13:37  + echo 'Configuring cluster'
23:13:37  + /tmp/karaf-0.21.3-SNAPSHOT/bin/configure_cluster.sh 2 10.30.170.232 10.30.170.47 10.30.170.110
23:13:37  ################################################
23:13:37  ##             Configure Cluster              ##
23:13:37  ################################################
23:13:37   NOTE: Cluster configuration files not found. Copying from
23:13:37   /tmp/karaf-0.21.3-SNAPSHOT/system/org/opendaylight/controller/sal-clustering-config/10.0.11
23:13:37  Configuring unique name in akka.conf
23:13:37  Configuring hostname in akka.conf
23:13:37  Configuring data and rpc seed nodes in akka.conf
23:13:37  modules = [
23:13:37  	
23:13:37  	{
23:13:37  		name = "inventory"
23:13:37  		namespace = "urn:opendaylight:inventory"
23:13:37  		shard-strategy = "module"
23:13:37  	},
23:13:37  	{
23:13:37  		name = "topology"
23:13:37  		namespace = "urn:TBD:params:xml:ns:yang:network-topology"
23:13:37  		shard-strategy = "module"
23:13:37  	},
23:13:37  	{
23:13:37  		name = "toaster"
23:13:37  		namespace = "http://netconfcentral.org/ns/toaster"
23:13:37  		shard-strategy = "module"
23:13:37  	}
23:13:37  ]
23:13:37  Configuring replication type in module-shards.conf
23:13:37  ################################################
23:13:37  ##   NOTE: Manually restart controller to     ##
23:13:37  ##         apply configuration.               ##
23:13:37  ################################################
23:13:37  Dump akka.conf
23:13:37  + echo 'Dump akka.conf'
23:13:37  + cat /tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/akka.conf
23:13:37  
23:13:37  odl-cluster-data {
23:13:37    akka {
23:13:37      remote {
23:13:37        artery {
23:13:37          enabled = on
23:13:37          transport = tcp
23:13:37          canonical.hostname = "10.30.170.47"
23:13:37          canonical.port = 2550
23:13:37        }
23:13:37      }
23:13:37  
23:13:37      cluster {
23:13:37        # Using artery.
23:13:37        seed-nodes = ["akka://opendaylight-cluster-data@10.30.170.232:2550",
23:13:37  				"akka://opendaylight-cluster-data@10.30.170.47:2550",
23:13:37  				"akka://opendaylight-cluster-data@10.30.170.110:2550"]
23:13:37  
23:13:37        roles = ["member-2"]
23:13:37  
23:13:37        # when under load we might trip a false positive on the failure detector
23:13:37        # failure-detector {
23:13:37          # heartbeat-interval = 4 s
23:13:37          # acceptable-heartbeat-pause = 16s
23:13:37        # }
23:13:37      }
23:13:37  
23:13:37      persistence {
23:13:37        # By default the snapshots/journal directories live in KARAF_HOME. You can choose to put it somewhere else by
23:13:37        # modifying the following two properties. The directory location specified may be a relative or absolute path. 
23:13:37        # The relative path is always relative to KARAF_HOME.
23:13:37  
23:13:37        # snapshot-store.local.dir = "target/snapshots"
23:13:37  
23:13:37        # Use lz4 compression for LocalSnapshotStore snapshots
23:13:37        snapshot-store.local.use-lz4-compression = false
23:13:37        # Size of blocks for lz4 compression: 64KB, 256KB, 1MB or 4MB
23:13:37        snapshot-store.local.lz4-blocksize = 256KB
23:13:37      }
23:13:37      disable-default-actor-system-quarantined-event-handling = "false"
23:13:37    }
23:13:37  }
23:13:37  Dump modules.conf
23:13:37  + echo 'Dump modules.conf'
23:13:37  + cat /tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/modules.conf
23:13:37  modules = [
23:13:37  	
23:13:37  	{
23:13:37  		name = "inventory"
23:13:37  		namespace = "urn:opendaylight:inventory"
23:13:37  		shard-strategy = "module"
23:13:37  	},
23:13:37  	{
23:13:37  		name = "topology"
23:13:37  		namespace = "urn:TBD:params:xml:ns:yang:network-topology"
23:13:37  		shard-strategy = "module"
23:13:37  	},
23:13:37  	{
23:13:37  		name = "toaster"
23:13:37  		namespace = "http://netconfcentral.org/ns/toaster"
23:13:37  		shard-strategy = "module"
23:13:37  	}
23:13:37  ]
23:13:37  Dump module-shards.conf
23:13:37  + echo 'Dump module-shards.conf'
23:13:37  + cat /tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/module-shards.conf
23:13:37  module-shards = [
23:13:37  	{
23:13:37  		name = "default"
23:13:37  		shards = [
23:13:37  			{
23:13:37  				name = "default"
23:13:37  				replicas = ["member-1",
23:13:37  				"member-2",
23:13:37  				"member-3"]
23:13:37  			}
23:13:37  		]
23:13:37  	},
23:13:37  	{
23:13:37  		name = "inventory"
23:13:37  		shards = [
23:13:37  			{
23:13:37  				name="inventory"
23:13:37  				replicas = ["member-1",
23:13:37  				"member-2",
23:13:37  				"member-3"]
23:13:37  			}
23:13:37  		]
23:13:37  	},
23:13:37  	{
23:13:37  		name = "topology"
23:13:37  		shards = [
23:13:37  			{
23:13:37  				name="topology"
23:13:37  				replicas = ["member-1",
23:13:37  				"member-2",
23:13:37  				"member-3"]
23:13:37  			}
23:13:37  		]
23:13:37  	},
23:13:37  	{
23:13:37  		name = "toaster"
23:13:37  		shards = [
23:13:37  			{
23:13:37  				name="toaster"
23:13:37  				replicas = ["member-1",
23:13:37  				"member-2",
23:13:37  				"member-3"]
23:13:37  			}
23:13:37  		]
23:13:37  	}
23:13:37  ]
23:13:37  Configuring member-3 with IP address 10.30.170.110
23:13:37  Warning: Permanently added '10.30.170.110' (ECDSA) to the list of known hosts.
23:13:38  Warning: Permanently added '10.30.170.110' (ECDSA) to the list of known hosts.
23:13:38  + source /tmp/common-functions.sh karaf-0.21.3-SNAPSHOT scandium
23:13:38  ++ [[ /tmp/common-functions.sh == \/\t\m\p\/\c\o\n\f\i\g\u\r\a\t\i\o\n\-\s\c\r\i\p\t\.\s\h ]]
23:13:38  common-functions.sh is being sourced
23:13:38  ++ echo 'common-functions.sh is being sourced'
23:13:38  ++ BUNDLEFOLDER=karaf-0.21.3-SNAPSHOT
23:13:38  ++ DISTROSTREAM=scandium
23:13:38  ++ export MAVENCONF=/tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:13:38  ++ MAVENCONF=/tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:13:38  ++ export FEATURESCONF=/tmp/karaf-0.21.3-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:13:38  ++ FEATURESCONF=/tmp/karaf-0.21.3-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:13:38  ++ export CUSTOMPROP=/tmp/karaf-0.21.3-SNAPSHOT/etc/custom.properties
23:13:38  ++ CUSTOMPROP=/tmp/karaf-0.21.3-SNAPSHOT/etc/custom.properties
23:13:38  ++ export LOGCONF=/tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:13:38  ++ LOGCONF=/tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:13:38  ++ export MEMCONF=/tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:38  ++ MEMCONF=/tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:38  ++ export CONTROLLERMEM=
23:13:38  ++ CONTROLLERMEM=
23:13:38  ++ case "${DISTROSTREAM}" in
23:13:38  ++ CLUSTER_SYSTEM=akka
23:13:38  ++ export AKKACONF=/tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/akka.conf
23:13:38  ++ AKKACONF=/tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/akka.conf
23:13:38  ++ export MODULESCONF=/tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/modules.conf
23:13:38  ++ MODULESCONF=/tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/modules.conf
23:13:38  ++ export MODULESHARDSCONF=/tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/module-shards.conf
23:13:38  ++ MODULESHARDSCONF=/tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/module-shards.conf
23:13:38  ++ print_common_env
23:13:38  ++ cat
23:13:38  common-functions environment:
23:13:38  MAVENCONF: /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:13:38  ACTUALFEATURES: 
23:13:38  FEATURESCONF: /tmp/karaf-0.21.3-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:13:38  CUSTOMPROP: /tmp/karaf-0.21.3-SNAPSHOT/etc/custom.properties
23:13:38  LOGCONF: /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:13:38  MEMCONF: /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:38  CONTROLLERMEM: 
23:13:38  AKKACONF: /tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/akka.conf
23:13:38  MODULESCONF: /tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/modules.conf
23:13:38  MODULESHARDSCONF: /tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/module-shards.conf
23:13:38  SUITES: 
23:13:38  
23:13:38  ++ SSH='ssh -t -t'
23:13:38  ++ extra_services_cntl='     dnsmasq.service     httpd.service     libvirtd.service     openvswitch.service     ovs-vswitchd.service     ovsdb-server.service     rabbitmq-server.service '
23:13:38  ++ extra_services_cmp='     libvirtd.service     openvswitch.service     ovs-vswitchd.service     ovsdb-server.service '
23:13:38  Changing to /tmp
23:13:38  Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.3-SNAPSHOT/karaf-0.21.3-20250816.212524-110.zip
23:13:38  + echo 'Changing to /tmp'
23:13:38  + cd /tmp
23:13:38  + echo 'Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.3-SNAPSHOT/karaf-0.21.3-20250816.212524-110.zip'
23:13:38  + wget --progress=dot:mega https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.3-SNAPSHOT/karaf-0.21.3-20250816.212524-110.zip
23:13:38  --2025-08-16 23:13:38--  https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.3-SNAPSHOT/karaf-0.21.3-20250816.212524-110.zip
23:13:38  Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6
23:13:38  Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected.
23:13:38  HTTP request sent, awaiting response... 200 OK
23:13:38  Length: 187828019 (179M) [application/zip]
23:13:38  Saving to: ‘karaf-0.21.3-20250816.212524-110.zip’
23:13:38  
23:13:38       0K ........ ........ ........ ........ ........ ........  1%  117M 2s
23:13:40    3072K ........ ........ ........ ........ ........ ........  3%  107M 2s
23:13:40    6144K ........ ........ ........ ........ ........ ........  5%  191M 1s
23:13:40    9216K ........ ........ ........ ........ ........ ........  6%  203M 1s
23:13:40   12288K ........ ........ ........ ........ ........ ........  8%  230M 1s
23:13:40   15360K ........ ........ ........ ........ ........ ........ 10%  231M 1s
23:13:40   18432K ........ ........ ........ ........ ........ ........ 11%  250M 1s
23:13:40   21504K ........ ........ ........ ........ ........ ........ 13%  250M 1s
23:13:40   24576K ........ ........ ........ ........ ........ ........ 15%  248M 1s
23:13:40   27648K ........ ........ ........ ........ ........ ........ 16%  225M 1s
23:13:40   30720K ........ ........ ........ ........ ........ ........ 18%  288M 1s
23:13:40   33792K ........ ........ ........ ........ ........ ........ 20%  271M 1s
23:13:40   36864K ........ ........ ........ ........ ........ ........ 21%  317M 1s
23:13:40   39936K ........ ........ ........ ........ ........ ........ 23%  332M 1s
23:13:40   43008K ........ ........ ........ ........ ........ ........ 25% 84.3M 1s
23:13:40   46080K ........ ........ ........ ........ ........ ........ 26%  268M 1s
23:13:40   49152K ........ ........ ........ ........ ........ ........ 28%  322M 1s
23:13:40   52224K ........ ........ ........ ........ ........ ........ 30%  321M 1s
23:13:40   55296K ........ ........ ........ ........ ........ ........ 31%  332M 1s
23:13:40   58368K ........ ........ ........ ........ ........ ........ 33%  302M 1s
23:13:40   61440K ........ ........ ........ ........ ........ ........ 35%  325M 1s
23:13:40   64512K ........ ........ ........ ........ ........ ........ 36%  328M 1s
23:13:40   67584K ........ ........ ........ ........ ........ ........ 38%  287M 0s
23:13:40   70656K ........ ........ ........ ........ ........ ........ 40%  307M 0s
23:13:40   73728K ........ ........ ........ ........ ........ ........ 41%  311M 0s
23:13:40   76800K ........ ........ ........ ........ ........ ........ 43%  326M 0s
23:13:40   79872K ........ ........ ........ ........ ........ ........ 45%  346M 0s
23:13:40   82944K ........ ........ ........ ........ ........ ........ 46%  319M 0s
23:13:40   86016K ........ ........ ........ ........ ........ ........ 48%  217M 0s
23:13:40   89088K ........ ........ ........ ........ ........ ........ 50%  193M 0s
23:13:40   92160K ........ ........ ........ ........ ........ ........ 51%  174M 0s
23:13:40   95232K ........ ........ ........ ........ ........ ........ 53%  122M 0s
23:13:40   98304K ........ ........ ........ ........ ........ ........ 55%  137M 0s
23:13:40  101376K ........ ........ ........ ........ ........ ........ 56%  138M 0s
23:13:40  104448K ........ ........ ........ ........ ........ ........ 58%  140M 0s
23:13:40  107520K ........ ........ ........ ........ ........ ........ 60%  174M 0s
23:13:40  110592K ........ ........ ........ ........ ........ ........ 61%  214M 0s
23:13:40  113664K ........ ........ ........ ........ ........ ........ 63%  169M 0s
23:13:40  116736K ........ ........ ........ ........ ........ ........ 65%  159M 0s
23:13:40  119808K ........ ........ ........ ........ ........ ........ 66%  139M 0s
23:13:40  122880K ........ ........ ........ ........ ........ ........ 68%  241M 0s
23:13:40  125952K ........ ........ ........ ........ ........ ........ 70%  182M 0s
23:13:40  129024K ........ ........ ........ ........ ........ ........ 72%  156M 0s
23:13:40  132096K ........ ........ ........ ........ ........ ........ 73%  184M 0s
23:13:40  135168K ........ ........ ........ ........ ........ ........ 75%  235M 0s
23:13:40  138240K ........ ........ ........ ........ ........ ........ 77%  192M 0s
23:13:40  141312K ........ ........ ........ ........ ........ ........ 78%  148M 0s
23:13:40  144384K ........ ........ ........ ........ ........ ........ 80%  128M 0s
23:13:40  147456K ........ ........ ........ ........ ........ ........ 82%  193M 0s
23:13:40  150528K ........ ........ ........ ........ ........ ........ 83% 91.6M 0s
23:13:40  153600K ........ ........ ........ ........ ........ ........ 85%  227M 0s
23:13:40  156672K ........ ........ ........ ........ ........ ........ 87%  269M 0s
23:13:40  159744K ........ ........ ........ ........ ........ ........ 88%  228M 0s
23:13:40  162816K ........ ........ ........ ........ ........ ........ 90%  227M 0s
23:13:40  165888K ........ ........ ........ ........ ........ ........ 92%  231M 0s
23:13:40  168960K ........ ........ ........ ........ ........ ........ 93%  274M 0s
23:13:40  172032K ........ ........ ........ ........ ........ ........ 95%  258M 0s
23:13:40  175104K ........ ........ ........ ........ ........ ........ 97%  278M 0s
23:13:40  178176K ........ ........ ........ ........ ........ ........ 98%  250M 0s
23:13:40  181248K ........ ........ ........ ........ ..               100%  252M=0.9s
23:13:40  
23:13:40  2025-08-16 23:13:39 (202 MB/s) - ‘karaf-0.21.3-20250816.212524-110.zip’ saved [187828019/187828019]
23:13:40  
23:13:40  Extracting the new controller...
23:13:40  + echo 'Extracting the new controller...'
23:13:40  + unzip -q karaf-0.21.3-20250816.212524-110.zip
23:13:40  Adding external repositories...
23:13:40  + echo 'Adding external repositories...'
23:13:40  + sed -ie 's%org.ops4j.pax.url.mvn.repositories=%org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases%g' /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:13:40  + cat /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:13:40  ################################################################################
23:13:40  #
23:13:40  #    Licensed to the Apache Software Foundation (ASF) under one or more
23:13:40  #    contributor license agreements.  See the NOTICE file distributed with
23:13:40  #    this work for additional information regarding copyright ownership.
23:13:40  #    The ASF licenses this file to You under the Apache License, Version 2.0
23:13:40  #    (the "License"); you may not use this file except in compliance with
23:13:40  #    the License.  You may obtain a copy of the License at
23:13:40  #
23:13:40  #       http://www.apache.org/licenses/LICENSE-2.0
23:13:40  #
23:13:40  #    Unless required by applicable law or agreed to in writing, software
23:13:40  #    distributed under the License is distributed on an "AS IS" BASIS,
23:13:40  #    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23:13:40  #    See the License for the specific language governing permissions and
23:13:40  #    limitations under the License.
23:13:40  #
23:13:40  ################################################################################
23:13:40  
23:13:40  #
23:13:40  # If set to true, the following property will not allow any certificate to be used
23:13:40  # when accessing Maven repositories through SSL
23:13:40  #
23:13:40  #org.ops4j.pax.url.mvn.certificateCheck=
23:13:40  
23:13:40  #
23:13:40  # Path to the local Maven settings file.
23:13:40  # The repositories defined in this file will be automatically added to the list
23:13:40  # of default repositories if the 'org.ops4j.pax.url.mvn.repositories' property
23:13:40  # below is not set.
23:13:40  # The following locations are checked for the existence of the settings.xml file
23:13:40  #   * 1. looks for the specified url
23:13:40  #   * 2. if not found looks for ${user.home}/.m2/settings.xml
23:13:40  #   * 3. if not found looks for ${maven.home}/conf/settings.xml
23:13:40  #   * 4. if not found looks for ${M2_HOME}/conf/settings.xml
23:13:40  #
23:13:40  #org.ops4j.pax.url.mvn.settings=
23:13:40  
23:13:40  #
23:13:40  # Path to the local Maven repository which is used to avoid downloading
23:13:40  # artifacts when they already exist locally.
23:13:40  # The value of this property will be extracted from the settings.xml file
23:13:40  # above, or defaulted to:
23:13:40  #     System.getProperty( "user.home" ) + "/.m2/repository"
23:13:40  #
23:13:40  org.ops4j.pax.url.mvn.localRepository=${karaf.home}/${karaf.default.repository}
23:13:40  
23:13:40  #
23:13:40  # Default this to false. It's just weird to use undocumented repos
23:13:40  #
23:13:40  org.ops4j.pax.url.mvn.useFallbackRepositories=false
23:13:40  
23:13:40  #
23:13:40  # Uncomment if you don't wanna use the proxy settings
23:13:40  # from the Maven conf/settings.xml file
23:13:40  #
23:13:40  # org.ops4j.pax.url.mvn.proxySupport=false
23:13:40  
23:13:40  #
23:13:40  # Comma separated list of repositories scanned when resolving an artifact.
23:13:40  # Those repositories will be checked before iterating through the
23:13:40  #    below list of repositories and even before the local repository
23:13:40  # A repository url can be appended with zero or more of the following flags:
23:13:40  #    @snapshots  : the repository contains snaphots
23:13:40  #    @noreleases : the repository does not contain any released artifacts
23:13:40  #
23:13:40  # The following property value will add the system folder as a repo.
23:13:40  #
23:13:40  org.ops4j.pax.url.mvn.defaultRepositories=\
23:13:40      file:${karaf.home}/${karaf.default.repository}@id=system.repository@snapshots,\
23:13:40      file:${karaf.data}/kar@id=kar.repository@multi@snapshots,\
23:13:40      file:${karaf.base}/${karaf.default.repository}@id=child.system.repository@snapshots
23:13:40  
23:13:40  # Use the default local repo (e.g.~/.m2/repository) as a "remote" repo
23:13:40  #org.ops4j.pax.url.mvn.defaultLocalRepoAsRemote=false
23:13:40  
23:13:40  #
23:13:40  # Comma separated list of repositories scanned when resolving an artifact.
23:13:40  # The default list includes the following repositories:
23:13:40  #    http://repo1.maven.org/maven2@id=central
23:13:40  #    http://repository.springsource.com/maven/bundles/release@id=spring.ebr
23:13:40  #    http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external
23:13:40  #    http://zodiac.springsource.com/maven/bundles/release@id=gemini
23:13:40  #    http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases
23:13:40  #    https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases
23:13:40  #    https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases
23:13:40  # To add repositories to the default ones, prepend '+' to the list of repositories
23:13:40  # to add.
23:13:40  # A repository url can be appended with zero or more of the following flags:
23:13:40  #    @snapshots  : the repository contains snapshots
23:13:40  #    @noreleases : the repository does not contain any released artifacts
23:13:40  #    @id=repository.id : the id for the repository, just like in the settings.xml this is optional but recommended
23:13:40  #
23:13:40  org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases
23:13:40  
23:13:40  ### ^^^ No remote repositories. This is the only ODL change compared to Karaf defaults.Configuring the startup features...
23:13:40  + [[ True == \T\r\u\e ]]
23:13:40  + echo 'Configuring the startup features...'
23:13:40  + sed -ie 's/\(featuresBoot=\|featuresBoot =\)/featuresBoot = odl-infrautils-ready,odl-jolokia,odl-restconf-noauth,odl-clustering-test-app,/g' /tmp/karaf-0.21.3-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:13:40  + FEATURE_TEST_STRING=features-test
23:13:40  + FEATURE_TEST_VERSION=0.21.3-SNAPSHOT
23:13:40  + KARAF_VERSION=karaf4
23:13:40  + [[ integration == \i\n\t\e\g\r\a\t\i\o\n ]]
23:13:40  + sed -ie 's%\(featuresRepositories=\|featuresRepositories =\)%featuresRepositories = mvn:org.opendaylight.integration/features-test/0.21.3-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features,%g' /tmp/karaf-0.21.3-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:13:40  + [[ ! -z '' ]]
23:13:40  + cat /tmp/karaf-0.21.3-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:13:40  ################################################################################
23:13:40  #
23:13:40  #    Licensed to the Apache Software Foundation (ASF) under one or more
23:13:40  #    contributor license agreements.  See the NOTICE file distributed with
23:13:40  #    this work for additional information regarding copyright ownership.
23:13:40  #    The ASF licenses this file to You under the Apache License, Version 2.0
23:13:40  #    (the "License"); you may not use this file except in compliance with
23:13:40  #    the License.  You may obtain a copy of the License at
23:13:40  #
23:13:40  #       http://www.apache.org/licenses/LICENSE-2.0
23:13:40  #
23:13:40  #    Unless required by applicable law or agreed to in writing, software
23:13:40  #    distributed under the License is distributed on an "AS IS" BASIS,
23:13:40  #    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23:13:40  #    See the License for the specific language governing permissions and
23:13:40  #    limitations under the License.
23:13:40  #
23:13:40  ################################################################################
23:13:40  
23:13:40  #
23:13:40  # Comma separated list of features repositories to register by default
23:13:40  #
23:13:40  featuresRepositories = mvn:org.opendaylight.integration/features-test/0.21.3-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features, file:${karaf.etc}/09f88248-c0ed-4aad-82be-fc3ff480fbea.xml
23:13:40  
23:13:40  #
23:13:40  # Comma separated list of features to install at startup
23:13:40  #
23:13:40  featuresBoot = odl-infrautils-ready,odl-jolokia,odl-restconf-noauth,odl-clustering-test-app, 6edfc6d1-cf52-4045-8ea5-0bba3a9055c2
23:13:40  
23:13:40  #
23:13:40  # Resource repositories (OBR) that the features resolver can use
23:13:40  # to resolve requirements/capabilities
23:13:40  #
23:13:40  # The format of the resourceRepositories is 
23:13:40  # resourceRepositories=[xml:url|json:url],...
23:13:40  # for Instance:
23:13:40  #
23:13:40  #resourceRepositories=xml:http://host/path/to/index.xml
23:13:40  # or
23:13:40  #resourceRepositories=json:http://host/path/to/index.json
23:13:40  #
23:13:40  
23:13:40  #
23:13:40  # Defines if the boot features are started in asynchronous mode (in a dedicated thread)
23:13:40  #
23:13:40  featuresBootAsynchronous=false
23:13:40  
23:13:40  #
23:13:40  # Service requirements enforcement
23:13:40  #
23:13:40  # By default, the feature resolver checks the service requirements/capabilities of
23:13:40  # bundles for new features (xml schema >= 1.3.0) in order to automatically installs
23:13:40  # the required bundles.
23:13:40  # The following flag can have those values:
23:13:40  #   - disable: service requirements are completely ignored
23:13:40  #   - default: service requirements are ignored for old features
23:13:40  #   - enforce: service requirements are always verified
23:13:40  #
23:13:40  #serviceRequirements=default
23:13:40  
23:13:40  #
23:13:40  # Store cfg file for config element in feature
23:13:40  #
23:13:40  #configCfgStore=true
23:13:40  
23:13:40  #
23:13:40  # Define if the feature service automatically refresh bundles
23:13:40  #
23:13:40  autoRefresh=true
23:13:40  
23:13:40  #
23:13:40  # Configuration of features processing mechanism (overrides, blacklisting, modification of features)
23:13:40  # XML file defines instructions related to features processing
23:13:40  # versions.properties may declare properties to resolve placeholders in XML file
23:13:40  # both files are relative to ${karaf.etc}
23:13:40  #
23:13:40  #featureProcessing=org.apache.karaf.features.xml
23:13:40  #featureProcessingVersions=versions.properties
23:13:40  + configure_karaf_log karaf4 ''
23:13:40  + local -r karaf_version=karaf4
23:13:40  + local -r controllerdebugmap=
23:13:40  + local logapi=log4j
23:13:40  + grep log4j2 /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:13:40  log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
23:13:40  log4j2.rootLogger.level = INFO
23:13:40  #log4j2.rootLogger.type = asyncRoot
23:13:40  #log4j2.rootLogger.includeLocation = false
23:13:40  log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile
23:13:40  log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi
23:13:40  log4j2.rootLogger.appenderRef.Console.ref = Console
23:13:40  log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter
23:13:40  log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF}
23:13:40  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter
23:13:40  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair
23:13:40  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker
23:13:40  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL
23:13:40  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or
23:13:40  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY
23:13:40  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL
23:13:40  log4j2.logger.spifly.name = org.apache.aries.spifly
23:13:40  log4j2.logger.spifly.level = WARN
23:13:40  log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit
23:13:40  log4j2.logger.audit.level = INFO
23:13:40  log4j2.logger.audit.additivity = false
23:13:40  log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile
23:13:40  # Console appender not used by default (see log4j2.rootLogger.appenderRefs)
23:13:40  log4j2.appender.console.type = Console
23:13:40  log4j2.appender.console.name = Console
23:13:40  log4j2.appender.console.layout.type = PatternLayout
23:13:40  log4j2.appender.console.layout.pattern = ${log4j2.pattern}
23:13:40  log4j2.appender.rolling.type = RollingRandomAccessFile
23:13:40  log4j2.appender.rolling.name = RollingFile
23:13:40  log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
23:13:40  log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i
23:13:40  #log4j2.appender.rolling.immediateFlush = false
23:13:40  log4j2.appender.rolling.append = true
23:13:40  log4j2.appender.rolling.layout.type = PatternLayout
23:13:40  log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
23:13:40  log4j2.appender.rolling.policies.type = Policies
23:13:40  log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
23:13:40  log4j2.appender.rolling.policies.size.size = 64MB
23:13:40  log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
23:13:40  log4j2.appender.rolling.strategy.max = 7
23:13:40  log4j2.appender.audit.type = RollingRandomAccessFile
23:13:40  log4j2.appender.audit.name = AuditRollingFile
23:13:40  log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log
23:13:40  log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i
23:13:40  log4j2.appender.audit.append = true
23:13:40  log4j2.appender.audit.layout.type = PatternLayout
23:13:40  log4j2.appender.audit.layout.pattern = ${log4j2.pattern}
23:13:40  log4j2.appender.audit.policies.type = Policies
23:13:40  log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy
23:13:40  log4j2.appender.audit.policies.size.size = 8MB
23:13:40  log4j2.appender.audit.strategy.type = DefaultRolloverStrategy
23:13:40  log4j2.appender.audit.strategy.max = 7
23:13:40  log4j2.appender.osgi.type = PaxOsgi
23:13:40  log4j2.appender.osgi.name = PaxOsgi
23:13:40  log4j2.appender.osgi.filter = *
23:13:40  #log4j2.logger.aether.name = shaded.org.eclipse.aether
23:13:40  #log4j2.logger.aether.level = TRACE
23:13:40  #log4j2.logger.http-headers.name = shaded.org.apache.http.headers
23:13:40  #log4j2.logger.http-headers.level = DEBUG
23:13:40  #log4j2.logger.maven.name = org.ops4j.pax.url.mvn
23:13:40  #log4j2.logger.maven.level = TRACE
23:13:40  + logapi=log4j2
23:13:40  Configuring the karaf log... karaf_version: karaf4, logapi: log4j2
23:13:40  + echo 'Configuring the karaf log... karaf_version: karaf4, logapi: log4j2'
23:13:40  + '[' log4j2 == log4j2 ']'
23:13:40  + sed -ie 's/log4j2.appender.rolling.policies.size.size = 64MB/log4j2.appender.rolling.policies.size.size = 1GB/g' /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:13:40  + orgmodule=org.opendaylight.yangtools.yang.parser.repo.YangTextSchemaContextResolver
23:13:40  + orgmodule_=org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver
23:13:40  + echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN'
23:13:40  + echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN'
23:13:40  + unset IFS
23:13:40  controllerdebugmap: 
23:13:40  + echo 'controllerdebugmap: '
23:13:40  cat /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:13:40  + '[' -n '' ']'
23:13:40  + echo 'cat /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg'
23:13:40  + cat /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:13:40  ################################################################################
23:13:40  #
23:13:40  #    Licensed to the Apache Software Foundation (ASF) under one or more
23:13:40  #    contributor license agreements.  See the NOTICE file distributed with
23:13:40  #    this work for additional information regarding copyright ownership.
23:13:40  #    The ASF licenses this file to You under the Apache License, Version 2.0
23:13:40  #    (the "License"); you may not use this file except in compliance with
23:13:40  #    the License.  You may obtain a copy of the License at
23:13:40  #
23:13:40  #       http://www.apache.org/licenses/LICENSE-2.0
23:13:40  #
23:13:40  #    Unless required by applicable law or agreed to in writing, software
23:13:40  #    distributed under the License is distributed on an "AS IS" BASIS,
23:13:40  #    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23:13:40  #    See the License for the specific language governing permissions and
23:13:40  #    limitations under the License.
23:13:40  #
23:13:40  ################################################################################
23:13:40  
23:13:40  # Common pattern layout for appenders
23:13:40  log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
23:13:40  
23:13:40  # Root logger
23:13:40  log4j2.rootLogger.level = INFO
23:13:40  # uncomment to use asynchronous loggers, which require mvn:com.lmax/disruptor/3.3.2 library
23:13:40  #log4j2.rootLogger.type = asyncRoot
23:13:40  #log4j2.rootLogger.includeLocation = false
23:13:40  log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile
23:13:40  log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi
23:13:40  log4j2.rootLogger.appenderRef.Console.ref = Console
23:13:40  log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter
23:13:40  log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF}
23:13:40  
23:13:40  # Filters for logs marked by org.opendaylight.odlparent.Markers
23:13:40  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter
23:13:40  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair
23:13:40  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker
23:13:40  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL
23:13:40  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or
23:13:40  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY
23:13:40  log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL
23:13:40  
23:13:40  # Loggers configuration
23:13:40  
23:13:40  # Spifly logger
23:13:40  log4j2.logger.spifly.name = org.apache.aries.spifly
23:13:40  log4j2.logger.spifly.level = WARN
23:13:40  
23:13:40  # Security audit logger
23:13:40  log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit
23:13:40  log4j2.logger.audit.level = INFO
23:13:40  log4j2.logger.audit.additivity = false
23:13:40  log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile
23:13:40  
23:13:40  # Appenders configuration
23:13:40  
23:13:40  # Console appender not used by default (see log4j2.rootLogger.appenderRefs)
23:13:40  log4j2.appender.console.type = Console
23:13:40  log4j2.appender.console.name = Console
23:13:40  log4j2.appender.console.layout.type = PatternLayout
23:13:40  log4j2.appender.console.layout.pattern = ${log4j2.pattern}
23:13:40  
23:13:40  # Rolling file appender
23:13:40  log4j2.appender.rolling.type = RollingRandomAccessFile
23:13:40  log4j2.appender.rolling.name = RollingFile
23:13:40  log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
23:13:40  log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i
23:13:40  # uncomment to not force a disk flush
23:13:40  #log4j2.appender.rolling.immediateFlush = false
23:13:40  log4j2.appender.rolling.append = true
23:13:40  log4j2.appender.rolling.layout.type = PatternLayout
23:13:40  log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
23:13:40  log4j2.appender.rolling.policies.type = Policies
23:13:40  log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
23:13:40  log4j2.appender.rolling.policies.size.size = 1GB
23:13:40  log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
23:13:40  log4j2.appender.rolling.strategy.max = 7
23:13:40  
23:13:40  # Audit file appender
23:13:40  log4j2.appender.audit.type = RollingRandomAccessFile
23:13:40  log4j2.appender.audit.name = AuditRollingFile
23:13:40  log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log
23:13:40  log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i
23:13:40  log4j2.appender.audit.append = true
23:13:40  log4j2.appender.audit.layout.type = PatternLayout
23:13:40  log4j2.appender.audit.layout.pattern = ${log4j2.pattern}
23:13:40  log4j2.appender.audit.policies.type = Policies
23:13:40  log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy
23:13:40  log4j2.appender.audit.policies.size.size = 8MB
23:13:40  log4j2.appender.audit.strategy.type = DefaultRolloverStrategy
23:13:40  log4j2.appender.audit.strategy.max = 7
23:13:40  
23:13:40  # OSGi appender
23:13:40  log4j2.appender.osgi.type = PaxOsgi
23:13:40  log4j2.appender.osgi.name = PaxOsgi
23:13:40  log4j2.appender.osgi.filter = *
23:13:40  
23:13:40  # help with identification of maven-related problems with pax-url-aether
23:13:40  #log4j2.logger.aether.name = shaded.org.eclipse.aether
23:13:40  #log4j2.logger.aether.level = TRACE
23:13:40  #log4j2.logger.http-headers.name = shaded.org.apache.http.headers
23:13:40  #log4j2.logger.http-headers.level = DEBUG
23:13:40  #log4j2.logger.maven.name = org.ops4j.pax.url.mvn
23:13:40  #log4j2.logger.maven.level = TRACE
23:13:40  log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN
23:13:40  log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN
23:13:40  Configure
23:13:40      java home: /usr/lib/jvm/java-21-openjdk-amd64
23:13:40  + set_java_vars /usr/lib/jvm/java-21-openjdk-amd64 2048m /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:40  + local -r java_home=/usr/lib/jvm/java-21-openjdk-amd64
23:13:40  + local -r controllermem=2048m
23:13:40  + local -r memconf=/tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:40  + echo Configure
23:13:40  + echo '    java home: /usr/lib/jvm/java-21-openjdk-amd64'
23:13:40  + echo '    max memory: 2048m'
23:13:40      max memory: 2048m
23:13:40      memconf: /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:40  + echo '    memconf: /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv'
23:13:40  + sed -ie 's%^# export JAVA_HOME%export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-21-openjdk-amd64}%g' /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:40  + sed -ie 's/JAVA_MAX_MEM="2048m"/JAVA_MAX_MEM=2048m/g' /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:40  cat /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:40  + echo 'cat /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv'
23:13:40  + cat /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:40  #!/bin/sh
23:13:40  #
23:13:40  #    Licensed to the Apache Software Foundation (ASF) under one or more
23:13:40  #    contributor license agreements.  See the NOTICE file distributed with
23:13:40  #    this work for additional information regarding copyright ownership.
23:13:40  #    The ASF licenses this file to You under the Apache License, Version 2.0
23:13:40  #    (the "License"); you may not use this file except in compliance with
23:13:40  #    the License.  You may obtain a copy of the License at
23:13:40  #
23:13:40  #       http://www.apache.org/licenses/LICENSE-2.0
23:13:40  #
23:13:40  #    Unless required by applicable law or agreed to in writing, software
23:13:40  #    distributed under the License is distributed on an "AS IS" BASIS,
23:13:40  #    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
23:13:40  #    See the License for the specific language governing permissions and
23:13:40  #    limitations under the License.
23:13:40  #
23:13:40  
23:13:40  #
23:13:40  # handle specific scripts; the SCRIPT_NAME is exactly the name of the Karaf
23:13:40  # script: client, instance, shell, start, status, stop, karaf
23:13:40  #
23:13:40  # if [ "${KARAF_SCRIPT}" == "SCRIPT_NAME" ]; then
23:13:40  #   Actions go here...
23:13:40  # fi
23:13:40  
23:13:40  #
23:13:40  # general settings which should be applied for all scripts go here; please keep
23:13:40  # in mind that it is possible that scripts might be executed more than once, e.g.
23:13:40  # in example of the start script where the start script is executed first and the
23:13:40  # karaf script afterwards.
23:13:40  #
23:13:40  
23:13:40  #
23:13:40  # The following section shows the possible configuration options for the default 
23:13:40  # karaf scripts
23:13:40  #
23:13:40  export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-21-openjdk-amd64} # Location of Java installation
23:13:40  # export JAVA_OPTS # Generic JVM options, for instance, where you can pass the memory configuration
23:13:40  # export JAVA_NON_DEBUG_OPTS # Additional non-debug JVM options
23:13:40  # export EXTRA_JAVA_OPTS # Additional JVM options
23:13:40  # export KARAF_HOME # Karaf home folder
23:13:40  # export KARAF_DATA # Karaf data folder
23:13:40  # export KARAF_BASE # Karaf base folder
23:13:40  # export KARAF_ETC  # Karaf etc  folder
23:13:40  # export KARAF_LOG  # Karaf log  folder
23:13:40  # export KARAF_SYSTEM_OPTS # First citizen Karaf options
23:13:40  # export KARAF_OPTS # Additional available Karaf options
23:13:40  # export KARAF_DEBUG # Enable debug mode
23:13:40  # export KARAF_REDIRECT # Enable/set the std/err redirection when using bin/start
23:13:40  # export KARAF_NOROOT # Prevent execution as root if set to true
23:13:40  Set Java version
23:13:40  + echo 'Set Java version'
23:13:40  + sudo /usr/sbin/alternatives --install /usr/bin/java java /usr/lib/jvm/java-21-openjdk-amd64/bin/java 1
23:13:41  sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper
23:13:41  sudo: a password is required
23:13:41  + sudo /usr/sbin/alternatives --set java /usr/lib/jvm/java-21-openjdk-amd64/bin/java
23:13:41  sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper
23:13:41  sudo: a password is required
23:13:41  JDK default version ...
23:13:41  + echo 'JDK default version ...'
23:13:41  + java -version
23:13:42  openjdk version "21.0.5" 2024-10-15
23:13:42  OpenJDK Runtime Environment (build 21.0.5+11-Ubuntu-1ubuntu122.04)
23:13:42  OpenJDK 64-Bit Server VM (build 21.0.5+11-Ubuntu-1ubuntu122.04, mixed mode, sharing)
23:13:42  Set JAVA_HOME
23:13:42  + echo 'Set JAVA_HOME'
23:13:42  + export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
23:13:42  + JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64
23:13:42  ++ readlink -e /usr/lib/jvm/java-21-openjdk-amd64/bin/java
23:13:42  Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-21-openjdk-amd64/bin/java
23:13:42  + JAVA_RESOLVED=/usr/lib/jvm/java-21-openjdk-amd64/bin/java
23:13:42  + echo 'Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-21-openjdk-amd64/bin/java'
23:13:42  Listing all open ports on controller system...
23:13:42  + echo 'Listing all open ports on controller system...'
23:13:42  + netstat -pnatu
23:13:42  /tmp/configuration-script.sh: line 40: netstat: command not found
23:13:42  Configuring cluster
23:13:42  + '[' -f /tmp/custom_shard_config.txt ']'
23:13:42  + echo 'Configuring cluster'
23:13:42  + /tmp/karaf-0.21.3-SNAPSHOT/bin/configure_cluster.sh 3 10.30.170.232 10.30.170.47 10.30.170.110
23:13:42  ################################################
23:13:42  ##             Configure Cluster              ##
23:13:42  ################################################
23:13:42   NOTE: Cluster configuration files not found. Copying from
23:13:42   /tmp/karaf-0.21.3-SNAPSHOT/system/org/opendaylight/controller/sal-clustering-config/10.0.11
23:13:42  Configuring unique name in akka.conf
23:13:42  Configuring hostname in akka.conf
23:13:42  Configuring data and rpc seed nodes in akka.conf
23:13:42  modules = [
23:13:42  	
23:13:42  	{
23:13:42  		name = "inventory"
23:13:42  		namespace = "urn:opendaylight:inventory"
23:13:42  		shard-strategy = "module"
23:13:42  	},
23:13:42  	{
23:13:42  		name = "topology"
23:13:42  		namespace = "urn:TBD:params:xml:ns:yang:network-topology"
23:13:42  		shard-strategy = "module"
23:13:42  	},
23:13:42  	{
23:13:42  		name = "toaster"
23:13:42  		namespace = "http://netconfcentral.org/ns/toaster"
23:13:42  		shard-strategy = "module"
23:13:42  	}
23:13:42  ]
23:13:42  Configuring replication type in module-shards.conf
23:13:42  ################################################
23:13:42  ##   NOTE: Manually restart controller to     ##
23:13:42  ##         apply configuration.               ##
23:13:42  ################################################
23:13:42  Dump akka.conf
23:13:42  + echo 'Dump akka.conf'
23:13:42  + cat /tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/akka.conf
23:13:42  
23:13:42  odl-cluster-data {
23:13:42    akka {
23:13:42      remote {
23:13:42        artery {
23:13:42          enabled = on
23:13:42          transport = tcp
23:13:42          canonical.hostname = "10.30.170.110"
23:13:42          canonical.port = 2550
23:13:42        }
23:13:42      }
23:13:42  
23:13:42      cluster {
23:13:42        # Using artery.
23:13:42        seed-nodes = ["akka://opendaylight-cluster-data@10.30.170.232:2550",
23:13:42  				"akka://opendaylight-cluster-data@10.30.170.47:2550",
23:13:42  				"akka://opendaylight-cluster-data@10.30.170.110:2550"]
23:13:42  
23:13:42        roles = ["member-3"]
23:13:42  
23:13:42        # when under load we might trip a false positive on the failure detector
23:13:42        # failure-detector {
23:13:42          # heartbeat-interval = 4 s
23:13:42          # acceptable-heartbeat-pause = 16s
23:13:42        # }
23:13:42      }
23:13:42  
23:13:42      persistence {
23:13:42        # By default the snapshots/journal directories live in KARAF_HOME. You can choose to put it somewhere else by
23:13:42        # modifying the following two properties. The directory location specified may be a relative or absolute path. 
23:13:42        # The relative path is always relative to KARAF_HOME.
23:13:42  
23:13:42        # snapshot-store.local.dir = "target/snapshots"
23:13:42  
23:13:42        # Use lz4 compression for LocalSnapshotStore snapshots
23:13:42        snapshot-store.local.use-lz4-compression = false
23:13:42        # Size of blocks for lz4 compression: 64KB, 256KB, 1MB or 4MB
23:13:42        snapshot-store.local.lz4-blocksize = 256KB
23:13:42      }
23:13:42      disable-default-actor-system-quarantined-event-handling = "false"
23:13:42    }
23:13:42  }
23:13:42  Dump modules.conf
23:13:42  + echo 'Dump modules.conf'
23:13:42  + cat /tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/modules.conf
23:13:42  modules = [
23:13:42  	
23:13:42  	{
23:13:42  		name = "inventory"
23:13:42  		namespace = "urn:opendaylight:inventory"
23:13:42  		shard-strategy = "module"
23:13:42  	},
23:13:42  	{
23:13:42  		name = "topology"
23:13:42  		namespace = "urn:TBD:params:xml:ns:yang:network-topology"
23:13:42  		shard-strategy = "module"
23:13:42  	},
23:13:42  	{
23:13:42  		name = "toaster"
23:13:42  		namespace = "http://netconfcentral.org/ns/toaster"
23:13:42  		shard-strategy = "module"
23:13:42  	}
23:13:42  ]
23:13:42  Dump module-shards.conf
23:13:42  + echo 'Dump module-shards.conf'
23:13:42  + cat /tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/module-shards.conf
23:13:42  module-shards = [
23:13:42  	{
23:13:42  		name = "default"
23:13:42  		shards = [
23:13:42  			{
23:13:42  				name = "default"
23:13:42  				replicas = ["member-1",
23:13:42  				"member-2",
23:13:42  				"member-3"]
23:13:42  			}
23:13:42  		]
23:13:42  	},
23:13:42  	{
23:13:42  		name = "inventory"
23:13:42  		shards = [
23:13:42  			{
23:13:42  				name="inventory"
23:13:42  				replicas = ["member-1",
23:13:42  				"member-2",
23:13:42  				"member-3"]
23:13:42  			}
23:13:42  		]
23:13:42  	},
23:13:42  	{
23:13:42  		name = "topology"
23:13:42  		shards = [
23:13:42  			{
23:13:42  				name="topology"
23:13:42  				replicas = ["member-1",
23:13:42  				"member-2",
23:13:42  				"member-3"]
23:13:42  			}
23:13:42  		]
23:13:42  	},
23:13:42  	{
23:13:42  		name = "toaster"
23:13:42  		shards = [
23:13:42  			{
23:13:42  				name="toaster"
23:13:42  				replicas = ["member-1",
23:13:42  				"member-2",
23:13:42  				"member-3"]
23:13:42  			}
23:13:42  		]
23:13:42  	}
23:13:42  ]
23:13:42  Locating config plan to use...
23:13:42  Finished running config plans
23:13:42  Starting member-1 with IP address 10.30.170.232
23:13:43  Warning: Permanently added '10.30.170.232' (ECDSA) to the list of known hosts.
23:13:43  Warning: Permanently added '10.30.170.232' (ECDSA) to the list of known hosts.
23:13:43  Redirecting karaf console output to karaf_console.log
23:13:43  Starting controller...
23:13:43  start: Redirecting Karaf output to /tmp/karaf-0.21.3-SNAPSHOT/data/log/karaf_console.log
23:13:43  Starting member-2 with IP address 10.30.170.47
23:13:43  Warning: Permanently added '10.30.170.47' (ECDSA) to the list of known hosts.
23:13:44  Warning: Permanently added '10.30.170.47' (ECDSA) to the list of known hosts.
23:13:44  Redirecting karaf console output to karaf_console.log
23:13:44  Starting controller...
23:13:44  start: Redirecting Karaf output to /tmp/karaf-0.21.3-SNAPSHOT/data/log/karaf_console.log
23:13:44  Starting member-3 with IP address 10.30.170.110
23:13:44  Warning: Permanently added '10.30.170.110' (ECDSA) to the list of known hosts.
23:13:44  Warning: Permanently added '10.30.170.110' (ECDSA) to the list of known hosts.
23:13:44  Redirecting karaf console output to karaf_console.log
23:13:44  Starting controller...
23:13:44  start: Redirecting Karaf output to /tmp/karaf-0.21.3-SNAPSHOT/data/log/karaf_console.log
23:13:44  [controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium] $ /bin/bash /tmp/jenkins8049900210795019888.sh
23:13:44  common-functions.sh is being sourced
23:13:44  common-functions environment:
23:13:44  MAVENCONF: /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
23:13:44  ACTUALFEATURES: 
23:13:44  FEATURESCONF: /tmp/karaf-0.21.3-SNAPSHOT/etc/org.apache.karaf.features.cfg
23:13:44  CUSTOMPROP: /tmp/karaf-0.21.3-SNAPSHOT/etc/custom.properties
23:13:44  LOGCONF: /tmp/karaf-0.21.3-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
23:13:44  MEMCONF: /tmp/karaf-0.21.3-SNAPSHOT/bin/setenv
23:13:44  CONTROLLERMEM: 2048m
23:13:44  AKKACONF: /tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/akka.conf
23:13:44  MODULESCONF: /tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/modules.conf
23:13:44  MODULESHARDSCONF: /tmp/karaf-0.21.3-SNAPSHOT/configuration/initial/module-shards.conf
23:13:44  SUITES: 
23:13:44  
23:13:44  + echo '#################################################'
23:13:44  #################################################
23:13:44  + echo '##         Verify Cluster is UP                ##'
23:13:44  ##         Verify Cluster is UP                ##
23:13:44  + echo '#################################################'
23:13:44  #################################################
23:13:44  + create_post_startup_script
23:13:44  + cat
23:13:44  + copy_and_run_post_startup_script
23:13:44  + seed_index=1
23:13:44  ++ seq 1 3
23:13:44  + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
23:13:44  + CONTROLLERIP=ODL_SYSTEM_1_IP
23:13:44  + echo 'Execute the post startup script on controller 10.30.170.232'
23:13:44  Execute the post startup script on controller 10.30.170.232
23:13:44  + scp /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/post-startup-script.sh 10.30.170.232:/tmp/
23:13:44  Warning: Permanently added '10.30.170.232' (ECDSA) to the list of known hosts.
23:13:44  + ssh 10.30.170.232 'bash /tmp/post-startup-script.sh 1'
23:13:44  Warning: Permanently added '10.30.170.232' (ECDSA) to the list of known hosts.
23:13:45  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:13:50  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:13:55  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:14:00  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:14:05  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:14:10  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:14:15  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:14:20  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:14:25  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:14:30  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:14:35  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:14:40  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:14:45  Waiting up to 3 minutes for controller to come up, checking every 5 seconds...
23:14:50  2025-08-16T23:14:16,671 | INFO  | SystemReadyService-0 | SimpleSystemReadyMonitor         | 146 - org.opendaylight.infrautils.ready-api - 7.1.5 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.]
23:14:50  Controller is UP
23:14:50  2025-08-16T23:14:16,671 | INFO  | SystemReadyService-0 | SimpleSystemReadyMonitor         | 146 - org.opendaylight.infrautils.ready-api - 7.1.5 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.]
23:14:50  Listing all open ports on controller system...
23:14:50  /tmp/post-startup-script.sh: line 51: netstat: command not found
23:14:50  looking for "BindException: Address already in use" in log file
23:14:50  looking for "server is unhealthy" in log file
23:14:50  + '[' 1 == 0 ']'
23:14:50  + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
23:14:50  + CONTROLLERIP=ODL_SYSTEM_2_IP
23:14:50  + echo 'Execute the post startup script on controller 10.30.170.47'
23:14:50  Execute the post startup script on controller 10.30.170.47
23:14:50  + scp /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/post-startup-script.sh 10.30.170.47:/tmp/
23:14:50  Warning: Permanently added '10.30.170.47' (ECDSA) to the list of known hosts.
23:14:50  + ssh 10.30.170.47 'bash /tmp/post-startup-script.sh 2'
23:14:50  Warning: Permanently added '10.30.170.47' (ECDSA) to the list of known hosts.
23:14:50  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:14:56  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:15:01  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:15:06  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:15:11  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:15:16  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:15:21  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:15:26  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:15:31  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:15:36  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:15:41  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:15:46  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:15:51  Waiting up to 3 minutes for controller to come up, checking every 5 seconds...
23:15:56  2025-08-16T23:14:16,483 | INFO  | SystemReadyService-0 | SimpleSystemReadyMonitor         | 146 - org.opendaylight.infrautils.ready-api - 7.1.5 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.]
23:15:56  Controller is UP
23:15:56  2025-08-16T23:14:16,483 | INFO  | SystemReadyService-0 | SimpleSystemReadyMonitor         | 146 - org.opendaylight.infrautils.ready-api - 7.1.5 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.]
23:15:56  Listing all open ports on controller system...
23:15:56  /tmp/post-startup-script.sh: line 51: netstat: command not found
23:15:56  looking for "BindException: Address already in use" in log file
23:15:56  looking for "server is unhealthy" in log file
23:15:56  + '[' 2 == 0 ']'
23:15:56  + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
23:15:56  + CONTROLLERIP=ODL_SYSTEM_3_IP
23:15:56  + echo 'Execute the post startup script on controller 10.30.170.110'
23:15:56  Execute the post startup script on controller 10.30.170.110
23:15:56  + scp /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/post-startup-script.sh 10.30.170.110:/tmp/
23:15:56  Warning: Permanently added '10.30.170.110' (ECDSA) to the list of known hosts.
23:15:56  + ssh 10.30.170.110 'bash /tmp/post-startup-script.sh 3'
23:15:56  Warning: Permanently added '10.30.170.110' (ECDSA) to the list of known hosts.
23:15:56  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:16:01  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:16:06  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:16:11  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:16:16  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:16:21  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:16:26  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:16:31  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:16:36  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:16:41  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:16:46  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:16:51  /tmp/post-startup-script.sh: line 4: netstat: command not found
23:16:56  Waiting up to 3 minutes for controller to come up, checking every 5 seconds...
23:17:01  2025-08-16T23:14:15,744 | INFO  | SystemReadyService-0 | SimpleSystemReadyMonitor         | 146 - org.opendaylight.infrautils.ready-api - 7.1.5 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.]
23:17:01  Controller is UP
23:17:01  2025-08-16T23:14:15,744 | INFO  | SystemReadyService-0 | SimpleSystemReadyMonitor         | 146 - org.opendaylight.infrautils.ready-api - 7.1.5 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.]
23:17:01  Listing all open ports on controller system...
23:17:01  /tmp/post-startup-script.sh: line 51: netstat: command not found
23:17:01  looking for "BindException: Address already in use" in log file
23:17:01  looking for "server is unhealthy" in log file
23:17:01  + '[' 0 == 0 ']'
23:17:01  + seed_index=1
23:17:01  + dump_controller_threads
23:17:01  ++ seq 1 3
23:17:01  + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
23:17:01  + CONTROLLERIP=ODL_SYSTEM_1_IP
23:17:01  + echo 'Let'\''s take the karaf thread dump'
23:17:01  Let's take the karaf thread dump
23:17:01  + ssh 10.30.170.232 'sudo ps aux'
23:17:01  Warning: Permanently added '10.30.170.232' (ECDSA) to the list of known hosts.
23:17:02  ++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/ps_before.log
23:17:02  ++ grep -v grep
23:17:02  ++ tr -s ' '
23:17:02  ++ cut -f2 '-d '
23:17:02  + pid=2050
23:17:02  + echo 'karaf main: org.apache.karaf.main.Main, pid:2050'
23:17:02  karaf main: org.apache.karaf.main.Main, pid:2050
23:17:02  + ssh 10.30.170.232 '/usr/lib/jvm/java-21-openjdk-amd64/bin/jstack -l 2050'
23:17:02  Warning: Permanently added '10.30.170.232' (ECDSA) to the list of known hosts.
23:17:03  + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
23:17:03  + CONTROLLERIP=ODL_SYSTEM_2_IP
23:17:03  + echo 'Let'\''s take the karaf thread dump'
23:17:03  Let's take the karaf thread dump
23:17:03  + ssh 10.30.170.47 'sudo ps aux'
23:17:03  Warning: Permanently added '10.30.170.47' (ECDSA) to the list of known hosts.
23:17:03  ++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/ps_before.log
23:17:03  ++ grep -v grep
23:17:03  ++ tr -s ' '
23:17:03  ++ cut -f2 '-d '
23:17:03  + pid=2058
23:17:03  + echo 'karaf main: org.apache.karaf.main.Main, pid:2058'
23:17:03  karaf main: org.apache.karaf.main.Main, pid:2058
23:17:03  + ssh 10.30.170.47 '/usr/lib/jvm/java-21-openjdk-amd64/bin/jstack -l 2058'
23:17:03  Warning: Permanently added '10.30.170.47' (ECDSA) to the list of known hosts.
23:17:04  + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
23:17:04  + CONTROLLERIP=ODL_SYSTEM_3_IP
23:17:04  + echo 'Let'\''s take the karaf thread dump'
23:17:04  Let's take the karaf thread dump
23:17:04  + ssh 10.30.170.110 'sudo ps aux'
23:17:04  Warning: Permanently added '10.30.170.110' (ECDSA) to the list of known hosts.
23:17:04  ++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/ps_before.log
23:17:04  ++ grep -v grep
23:17:04  ++ cut -f2 '-d '
23:17:04  ++ tr -s ' '
23:17:04  + pid=2037
23:17:04  + echo 'karaf main: org.apache.karaf.main.Main, pid:2037'
23:17:04  karaf main: org.apache.karaf.main.Main, pid:2037
23:17:04  + ssh 10.30.170.110 '/usr/lib/jvm/java-21-openjdk-amd64/bin/jstack -l 2037'
23:17:04  Warning: Permanently added '10.30.170.110' (ECDSA) to the list of known hosts.
23:17:05  + '[' 0 -gt 0 ']'
23:17:05  + echo 'Generating controller variables...'
23:17:05  Generating controller variables...
23:17:05  ++ seq 1 3
23:17:05  + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
23:17:05  + CONTROLLERIP=ODL_SYSTEM_1_IP
23:17:05  + odl_variables=' -v ODL_SYSTEM_1_IP:10.30.170.232'
23:17:05  + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
23:17:05  + CONTROLLERIP=ODL_SYSTEM_2_IP
23:17:05  + odl_variables=' -v ODL_SYSTEM_1_IP:10.30.170.232 -v ODL_SYSTEM_2_IP:10.30.170.47'
23:17:05  + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
23:17:05  + CONTROLLERIP=ODL_SYSTEM_3_IP
23:17:05  + odl_variables=' -v ODL_SYSTEM_1_IP:10.30.170.232 -v ODL_SYSTEM_2_IP:10.30.170.47 -v ODL_SYSTEM_3_IP:10.30.170.110'
23:17:05  + echo 'Generating mininet variables...'
23:17:05  Generating mininet variables...
23:17:05  ++ seq 1 1
23:17:05  + for i in $(seq 1 "${NUM_TOOLS_SYSTEM}")
23:17:05  + MININETIP=TOOLS_SYSTEM_1_IP
23:17:05  + tools_variables=' -v TOOLS_SYSTEM_1_IP:10.30.170.56'
23:17:05  + get_test_suites SUITES
23:17:05  + local __suite_list=SUITES
23:17:05  + echo 'Locating test plan to use...'
23:17:05  Locating test plan to use...
23:17:05  + testplan_filepath=/w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/test/csit/testplans/controller-ddb-expl-lead-movement-longevity-scandium.txt
23:17:05  + '[' '!' -f /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/test/csit/testplans/controller-ddb-expl-lead-movement-longevity-scandium.txt ']'
23:17:05  + testplan_filepath=/w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/test/csit/testplans/controller-ddb-expl-lead-movement-longevity.txt
23:17:05  + '[' disabled '!=' disabled ']'
23:17:05  + echo 'Changing the testplan path...'
23:17:05  Changing the testplan path...
23:17:05  + sed s:integration:/w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium: /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/test/csit/testplans/controller-ddb-expl-lead-movement-longevity.txt
23:17:05  + cat testplan.txt
23:17:05  # Copyright (c) 2017 Cisco Systems, Inc. and others. All rights reserved.
23:17:05  #
23:17:05  # This program and the accompanying materials are made available under the
23:17:05  # terms of the Eclipse Public License v1.0 which accompanies this distribution,
23:17:05  # and is available at http://www.eclipse.org/legal/epl-v10.html
23:17:05  
23:17:05  # Place the suites in run order:
23:17:05  /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
23:17:05  /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement_longevity.robot
23:17:05  + '[' -z '' ']'
23:17:05  ++ grep -E -v '(^[[:space:]]*#|^[[:space:]]*$)' testplan.txt
23:17:05  ++ tr '\012' ' '
23:17:05  + suite_list='/w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement_longevity.robot '
23:17:05  + eval 'SUITES='\''/w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement_longevity.robot '\'''
23:17:05  ++ SUITES='/w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement_longevity.robot '
23:17:05  + echo 'Starting Robot test suites /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement_longevity.robot  ...'
23:17:05  Starting Robot test suites /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement_longevity.robot  ...
23:17:05  + robot -N controller-ddb-expl-lead-movement-longevity.txt --removekeywords wuks -e exclude -e skip_if_scandium -v BUNDLEFOLDER:karaf-0.21.3-SNAPSHOT -v BUNDLE_URL:https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.3-SNAPSHOT/karaf-0.21.3-20250816.212524-110.zip -v CONTROLLER:10.30.170.232 -v CONTROLLER1:10.30.170.47 -v CONTROLLER2:10.30.170.110 -v CONTROLLER_USER:jenkins -v JAVA_HOME:/usr/lib/jvm/java-21-openjdk-amd64 -v JDKVERSION:openjdk21 -v JENKINS_WORKSPACE:/w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium -v MININET:10.30.170.56 -v MININET1: -v MININET2: -v MININET_USER:jenkins -v NEXUSURL_PREFIX:https://nexus.opendaylight.org -v NUM_ODL_SYSTEM:3 -v NUM_TOOLS_SYSTEM:1 -v ODL_STREAM:scandium -v ODL_SYSTEM_IP:10.30.170.232 -v ODL_SYSTEM_1_IP:10.30.170.232 -v ODL_SYSTEM_2_IP:10.30.170.47 -v ODL_SYSTEM_3_IP:10.30.170.110 -v ODL_SYSTEM_USER:jenkins -v TOOLS_SYSTEM_IP:10.30.170.56 -v TOOLS_SYSTEM_1_IP:10.30.170.56 -v TOOLS_SYSTEM_USER:jenkins -v USER_HOME:/home/jenkins -v IS_KARAF_APPL:True -v WORKSPACE:/tmp /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement_longevity.robot
23:17:05  ==============================================================================
23:17:05  controller-ddb-expl-lead-movement-longevity.txt                               
23:17:05  ==============================================================================
23:17:06  controller-ddb-expl-lead-movement-longevity.txt.Restart Odl With Tell Based...
23:17:06  ==============================================================================
23:17:10  Stop_All_Members :: Stop every odl node.                              | PASS |
23:17:32  ------------------------------------------------------------------------------
23:17:32  Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS |
23:17:40  ------------------------------------------------------------------------------
23:17:40  Start_All_And_Sync :: Start each member and wait for sync.            | FAIL |
23:23:45  Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.232:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore
23:23:45  ------------------------------------------------------------------------------
23:23:45  controller-ddb-expl-lead-movement-longevity.txt.Restart Odl With T... | FAIL |
23:23:45  3 tests, 2 passed, 1 failed
23:23:45  ==============================================================================
23:23:45  controller-ddb-expl-lead-movement-longevity.txt.Explicit Leader Movement Lo...
23:23:45  ==============================================================================
23:23:50  Explicit_Leader_Movement_Test :: Leader move for 24 hours from one... | FAIL |
23:23:50  ... click for list of related bugs or create a new one if needed (with the
23:23:50  "controller_ddb_expl_lead_movement_longevity_txt_explicit_leader_movement_longevity_explicit_leader_movement_test"
23:23:50  reference somewhere inside)
23:23:50  "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_ddb_expl_lead_movement_longevity_txt_explicit_leader_movement_longevity_explicit_leader_movement_test&order=bug_status"
23:23:50  
23:23:50  HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.232:8181/jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore
23:23:50  ------------------------------------------------------------------------------
23:23:50  controller-ddb-expl-lead-movement-longevity.txt.Explicit Leader Mo... | FAIL |
23:23:50  1 test, 0 passed, 1 failed
23:23:50  ==============================================================================
23:23:50  controller-ddb-expl-lead-movement-longevity.txt                       | FAIL |
23:23:50  4 tests, 2 passed, 2 failed
23:23:50  ==============================================================================
23:23:50  Output:  /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/output.xml
23:23:51  Log:     /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/log.html
23:23:51  Report:  /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/report.html
23:23:51  + true
23:23:51  + echo 'Examining the files in data/log and checking filesize'
23:23:51  Examining the files in data/log and checking filesize
23:23:51  + ssh 10.30.170.232 'ls -altr /tmp/karaf-0.21.3-SNAPSHOT/data/log/'
23:23:51  Warning: Permanently added '10.30.170.232' (ECDSA) to the list of known hosts.
23:23:51  total 556
23:23:51  drwxrwxr-x 2 jenkins jenkins   4096 Aug 16 23:13 .
23:23:51  -rw-rw-r-- 1 jenkins jenkins   1720 Aug 16 23:13 karaf_console.log
23:23:51  drwxrwxr-x 9 jenkins jenkins   4096 Aug 16 23:17 ..
23:23:51  -rw-rw-r-- 1 jenkins jenkins 554303 Aug 16 23:23 karaf.log
23:23:51  + ssh 10.30.170.232 'du -hs /tmp/karaf-0.21.3-SNAPSHOT/data/log/*'
23:23:51  Warning: Permanently added '10.30.170.232' (ECDSA) to the list of known hosts.
23:23:51  544K	/tmp/karaf-0.21.3-SNAPSHOT/data/log/karaf.log
23:23:51  4.0K	/tmp/karaf-0.21.3-SNAPSHOT/data/log/karaf_console.log
23:23:51  + ssh 10.30.170.47 'ls -altr /tmp/karaf-0.21.3-SNAPSHOT/data/log/'
23:23:51  Warning: Permanently added '10.30.170.47' (ECDSA) to the list of known hosts.
23:23:51  total 560
23:23:51  drwxrwxr-x 2 jenkins jenkins   4096 Aug 16 23:13 .
23:23:51  -rw-rw-r-- 1 jenkins jenkins   1720 Aug 16 23:13 karaf_console.log
23:23:51  drwxrwxr-x 9 jenkins jenkins   4096 Aug 16 23:17 ..
23:23:51  -rw-rw-r-- 1 jenkins jenkins 558732 Aug 16 23:23 karaf.log
23:23:51  + ssh 10.30.170.47 'du -hs /tmp/karaf-0.21.3-SNAPSHOT/data/log/*'
23:23:51  Warning: Permanently added '10.30.170.47' (ECDSA) to the list of known hosts.
23:23:52  548K	/tmp/karaf-0.21.3-SNAPSHOT/data/log/karaf.log
23:23:52  4.0K	/tmp/karaf-0.21.3-SNAPSHOT/data/log/karaf_console.log
23:23:52  + ssh 10.30.170.110 'ls -altr /tmp/karaf-0.21.3-SNAPSHOT/data/log/'
23:23:52  Warning: Permanently added '10.30.170.110' (ECDSA) to the list of known hosts.
23:23:52  total 596
23:23:52  drwxrwxr-x 2 jenkins jenkins   4096 Aug 16 23:13 .
23:23:52  -rw-rw-r-- 1 jenkins jenkins   1720 Aug 16 23:13 karaf_console.log
23:23:52  drwxrwxr-x 9 jenkins jenkins   4096 Aug 16 23:17 ..
23:23:52  -rw-rw-r-- 1 jenkins jenkins 596244 Aug 16 23:23 karaf.log
23:23:52  + ssh 10.30.170.110 'du -hs /tmp/karaf-0.21.3-SNAPSHOT/data/log/*'
23:23:52  Warning: Permanently added '10.30.170.110' (ECDSA) to the list of known hosts.
23:23:52  584K	/tmp/karaf-0.21.3-SNAPSHOT/data/log/karaf.log
23:23:52  4.0K	/tmp/karaf-0.21.3-SNAPSHOT/data/log/karaf_console.log
23:23:52  + set +e
23:23:52  ++ seq 1 3
23:23:52  + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
23:23:52  + CONTROLLERIP=ODL_SYSTEM_1_IP
23:23:52  + echo 'Let'\''s take the karaf thread dump again'
23:23:52  Let's take the karaf thread dump again
23:23:52  + ssh 10.30.170.232 'sudo ps aux'
23:23:52  Warning: Permanently added '10.30.170.232' (ECDSA) to the list of known hosts.
23:23:52  ++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/ps_after.log
23:23:52  ++ grep -v grep
23:23:52  ++ tr -s ' '
23:23:52  ++ cut -f2 '-d '
23:23:52  + pid=4624
23:23:52  + echo 'karaf main: org.apache.karaf.main.Main, pid:4624'
23:23:52  karaf main: org.apache.karaf.main.Main, pid:4624
23:23:52  + ssh 10.30.170.232 '/usr/lib/jvm/java-21-openjdk-amd64/bin/jstack -l 4624'
23:23:52  Warning: Permanently added '10.30.170.232' (ECDSA) to the list of known hosts.
23:23:53  + echo 'killing karaf process...'
23:23:53  killing karaf process...
23:23:53  + ssh 10.30.170.232 bash -c 'ps axf | grep karaf | grep -v grep | awk '\''{print "kill -9 " $1}'\'' | sh'
23:23:53  Warning: Permanently added '10.30.170.232' (ECDSA) to the list of known hosts.
23:23:53  + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
23:23:53  + CONTROLLERIP=ODL_SYSTEM_2_IP
23:23:53  + echo 'Let'\''s take the karaf thread dump again'
23:23:53  Let's take the karaf thread dump again
23:23:53  + ssh 10.30.170.47 'sudo ps aux'
23:23:53  Warning: Permanently added '10.30.170.47' (ECDSA) to the list of known hosts.
23:23:53  ++ tr -s ' '
23:23:53  ++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/ps_after.log
23:23:53  ++ cut -f2 '-d '
23:23:53  ++ grep -v grep
23:23:53  + pid=4599
23:23:53  + echo 'karaf main: org.apache.karaf.main.Main, pid:4599'
23:23:53  karaf main: org.apache.karaf.main.Main, pid:4599
23:23:53  + ssh 10.30.170.47 '/usr/lib/jvm/java-21-openjdk-amd64/bin/jstack -l 4599'
23:23:53  Warning: Permanently added '10.30.170.47' (ECDSA) to the list of known hosts.
23:23:54  + echo 'killing karaf process...'
23:23:54  killing karaf process...
23:23:54  + ssh 10.30.170.47 bash -c 'ps axf | grep karaf | grep -v grep | awk '\''{print "kill -9 " $1}'\'' | sh'
23:23:54  Warning: Permanently added '10.30.170.47' (ECDSA) to the list of known hosts.
23:23:54  + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
23:23:54  + CONTROLLERIP=ODL_SYSTEM_3_IP
23:23:54  + echo 'Let'\''s take the karaf thread dump again'
23:23:54  Let's take the karaf thread dump again
23:23:54  + ssh 10.30.170.110 'sudo ps aux'
23:23:54  Warning: Permanently added '10.30.170.110' (ECDSA) to the list of known hosts.
23:23:54  ++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/ps_after.log
23:23:54  ++ grep -v grep
23:23:54  ++ tr -s ' '
23:23:54  ++ cut -f2 '-d '
23:23:54  + pid=5074
23:23:54  + echo 'karaf main: org.apache.karaf.main.Main, pid:5074'
23:23:54  karaf main: org.apache.karaf.main.Main, pid:5074
23:23:54  + ssh 10.30.170.110 '/usr/lib/jvm/java-21-openjdk-amd64/bin/jstack -l 5074'
23:23:54  Warning: Permanently added '10.30.170.110' (ECDSA) to the list of known hosts.
23:23:55  + echo 'killing karaf process...'
23:23:55  killing karaf process...
23:23:55  + ssh 10.30.170.110 bash -c 'ps axf | grep karaf | grep -v grep | awk '\''{print "kill -9 " $1}'\'' | sh'
23:23:55  Warning: Permanently added '10.30.170.110' (ECDSA) to the list of known hosts.
23:23:55  + sleep 5
23:24:00  ++ seq 1 3
23:24:00  + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
23:24:00  + CONTROLLERIP=ODL_SYSTEM_1_IP
23:24:00  + echo 'Compressing karaf.log 1'
23:24:00  Compressing karaf.log 1
23:24:00  + ssh 10.30.170.232 gzip --best /tmp/karaf-0.21.3-SNAPSHOT/data/log/karaf.log
23:24:00  Warning: Permanently added '10.30.170.232' (ECDSA) to the list of known hosts.
23:24:00  + echo 'Fetching compressed karaf.log 1'
23:24:00  Fetching compressed karaf.log 1
23:24:00  + scp 10.30.170.232:/tmp/karaf-0.21.3-SNAPSHOT/data/log/karaf.log.gz odl1_karaf.log.gz
23:24:00  Warning: Permanently added '10.30.170.232' (ECDSA) to the list of known hosts.
23:24:00  + ssh 10.30.170.232 rm -f /tmp/karaf-0.21.3-SNAPSHOT/data/log/karaf.log.gz
23:24:00  Warning: Permanently added '10.30.170.232' (ECDSA) to the list of known hosts.
23:24:01  + scp 10.30.170.232:/tmp/karaf-0.21.3-SNAPSHOT/data/log/karaf_console.log odl1_karaf_console.log
23:24:01  Warning: Permanently added '10.30.170.232' (ECDSA) to the list of known hosts.
23:24:01  + ssh 10.30.170.232 rm -f /tmp/karaf-0.21.3-SNAPSHOT/data/log/karaf_console.log
23:24:01  Warning: Permanently added '10.30.170.232' (ECDSA) to the list of known hosts.
23:24:01  + echo 'Fetch GC logs'
23:24:01  Fetch GC logs
23:24:01  + mkdir -p gclogs-1
23:24:01  + scp '10.30.170.232:/tmp/karaf-0.21.3-SNAPSHOT/data/log/*.log' gclogs-1/
23:24:01  Warning: Permanently added '10.30.170.232' (ECDSA) to the list of known hosts.
23:24:01  scp: /tmp/karaf-0.21.3-SNAPSHOT/data/log/*.log: No such file or directory
23:24:01  + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
23:24:01  + CONTROLLERIP=ODL_SYSTEM_2_IP
23:24:01  + echo 'Compressing karaf.log 2'
23:24:01  Compressing karaf.log 2
23:24:01  + ssh 10.30.170.47 gzip --best /tmp/karaf-0.21.3-SNAPSHOT/data/log/karaf.log
23:24:01  Warning: Permanently added '10.30.170.47' (ECDSA) to the list of known hosts.
23:24:01  + echo 'Fetching compressed karaf.log 2'
23:24:01  Fetching compressed karaf.log 2
23:24:01  + scp 10.30.170.47:/tmp/karaf-0.21.3-SNAPSHOT/data/log/karaf.log.gz odl2_karaf.log.gz
23:24:01  Warning: Permanently added '10.30.170.47' (ECDSA) to the list of known hosts.
23:24:02  + ssh 10.30.170.47 rm -f /tmp/karaf-0.21.3-SNAPSHOT/data/log/karaf.log.gz
23:24:02  Warning: Permanently added '10.30.170.47' (ECDSA) to the list of known hosts.
23:24:02  + scp 10.30.170.47:/tmp/karaf-0.21.3-SNAPSHOT/data/log/karaf_console.log odl2_karaf_console.log
23:24:02  Warning: Permanently added '10.30.170.47' (ECDSA) to the list of known hosts.
23:24:02  + ssh 10.30.170.47 rm -f /tmp/karaf-0.21.3-SNAPSHOT/data/log/karaf_console.log
23:24:02  Warning: Permanently added '10.30.170.47' (ECDSA) to the list of known hosts.
23:24:02  + echo 'Fetch GC logs'
23:24:02  Fetch GC logs
23:24:02  + mkdir -p gclogs-2
23:24:02  + scp '10.30.170.47:/tmp/karaf-0.21.3-SNAPSHOT/data/log/*.log' gclogs-2/
23:24:02  Warning: Permanently added '10.30.170.47' (ECDSA) to the list of known hosts.
23:24:02  scp: /tmp/karaf-0.21.3-SNAPSHOT/data/log/*.log: No such file or directory
23:24:03  + for i in $(seq 1 "${NUM_ODL_SYSTEM}")
23:24:03  + CONTROLLERIP=ODL_SYSTEM_3_IP
23:24:03  + echo 'Compressing karaf.log 3'
23:24:03  Compressing karaf.log 3
23:24:03  + ssh 10.30.170.110 gzip --best /tmp/karaf-0.21.3-SNAPSHOT/data/log/karaf.log
23:24:03  Warning: Permanently added '10.30.170.110' (ECDSA) to the list of known hosts.
23:24:03  + echo 'Fetching compressed karaf.log 3'
23:24:03  Fetching compressed karaf.log 3
23:24:03  + scp 10.30.170.110:/tmp/karaf-0.21.3-SNAPSHOT/data/log/karaf.log.gz odl3_karaf.log.gz
23:24:03  Warning: Permanently added '10.30.170.110' (ECDSA) to the list of known hosts.
23:24:03  + ssh 10.30.170.110 rm -f /tmp/karaf-0.21.3-SNAPSHOT/data/log/karaf.log.gz
23:24:03  Warning: Permanently added '10.30.170.110' (ECDSA) to the list of known hosts.
23:24:03  + scp 10.30.170.110:/tmp/karaf-0.21.3-SNAPSHOT/data/log/karaf_console.log odl3_karaf_console.log
23:24:03  Warning: Permanently added '10.30.170.110' (ECDSA) to the list of known hosts.
23:24:03  + ssh 10.30.170.110 rm -f /tmp/karaf-0.21.3-SNAPSHOT/data/log/karaf_console.log
23:24:03  Warning: Permanently added '10.30.170.110' (ECDSA) to the list of known hosts.
23:24:04  + echo 'Fetch GC logs'
23:24:04  Fetch GC logs
23:24:04  + mkdir -p gclogs-3
23:24:04  + scp '10.30.170.110:/tmp/karaf-0.21.3-SNAPSHOT/data/log/*.log' gclogs-3/
23:24:04  Warning: Permanently added '10.30.170.110' (ECDSA) to the list of known hosts.
23:24:04  scp: /tmp/karaf-0.21.3-SNAPSHOT/data/log/*.log: No such file or directory
23:24:04  + echo 'Examine copied files'
23:24:04  Examine copied files
23:24:04  + ls -lt
23:24:04  total 3032
23:24:04  drwxrwxr-x. 2 jenkins jenkins       6 Aug 16 23:24 gclogs-3
23:24:04  -rw-rw-r--. 1 jenkins jenkins    1720 Aug 16 23:24 odl3_karaf_console.log
23:24:04  -rw-rw-r--. 1 jenkins jenkins   43363 Aug 16 23:24 odl3_karaf.log.gz
23:24:04  drwxrwxr-x. 2 jenkins jenkins       6 Aug 16 23:24 gclogs-2
23:24:04  -rw-rw-r--. 1 jenkins jenkins    1720 Aug 16 23:24 odl2_karaf_console.log
23:24:04  -rw-rw-r--. 1 jenkins jenkins   42050 Aug 16 23:24 odl2_karaf.log.gz
23:24:04  drwxrwxr-x. 2 jenkins jenkins       6 Aug 16 23:24 gclogs-1
23:24:04  -rw-rw-r--. 1 jenkins jenkins    1720 Aug 16 23:24 odl1_karaf_console.log
23:24:04  -rw-rw-r--. 1 jenkins jenkins   41334 Aug 16 23:24 odl1_karaf.log.gz
23:24:04  -rw-rw-r--. 1 jenkins jenkins   81324 Aug 16 23:23 karaf_3_5074_threads_after.log
23:24:04  -rw-rw-r--. 1 jenkins jenkins   13666 Aug 16 23:23 ps_after.log
23:24:04  -rw-rw-r--. 1 jenkins jenkins   83598 Aug 16 23:23 karaf_2_4599_threads_after.log
23:24:04  -rw-rw-r--. 1 jenkins jenkins   92615 Aug 16 23:23 karaf_1_4624_threads_after.log
23:24:04  -rw-rw-r--. 1 jenkins jenkins  247619 Aug 16 23:23 report.html
23:24:04  -rw-rw-r--. 1 jenkins jenkins  463343 Aug 16 23:23 log.html
23:24:04  -rw-rw-r--. 1 jenkins jenkins 1654890 Aug 16 23:23 output.xml
23:24:04  -rw-rw-r--. 1 jenkins jenkins     661 Aug 16 23:17 testplan.txt
23:24:04  -rw-rw-r--. 1 jenkins jenkins   74857 Aug 16 23:17 karaf_3_2037_threads_before.log
23:24:04  -rw-rw-r--. 1 jenkins jenkins   13727 Aug 16 23:17 ps_before.log
23:24:04  -rw-rw-r--. 1 jenkins jenkins   74863 Aug 16 23:17 karaf_2_2058_threads_before.log
23:24:04  -rw-rw-r--. 1 jenkins jenkins   77422 Aug 16 23:17 karaf_1_2050_threads_before.log
23:24:04  -rw-rw-r--. 1 jenkins jenkins    3106 Aug 16 23:13 post-startup-script.sh
23:24:04  -rw-rw-r--. 1 jenkins jenkins     252 Aug 16 23:13 startup-script.sh
23:24:04  -rw-rw-r--. 1 jenkins jenkins    3463 Aug 16 23:13 configuration-script.sh
23:24:04  -rw-rw-r--. 1 jenkins jenkins     337 Aug 16 23:13 detect_variables.env
23:24:04  -rw-rw-r--. 1 jenkins jenkins    2619 Aug 16 23:13 pom.xml
23:24:04  -rw-rw-r--. 1 jenkins jenkins      92 Aug 16 23:13 set_variables.env
23:24:04  -rw-rw-r--. 1 jenkins jenkins     356 Aug 16 23:13 slave_addresses.txt
23:24:04  -rw-rw-r--. 1 jenkins jenkins     570 Aug 16 23:12 requirements.txt
23:24:04  -rw-rw-r--. 1 jenkins jenkins      26 Aug 16 23:12 env.properties
23:24:04  -rw-rw-r--. 1 jenkins jenkins     333 Aug 16 23:10 stack-parameters.yaml
23:24:04  drwxrwxr-x. 7 jenkins jenkins    4096 Aug 16 23:09 test
23:24:04  drwxrwxr-x. 2 jenkins jenkins       6 Aug 16 23:09 test@tmp
23:24:04  -rw-rw-r--. 1 jenkins jenkins    1410 Aug 16 21:25 maven-metadata.xml
23:24:04  + true
23:24:04  [controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium] $ /bin/sh /tmp/jenkins10782451657248905593.sh
23:24:04  Cleaning up Robot installation...
23:24:04  $ ssh-agent -k
23:24:04  unset SSH_AUTH_SOCK;
23:24:04  unset SSH_AGENT_PID;
23:24:04  echo Agent pid 5282 killed;
23:24:04  [ssh-agent] Stopped.
23:24:04  Recording plot data
23:24:04  Robot results publisher started...
23:24:04  INFO: Checking test criticality is deprecated and will be dropped in a future release!
23:24:04  -Parsing output xml:
23:24:05  Done!
23:24:05  -Copying log files to build dir:
23:24:05  Done!
23:24:05  -Assigning results to build:
23:24:05  Done!
23:24:05  -Checking thresholds:
23:24:05  Done!
23:24:05  Done publishing Robot results.
23:24:05  Build step 'Publish Robot Framework test results' changed build result to UNSTABLE
23:24:05  [PostBuildScript] - [INFO] Executing post build scripts.
23:24:05  [controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium] $ /bin/bash /tmp/jenkins13168745993735231482.sh
23:24:05  Archiving csit artifacts
23:24:05  mv: cannot stat '*_1.png': No such file or directory
23:24:05  mv: cannot stat '/tmp/odl1_*': No such file or directory
23:24:05  mv: cannot stat '*_2.png': No such file or directory
23:24:05  mv: cannot stat '/tmp/odl2_*': No such file or directory
23:24:05  mv: cannot stat '*_3.png': No such file or directory
23:24:05  mv: cannot stat '/tmp/odl3_*': No such file or directory
23:24:05    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
23:24:05                                   Dload  Upload   Total   Spent    Left  Speed
23:24:05  
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  324k    0  324k    0     0  1320k      0 --:--:-- --:--:-- --:--:-- 1315k
23:24:06  Archive:  robot-plugin.zip
23:24:06    inflating: ./archives/robot-plugin/log.html  
23:24:06    inflating: ./archives/robot-plugin/output.xml  
23:24:06    inflating: ./archives/robot-plugin/report.html  
23:24:06  mv: cannot stat '*.log.gz': No such file or directory
23:24:06  mv: cannot stat '*.csv': No such file or directory
23:24:06  mv: cannot stat '*.png': No such file or directory
23:24:06  [PostBuildScript] - [INFO] Executing post build scripts.
23:24:06  [controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium] $ /bin/bash /tmp/jenkins988121261700999696.sh
23:24:06  [PostBuildScript] - [INFO] Executing post build scripts.
23:24:06  [EnvInject] - Injecting environment variables from a build step.
23:24:06  [EnvInject] - Injecting as environment variables the properties content 
23:24:06  OS_CLOUD=vex
23:24:06  OS_STACK_NAME=releng-controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium-69
23:24:06  
23:24:06  [EnvInject] - Variables injected successfully.
23:24:06  provisioning config files...
23:24:06  copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
23:24:06  [controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium] $ /bin/bash /tmp/jenkins7762205972443769537.sh
23:24:06  ---> openstack-stack-delete.sh
23:24:06  Setup pyenv:
23:24:06    system
23:24:06    3.8.13
23:24:06    3.9.13
23:24:06    3.10.13
23:24:06  * 3.11.7 (set by /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/.python-version)
23:24:07  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-8Z2X from file:/tmp/.os_lf_venv
23:24:09  ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
23:24:09  lftools 0.37.13 requires urllib3<2.1.0, but you have urllib3 2.5.0 which is incompatible.
23:24:09  lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes python-heatclient python-openstackclient 
23:24:23  lf-activate-venv(): INFO: Adding /tmp/venv-8Z2X/bin to PATH
23:24:23  INFO: Retrieving stack cost for: releng-controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium-69
23:24:27  DEBUG: Successfully retrieved stack cost: total: 0.38999999999999996
23:24:39  INFO: Deleting stack releng-controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium-69
23:24:39  Successfully deleted stack releng-controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium-69
23:24:39  [PostBuildScript] - [INFO] Executing post build scripts.
23:24:39  [controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium] $ /bin/bash /tmp/jenkins13967979443456763965.sh
23:24:39  ---> sysstat.sh
23:24:40  [controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium] $ /bin/bash /tmp/jenkins2611613657225624019.sh
23:24:40  ---> package-listing.sh
23:24:40  ++ facter osfamily
23:24:40  ++ tr '[:upper:]' '[:lower:]'
23:24:40  + OS_FAMILY=redhat
23:24:40  + workspace=/w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium
23:24:40  + START_PACKAGES=/tmp/packages_start.txt
23:24:40  + END_PACKAGES=/tmp/packages_end.txt
23:24:40  + DIFF_PACKAGES=/tmp/packages_diff.txt
23:24:40  + PACKAGES=/tmp/packages_start.txt
23:24:40  + '[' /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium ']'
23:24:40  + PACKAGES=/tmp/packages_end.txt
23:24:40  + case "${OS_FAMILY}" in
23:24:40  + rpm -qa
23:24:40  + sort
23:24:41  + '[' -f /tmp/packages_start.txt ']'
23:24:41  + '[' -f /tmp/packages_end.txt ']'
23:24:41  + diff /tmp/packages_start.txt /tmp/packages_end.txt
23:24:41  + '[' /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium ']'
23:24:41  + mkdir -p /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/archives/
23:24:41  + cp -f /tmp/packages_diff.txt /tmp/packages_end.txt /tmp/packages_start.txt /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/archives/
23:24:41  [controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium] $ /bin/bash /tmp/jenkins10521875215186345665.sh
23:24:41  ---> capture-instance-metadata.sh
23:24:41  Setup pyenv:
23:24:41    system
23:24:41    3.8.13
23:24:41    3.9.13
23:24:41    3.10.13
23:24:41  * 3.11.7 (set by /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/.python-version)
23:24:41  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-8Z2X from file:/tmp/.os_lf_venv
23:24:43  lf-activate-venv(): INFO: Installing: lftools 
23:24:52  lf-activate-venv(): INFO: Adding /tmp/venv-8Z2X/bin to PATH
23:24:52  INFO: Running in OpenStack, capturing instance metadata
23:24:53  [controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium] $ /bin/bash /tmp/jenkins16358488075305703165.sh
23:24:53  provisioning config files...
23:24:53  Could not find credentials [logs] for controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium #69
23:24:53  copy managed file [jenkins-log-archives-settings] to file:/w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium@tmp/config17128647681401958907tmp
23:24:53  Regular expression run condition: Expression=[^.*logs-s3.*], Label=[odl-logs-s3-cloudfront-index]
23:24:53  Run condition [Regular expression match] enabling perform for step [Provide Configuration files]
23:24:53  provisioning config files...
23:24:53  copy managed file [jenkins-s3-log-ship] to file:/home/jenkins/.aws/credentials
23:24:53  [EnvInject] - Injecting environment variables from a build step.
23:24:53  [EnvInject] - Injecting as environment variables the properties content 
23:24:53  SERVER_ID=logs
23:24:53  
23:24:53  [EnvInject] - Variables injected successfully.
23:24:53  [controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium] $ /bin/bash /tmp/jenkins10558760502119006330.sh
23:24:53  ---> create-netrc.sh
23:24:53  WARN: Log server credential not found.
23:24:53  [controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium] $ /bin/bash /tmp/jenkins4050425415737419440.sh
23:24:53  ---> python-tools-install.sh
23:24:53  Setup pyenv:
23:24:53    system
23:24:53    3.8.13
23:24:53    3.9.13
23:24:53    3.10.13
23:24:53  * 3.11.7 (set by /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/.python-version)
23:24:54  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-8Z2X from file:/tmp/.os_lf_venv
23:24:55  lf-activate-venv(): INFO: Installing: lftools 
23:25:05  lf-activate-venv(): INFO: Adding /tmp/venv-8Z2X/bin to PATH
23:25:05  [controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium] $ /bin/bash /tmp/jenkins13346114183268855779.sh
23:25:05  ---> sudo-logs.sh
23:25:05  Archiving 'sudo' log..
23:25:05  [controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium] $ /bin/bash /tmp/jenkins15149776504922199106.sh
23:25:05  ---> job-cost.sh
23:25:05  Setup pyenv:
23:25:05    system
23:25:05    3.8.13
23:25:05    3.9.13
23:25:05    3.10.13
23:25:05  * 3.11.7 (set by /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/.python-version)
23:25:06  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-8Z2X from file:/tmp/.os_lf_venv
23:25:07  lf-activate-venv(): INFO: Installing: zipp==1.1.0 python-openstackclient urllib3~=1.26.15 
23:25:14  lf-activate-venv(): INFO: Adding /tmp/venv-8Z2X/bin to PATH
23:25:14  DEBUG: total: 0.38999999999999996
23:25:14  INFO: Retrieving Stack Cost...
23:25:15  INFO: Retrieving Pricing Info for: v3-standard-2
23:25:15  INFO: Archiving Costs
23:25:15  [controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium] $ /bin/bash -l /tmp/jenkins9959083675564714793.sh
23:25:15  ---> logs-deploy.sh
23:25:15  Setup pyenv:
23:25:15    system
23:25:15    3.8.13
23:25:15    3.9.13
23:25:15    3.10.13
23:25:15  * 3.11.7 (set by /w/workspace/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/.python-version)
23:25:15  lf-activate-venv(): INFO: Reuse venv:/tmp/venv-8Z2X from file:/tmp/.os_lf_venv
23:25:17  lf-activate-venv(): INFO: Installing: lftools 
23:25:27  lf-activate-venv(): INFO: Adding /tmp/venv-8Z2X/bin to PATH
23:25:27  WARNING: Nexus logging server not set
23:25:27  INFO: S3 path logs/releng/vex-yul-odl-jenkins-1/controller-csit-3node-ddb-expl-lead-movement-longevity-only-scandium/69/
23:25:27  INFO: archiving logs to S3
23:25:28  ---> uname -a:
23:25:28  Linux prd-centos8-robot-2c-8g-1901.novalocal 4.18.0-553.5.1.el8.x86_64 #1 SMP Tue May 21 05:46:01 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
23:25:28  
23:25:28  
23:25:28  ---> lscpu:
23:25:28  Architecture:        x86_64
23:25:28  CPU op-mode(s):      32-bit, 64-bit
23:25:28  Byte Order:          Little Endian
23:25:28  CPU(s):              2
23:25:28  On-line CPU(s) list: 0,1
23:25:28  Thread(s) per core:  1
23:25:28  Core(s) per socket:  1
23:25:28  Socket(s):           2
23:25:28  NUMA node(s):        1
23:25:28  Vendor ID:           AuthenticAMD
23:25:28  CPU family:          23
23:25:28  Model:               49
23:25:28  Model name:          AMD EPYC-Rome Processor
23:25:28  Stepping:            0
23:25:28  CPU MHz:             2799.998
23:25:28  BogoMIPS:            5599.99
23:25:28  Virtualization:      AMD-V
23:25:28  Hypervisor vendor:   KVM
23:25:28  Virtualization type: full
23:25:28  L1d cache:           32K
23:25:28  L1i cache:           32K
23:25:28  L2 cache:            512K
23:25:28  L3 cache:            16384K
23:25:28  NUMA node0 CPU(s):   0,1
23:25:28  Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr wbnoinvd arat npt nrip_save umip rdpid arch_capabilities
23:25:28  
23:25:28  
23:25:28  ---> nproc:
23:25:28  2
23:25:28  
23:25:28  
23:25:28  ---> df -h:
23:25:28  Filesystem      Size  Used Avail Use% Mounted on
23:25:28  devtmpfs        3.8G     0  3.8G   0% /dev
23:25:28  tmpfs           3.8G     0  3.8G   0% /dev/shm
23:25:28  tmpfs           3.8G   17M  3.8G   1% /run
23:25:28  tmpfs           3.8G     0  3.8G   0% /sys/fs/cgroup
23:25:28  /dev/vda1        40G  8.3G   32G  21% /
23:25:28  tmpfs           770M     0  770M   0% /run/user/1001
23:25:28  
23:25:28  
23:25:28  ---> free -m:
23:25:28                total        used        free      shared  buff/cache   available
23:25:28  Mem:           7697         580        4936          19        2181        6818
23:25:28  Swap:          1023           0        1023
23:25:28  
23:25:28  
23:25:28  ---> ip addr:
23:25:28  1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
23:25:28      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
23:25:28      inet 127.0.0.1/8 scope host lo
23:25:28         valid_lft forever preferred_lft forever
23:25:28      inet6 ::1/128 scope host 
23:25:28         valid_lft forever preferred_lft forever
23:25:28  2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1458 qdisc mq state UP group default qlen 1000
23:25:28      link/ether fa:16:3e:28:0b:fa brd ff:ff:ff:ff:ff:ff
23:25:28      altname enp0s3
23:25:28      altname ens3
23:25:28      inet 10.30.171.216/23 brd 10.30.171.255 scope global dynamic noprefixroute eth0
23:25:28         valid_lft 85406sec preferred_lft 85406sec
23:25:28      inet6 fe80::f816:3eff:fe28:bfa/64 scope link 
23:25:28         valid_lft forever preferred_lft forever
23:25:28  
23:25:28  
23:25:28  ---> sar -b -r -n DEV:
23:25:28  Linux 4.18.0-553.5.1.el8.x86_64 (centos-stream-8-robot-7d7a37eb-bc14-4dd6-9530-dc22c5eae738.noval) 	08/16/2025 	_x86_64_	(2 CPU)
23:25:28  
23:25:28  23:08:51     LINUX RESTART	(2 CPU)
23:25:28  
23:25:28  11:09:01 PM       tps      rtps      wtps   bread/s   bwrtn/s
23:25:28  11:10:01 PM    214.52    157.21     57.31  14704.55  40155.18
23:25:28  11:11:01 PM    119.57      0.72    118.86     52.91  11095.15
23:25:28  11:12:01 PM     39.99      0.40     39.59     62.51   4311.26
23:25:28  11:13:01 PM     32.44      6.75     25.70   1248.19   1917.10
23:25:28  11:14:01 PM     62.06      2.73     59.33    439.59   7946.13
23:25:28  11:15:01 PM      4.15      0.00      4.15      0.00    149.16
23:25:28  11:16:01 PM      0.12      0.00      0.12      0.00      0.90
23:25:28  11:17:01 PM      0.13      0.00      0.13      0.00      1.07
23:25:28  11:18:01 PM      2.28      0.23      2.05     13.46    102.08
23:25:28  11:19:01 PM      0.42      0.00      0.42      0.00     15.01
23:25:28  11:20:01 PM      0.28      0.00      0.28      0.00      5.95
23:25:28  11:21:01 PM      0.30      0.00      0.30      0.00      7.38
23:25:28  11:22:01 PM      0.20      0.00      0.20      0.00      4.83
23:25:28  11:23:01 PM      0.17      0.00      0.17      0.00      5.22
23:25:28  11:24:01 PM      0.55      0.03      0.52      0.27     13.20
23:25:28  11:25:01 PM     35.93      0.66     35.27     75.38   1628.88
23:25:28  Average:        32.07     10.56     21.51   1038.64   4213.69
23:25:28  
23:25:28  11:09:01 PM kbmemfree   kbavail kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit  kbactive   kbinact   kbdirty
23:25:28  11:10:01 PM   5516588   7058700   2365836     30.01      2688   1734720    719708      8.06    185300   1897532     91852
23:25:28  11:11:01 PM   5175184   7012672   2707240     34.35      2688   2013260    689448      7.72    197888   2157508    103992
23:25:28  11:12:01 PM   5247668   7086840   2634756     33.43      2688   2015136    634340      7.10    218516   2066204     10772
23:25:28  11:13:01 PM   4918756   6975808   2963668     37.60      2688   2227192    739548      8.28    274504   2320272    154252
23:25:28  11:14:01 PM   4955068   7055604   2927356     37.14      2688   2268004    627692      7.03    267388   2277540        36
23:25:28  11:15:01 PM   4955376   7055912   2927048     37.13      2688   2268004    627692      7.03    267392   2277364         4
23:25:28  11:16:01 PM   4955608   7056124   2926816     37.13      2688   2268004    627692      7.03    267392   2277296         4
23:25:28  11:17:01 PM   4955616   7056136   2926808     37.13      2688   2268008    627692      7.03    267392   2277308         4
23:25:28  11:18:01 PM   4906408   7008472   2976016     37.76      2688   2269552    714600      8.00    267804   2325928       188
23:25:28  11:19:01 PM   4906140   7008284   2976284     37.76      2688   2269644    714600      8.00    267804   2326320        28
23:25:28  11:20:01 PM   4905352   7007584   2977072     37.77      2688   2269740    732300      8.20    267804   2326892        56
23:25:28  11:21:01 PM   4905228   7007560   2977196     37.77      2688   2269832    732300      8.20    267804   2327148        24
23:25:28  11:22:01 PM   4904680   7007104   2977744     37.78      2688   2269924    732300      8.20    267804   2327556        52
23:25:28  11:23:01 PM   4904492   7007008   2977932     37.78      2688   2270016    732300      8.20    267804   2327896        32
23:25:28  11:24:01 PM   4951188   7055060   2931236     37.19      2688   2271372    637224      7.13    268376   2280768      1316
23:25:28  11:25:01 PM   5065968   6987632   2816456     35.73      2688   2094520    698928      7.83    464420   1994244      9068
23:25:28  Average:      5008082   7027906   2874342     36.47      2688   2190433    686773      7.69    267837   2236736     23230
23:25:28  
23:25:28  11:09:01 PM     IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
23:25:28  11:10:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
23:25:28  11:10:01 PM      eth0    286.58    164.39   1523.91     57.37      0.00      0.00      0.00      0.00
23:25:28  11:11:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
23:25:28  11:11:01 PM      eth0     90.15     69.73   1111.61     11.00      0.00      0.00      0.00      0.00
23:25:28  11:12:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
23:25:28  11:12:01 PM      eth0     14.03      9.01      5.67      3.62      0.00      0.00      0.00      0.00
23:25:28  11:13:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
23:25:28  11:13:01 PM      eth0     43.03     33.88    327.69      6.81      0.00      0.00      0.00      0.00
23:25:28  11:14:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
23:25:28  11:14:01 PM      eth0    347.53    186.77     90.51     66.05      0.00      0.00      0.00      0.00
23:25:28  11:15:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
23:25:28  11:15:01 PM      eth0      2.72      2.03      0.56      0.48      0.00      0.00      0.00      0.00
23:25:28  11:16:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
23:25:28  11:16:01 PM      eth0      2.62      2.02      0.44      0.43      0.00      0.00      0.00      0.00
23:25:28  11:17:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
23:25:28  11:17:01 PM      eth0      1.90      1.33      0.30      0.24      0.00      0.00      0.00      0.00
23:25:28  11:18:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
23:25:28  11:18:01 PM      eth0     58.25     42.80     14.91      5.97      0.00      0.00      0.00      0.00
23:25:28  11:19:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
23:25:28  11:19:01 PM      eth0      4.07      0.30      0.25      0.05      0.00      0.00      0.00      0.00
23:25:28  11:20:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
23:25:28  11:20:01 PM      eth0      2.62      0.37      0.31      0.11      0.00      0.00      0.00      0.00
23:25:28  11:21:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
23:25:28  11:21:01 PM      eth0      0.58      0.30      0.10      0.05      0.00      0.00      0.00      0.00
23:25:28  11:22:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
23:25:28  11:22:01 PM      eth0      0.53      0.30      0.09      0.05      0.00      0.00      0.00      0.00
23:25:28  11:23:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
23:25:28  11:23:01 PM      eth0      1.03      0.32      0.19      0.08      0.00      0.00      0.00      0.00
23:25:28  11:24:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
23:25:28  11:24:01 PM      eth0     55.04     43.48     14.14      5.11      0.00      0.00      0.00      0.00
23:25:28  11:25:01 PM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
23:25:28  11:25:01 PM      eth0     49.23     38.26     87.04     29.11      0.00      0.00      0.00      0.00
23:25:28  Average:           lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
23:25:28  Average:         eth0     60.01     37.21    198.77     11.64      0.00      0.00      0.00      0.00
23:25:28  
23:25:28  
23:25:28  ---> sar -P ALL:
23:25:28  Linux 4.18.0-553.5.1.el8.x86_64 (centos-stream-8-robot-7d7a37eb-bc14-4dd6-9530-dc22c5eae738.noval) 	08/16/2025 	_x86_64_	(2 CPU)
23:25:28  
23:25:28  23:08:51     LINUX RESTART	(2 CPU)
23:25:28  
23:25:28  11:09:01 PM     CPU     %user     %nice   %system   %iowait    %steal     %idle
23:25:28  11:10:01 PM     all     48.59      0.07     10.26      3.46      0.09     37.53
23:25:28  11:10:01 PM       0     46.25      0.07     10.24      3.00      0.08     40.35
23:25:28  11:10:01 PM       1     50.93      0.07     10.27      3.92      0.10     34.71
23:25:28  11:11:01 PM     all     37.10      0.00      5.44      1.16      0.10     56.21
23:25:28  11:11:01 PM       0     40.18      0.00      6.00      1.12      0.10     52.61
23:25:28  11:11:01 PM       1     34.02      0.00      4.88      1.20      0.10     59.81
23:25:28  11:12:01 PM     all     11.83      0.00      1.65      0.79      0.06     85.66
23:25:28  11:12:01 PM       0     13.80      0.00      1.75      1.02      0.05     83.38
23:25:28  11:12:01 PM       1      9.86      0.00      1.55      0.57      0.07     87.95
23:25:28  11:13:01 PM     all     17.37      0.00      3.66      1.49      0.07     77.41
23:25:28  11:13:01 PM       0     16.30      0.00      3.43      1.30      0.07     78.90
23:25:28  11:13:01 PM       1     18.43      0.00      3.89      1.68      0.07     75.93
23:25:28  11:14:01 PM     all     16.78      0.00      3.52      1.45      0.08     78.17
23:25:28  11:14:01 PM       0     23.01      0.00      4.12      1.42      0.08     71.36
23:25:28  11:14:01 PM       1     10.56      0.00      2.91      1.47      0.08     84.98
23:25:28  11:15:01 PM     all      0.35      0.00      0.10      0.03      0.03     99.48
23:25:28  11:15:01 PM       0      0.13      0.00      0.08      0.03      0.03     99.72
23:25:28  11:15:01 PM       1      0.57      0.00      0.12      0.03      0.03     99.25
23:25:28  11:16:01 PM     all      0.37      0.00      0.07      0.00      0.03     99.53
23:25:28  11:16:01 PM       0      0.17      0.00      0.10      0.00      0.05     99.68
23:25:28  11:16:01 PM       1      0.57      0.00      0.03      0.00      0.02     99.38
23:25:28  11:17:01 PM     all      0.33      0.00      0.07      0.00      0.03     99.57
23:25:28  11:17:01 PM       0      0.15      0.00      0.10      0.00      0.03     99.72
23:25:28  11:17:01 PM       1      0.52      0.00      0.03      0.00      0.03     99.42
23:25:28  11:18:01 PM     all      8.15      0.00      0.61      0.07      0.07     91.10
23:25:28  11:18:01 PM       0     11.52      0.00      0.72      0.00      0.07     87.69
23:25:28  11:18:01 PM       1      4.78      0.00      0.50      0.13      0.07     94.52
23:25:28  11:19:01 PM     all      0.58      0.00      0.10      0.00      0.07     99.25
23:25:28  11:19:01 PM       0      0.50      0.00      0.12      0.00      0.08     99.30
23:25:28  11:19:01 PM       1      0.67      0.00      0.08      0.00      0.05     99.20
23:25:28  11:20:01 PM     all      0.69      0.00      0.09      0.00      0.07     99.15
23:25:28  11:20:01 PM       0      0.82      0.00      0.12      0.00      0.07     99.00
23:25:28  11:20:01 PM       1      0.57      0.00      0.07      0.00      0.07     99.30
23:25:28  
23:25:28  11:20:01 PM     CPU     %user     %nice   %system   %iowait    %steal     %idle
23:25:28  11:21:01 PM     all      0.59      0.00      0.10      0.00      0.07     99.24
23:25:28  11:21:01 PM       0      0.37      0.00      0.12      0.00      0.07     99.45
23:25:28  11:21:01 PM       1      0.82      0.00      0.08      0.00      0.07     99.03
23:25:28  11:22:01 PM     all      0.61      0.00      0.12      0.00      0.08     99.20
23:25:28  11:22:01 PM       0      0.50      0.00      0.12      0.00      0.08     99.30
23:25:28  11:22:01 PM       1      0.72      0.00      0.12      0.00      0.07     99.10
23:25:28  11:23:01 PM     all      0.66      0.00      0.09      0.00      0.08     99.17
23:25:28  11:23:01 PM       0      0.73      0.00      0.12      0.00      0.10     99.05
23:25:28  11:23:01 PM       1      0.58      0.00      0.07      0.00      0.05     99.30
23:25:28  11:24:01 PM     all      2.86      0.00      0.52      0.01      0.07     96.54
23:25:28  11:24:01 PM       0      3.74      0.00      0.67      0.00      0.07     95.53
23:25:28  11:24:01 PM       1      1.99      0.00      0.37      0.02      0.07     97.56
23:25:28  11:25:01 PM     all     29.05      0.00      3.77      0.18      0.08     66.92
23:25:28  11:25:01 PM       0     36.62      0.00      3.65      0.24      0.08     59.41
23:25:28  11:25:01 PM       1     21.49      0.00      3.88      0.12      0.08     74.43
23:25:28  Average:        all     10.97      0.00      1.88      0.54      0.07     86.53
23:25:28  Average:          0     12.14      0.00      1.96      0.51      0.07     85.31
23:25:28  Average:          1      9.80      0.00      1.80      0.57      0.06     87.75
23:25:28  
23:25:28  
23:25:28