Started by upstream project "integration-distribution-mri-test-calcium" build number 76
originally caused by:
 Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on prd-centos8-robot-2c-8g-1436 (centos8-robot-2c-8g) in workspace /w/workspace/controller-csit-3node-clustering-tell-all-calcium
[ssh-agent] Looking for ssh-agent implementation...
[ssh-agent]   Exec ssh-agent (binary ssh-agent on a remote machine)
$ ssh-agent
SSH_AUTH_SOCK=/tmp/ssh-bk7CKnFAoqw7/agent.5277
SSH_AGENT_PID=5278
[ssh-agent] Started.
Running ssh-add (command line suppressed)
Identity added: /w/workspace/controller-csit-3node-clustering-tell-all-calcium@tmp/private_key_15316969854878000790.key (/w/workspace/controller-csit-3node-clustering-tell-all-calcium@tmp/private_key_15316969854878000790.key)
[ssh-agent] Using credentials jenkins (Release Engineering Jenkins Key)
The recommended git tool is: NONE
using credential opendaylight-jenkins-ssh
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository git://devvexx.opendaylight.org/mirror/integration/test
 > git init /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test # timeout=10
Fetching upstream changes from git://devvexx.opendaylight.org/mirror/integration/test
 > git --version # timeout=10
 > git --version # 'git version 2.43.0'
using GIT_SSH to set credentials Release Engineering Jenkins Key
[INFO] Currently running in a labeled security context
[INFO] Currently SELinux is 'enforcing' on the host
 > /usr/bin/chcon --type=ssh_home_t /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test@tmp/jenkins-gitclient-ssh15544140727414173398.key
Verifying host key using known hosts file
You're using 'Known hosts file' strategy to verify ssh host keys, but your known_hosts file does not exist, please go to 'Manage Jenkins' -> 'Security' -> 'Git Host Key Verification Configuration' and configure host key verification.
 > git fetch --tags --force --progress -- git://devvexx.opendaylight.org/mirror/integration/test +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url git://devvexx.opendaylight.org/mirror/integration/test # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url git://devvexx.opendaylight.org/mirror/integration/test # timeout=10
Fetching upstream changes from git://devvexx.opendaylight.org/mirror/integration/test
using GIT_SSH to set credentials Release Engineering Jenkins Key
[INFO] Currently running in a labeled security context
[INFO] Currently SELinux is 'enforcing' on the host
 > /usr/bin/chcon --type=ssh_home_t /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test@tmp/jenkins-gitclient-ssh4324701705242078599.key
Verifying host key using known hosts file
You're using 'Known hosts file' strategy to verify ssh host keys, but your known_hosts file does not exist, please go to 'Manage Jenkins' -> 'Security' -> 'Git Host Key Verification Configuration' and configure host key verification.
 > git fetch --tags --force --progress -- git://devvexx.opendaylight.org/mirror/integration/test master # timeout=10
 > git rev-parse FETCH_HEAD^{commit} # timeout=10
Checking out Revision 1d2a1a30bcc48e3c8d571eb0d9b6891fe29dfed4 (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 1d2a1a30bcc48e3c8d571eb0d9b6891fe29dfed4 # timeout=10
Commit message: "Add bgpcep variables for titanium"
 > git rev-parse FETCH_HEAD^{commit} # timeout=10
 > git rev-list --no-walk d42b2c24fe4fcc7fe3ae0d352311e768f07a15b7 # timeout=10
No emails were triggered.
provisioning config files...
copy managed file [npmrc] to file:/home/jenkins/.npmrc
copy managed file [pipconf] to file:/home/jenkins/.config/pip/pip.conf
copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins1716220192724097776.sh
---> python-tools-install.sh
Setup pyenv:
  system
* 3.8.13 (set by /opt/pyenv/version)
* 3.9.13 (set by /opt/pyenv/version)
* 3.10.13 (set by /opt/pyenv/version)
* 3.11.7 (set by /opt/pyenv/version)
lf-activate-venv(): INFO: Creating python3 venv at /tmp/venv-QYyr
lf-activate-venv(): INFO: Save venv in file: /tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing: lftools 
lf-activate-venv(): INFO: Adding /tmp/venv-QYyr/bin to PATH
Generating Requirements File
Python 3.11.7
pip 25.0.1 from /tmp/venv-QYyr/lib/python3.11/site-packages/pip (python 3.11)
appdirs==1.4.4
argcomplete==3.5.3
aspy.yaml==1.3.0
attrs==25.1.0
autopage==0.5.2
beautifulsoup4==4.13.3
boto3==1.36.26
botocore==1.36.26
bs4==0.0.2
cachetools==5.5.2
certifi==2025.1.31
cffi==1.17.1
cfgv==3.4.0
chardet==5.2.0
charset-normalizer==3.4.1
click==8.1.8
cliff==4.8.0
cmd2==2.5.11
cryptography==3.3.2
debtcollector==3.0.0
decorator==5.2.0
defusedxml==0.7.1
Deprecated==1.2.18
distlib==0.3.9
dnspython==2.7.0
docker==4.2.2
dogpile.cache==1.3.4
durationpy==0.9
email_validator==2.2.0
filelock==3.17.0
future==1.0.0
gitdb==4.0.12
GitPython==3.1.44
google-auth==2.38.0
httplib2==0.22.0
identify==2.6.8
idna==3.10
importlib-resources==1.5.0
iso8601==2.1.0
Jinja2==3.1.5
jmespath==1.0.1
jsonpatch==1.33
jsonpointer==3.0.0
jsonschema==4.23.0
jsonschema-specifications==2024.10.1
keystoneauth1==5.10.0
kubernetes==32.0.1
lftools==0.37.10
lxml==5.3.1
MarkupSafe==3.0.2
msgpack==1.1.0
multi_key_dict==2.0.3
munch==4.0.0
netaddr==1.3.0
niet==1.4.2
nodeenv==1.9.1
oauth2client==4.1.3
oauthlib==3.2.2
openstacksdk==4.3.0
os-client-config==2.1.0
os-service-types==1.7.0
osc-lib==3.2.0
oslo.config==9.7.1
oslo.context==5.7.1
oslo.i18n==6.5.1
oslo.log==7.1.0
oslo.serialization==5.7.0
oslo.utils==8.2.0
packaging==24.2
pbr==6.1.1
platformdirs==4.3.6
prettytable==3.14.0
psutil==7.0.0
pyasn1==0.6.1
pyasn1_modules==0.4.1
pycparser==2.22
pygerrit2==2.0.15
PyGithub==2.6.1
PyJWT==2.10.1
PyNaCl==1.5.0
pyparsing==2.4.7
pyperclip==1.9.0
pyrsistent==0.20.0
python-cinderclient==9.6.0
python-dateutil==2.9.0.post0
python-heatclient==4.1.0
python-jenkins==1.8.2
python-keystoneclient==5.5.0
python-magnumclient==4.7.0
python-openstackclient==7.3.1
python-swiftclient==4.6.0
PyYAML==6.0.2
referencing==0.36.2
requests==2.32.3
requests-oauthlib==2.0.0
requestsexceptions==1.4.0
rfc3986==2.0.0
rpds-py==0.23.1
rsa==4.9
ruamel.yaml==0.18.10
ruamel.yaml.clib==0.2.12
s3transfer==0.11.2
simplejson==3.20.1
six==1.17.0
smmap==5.0.2
soupsieve==2.6
stevedore==5.4.1
tabulate==0.9.0
toml==0.10.2
tomlkit==0.13.2
tqdm==4.67.1
typing_extensions==4.12.2
tzdata==2025.1
urllib3==1.26.20
virtualenv==20.29.2
wcwidth==0.2.13
websocket-client==1.8.0
wrapt==1.17.2
xdg==6.0.0
xmltodict==0.14.2
yq==3.4.3
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties content 
OS_STACK_TEMPLATE=csit-2-instance-type.yaml
OS_CLOUD=vex
OS_STACK_NAME=releng-controller-csit-3node-clustering-tell-all-calcium-76
OS_STACK_TEMPLATE_DIR=openstack-hot

[EnvInject] - Variables injected successfully.
provisioning config files...
copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins13479462127160337634.sh
---> Create parameters file for OpenStack HOT
OpenStack Heat parameters generated
-----------------------------------
parameters:
    vm_0_count: '3'
    vm_0_flavor: 'v3-standard-4'
    vm_0_image: 'ZZCI - Ubuntu 20.04 - builder - x86_64 - 20250101-160150.998'
    vm_1_count: '0'
    vm_1_flavor: 'v3-standard-2'
    vm_1_image: 'ZZCI - Ubuntu 22.04 - mininet-ovs-217 - x86_64 - 20250201-060151.911'
    
    job_name: '28865-76'
    silo: 'releng'
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash -l /tmp/jenkins9539986079151283861.sh
---> Create HEAT stack
+ source /home/jenkins/lf-env.sh
+ lf-activate-venv --python python3 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
++ mktemp -d /tmp/venv-XXXX
+ lf_venv=/tmp/venv-xqIr
+ local venv_file=/tmp/.os_lf_venv
+ local python=python3
+ local options
+ local set_path=true
+ local install_args=
++ getopt -o np:v: -l no-path,system-site-packages,python:,venv-file: -n lf-activate-venv -- --python python3 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
+ options=' --python '\''python3'\'' -- '\''lftools[openstack]'\'' '\''kubernetes'\'' '\''niet'\'' '\''python-heatclient'\'' '\''python-openstackclient'\'' '\''python-magnumclient'\'' '\''yq'\'''
+ eval set -- ' --python '\''python3'\'' -- '\''lftools[openstack]'\'' '\''kubernetes'\'' '\''niet'\'' '\''python-heatclient'\'' '\''python-openstackclient'\'' '\''python-magnumclient'\'' '\''yq'\'''
++ set -- --python python3 -- 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
+ true
+ case $1 in
+ python=python3
+ shift 2
+ true
+ case $1 in
+ shift
+ break
+ case $python in
+ local pkg_list=
+ [[ -d /opt/pyenv ]]
+ echo 'Setup pyenv:'
Setup pyenv:
+ export PYENV_ROOT=/opt/pyenv
+ PYENV_ROOT=/opt/pyenv
+ export PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
+ PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
+ pyenv versions
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-tell-all-calcium/.python-version)
+ command -v pyenv
++ pyenv init - --no-rehash
+ eval 'PATH="$(bash --norc -ec '\''IFS=:; paths=($PATH); 
for i in ${!paths[@]}; do 
if [[ ${paths[i]} == "'\'''\''/opt/pyenv/shims'\'''\''" ]]; then unset '\''\'\'''\''paths[i]'\''\'\'''\''; 
fi; done; 
echo "${paths[*]}"'\'')"
export PATH="/opt/pyenv/shims:${PATH}"
export PYENV_SHELL=bash
source '\''/opt/pyenv/libexec/../completions/pyenv.bash'\''
pyenv() {
  local command
  command="${1:-}"
  if [ "$#" -gt 0 ]; then
    shift
  fi

  case "$command" in
  rehash|shell)
    eval "$(pyenv "sh-$command" "$@")"
    ;;
  *)
    command pyenv "$command" "$@"
    ;;
  esac
}'
+++ bash --norc -ec 'IFS=:; paths=($PATH); 
for i in ${!paths[@]}; do 
if [[ ${paths[i]} == "/opt/pyenv/shims" ]]; then unset '\''paths[i]'\''; 
fi; done; 
echo "${paths[*]}"'
++ PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
++ export PATH=/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
++ PATH=/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
++ export PYENV_SHELL=bash
++ PYENV_SHELL=bash
++ source /opt/pyenv/libexec/../completions/pyenv.bash
+++ complete -F _pyenv pyenv
++ lf-pyver python3
++ local py_version_xy=python3
++ local py_version_xyz=
++ awk '{ print $1 }'
++ pyenv versions
++ local command
++ command=versions
++ '[' 1 -gt 0 ']'
++ shift
++ case "$command" in
++ command pyenv versions
++ pyenv versions
++ sed 's/^[ *]* //'
++ grep -E '^[0-9.]*[0-9]$'
++ [[ ! -s /tmp/.pyenv_versions ]]
+++ grep '^3' /tmp/.pyenv_versions
+++ tail -n 1
+++ sort -V
++ py_version_xyz=3.11.7
++ [[ -z 3.11.7 ]]
++ echo 3.11.7
++ return 0
+ pyenv local 3.11.7
+ local command
+ command=local
+ '[' 2 -gt 0 ']'
+ shift
+ case "$command" in
+ command pyenv local 3.11.7
+ pyenv local 3.11.7
+ for arg in "$@"
+ case $arg in
+ pkg_list+='lftools[openstack] '
+ for arg in "$@"
+ case $arg in
+ pkg_list+='kubernetes '
+ for arg in "$@"
+ case $arg in
+ pkg_list+='niet '
+ for arg in "$@"
+ case $arg in
+ pkg_list+='python-heatclient '
+ for arg in "$@"
+ case $arg in
+ pkg_list+='python-openstackclient '
+ for arg in "$@"
+ case $arg in
+ pkg_list+='python-magnumclient '
+ for arg in "$@"
+ case $arg in
+ pkg_list+='yq '
+ [[ -f /tmp/.os_lf_venv ]]
++ cat /tmp/.os_lf_venv
+ lf_venv=/tmp/venv-QYyr
+ echo 'lf-activate-venv(): INFO: Reuse venv:/tmp/venv-QYyr from' file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-QYyr from file:/tmp/.os_lf_venv
+ /tmp/venv-QYyr/bin/python3 -m pip install --upgrade --quiet pip virtualenv
+ [[ -z lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq  ]]
+ echo 'lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq '
lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq 
+ /tmp/venv-QYyr/bin/python3 -m pip install --upgrade --quiet --upgrade-strategy eager 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
+ type python3
+ true
+ echo 'lf-activate-venv(): INFO: Adding /tmp/venv-QYyr/bin to PATH'
lf-activate-venv(): INFO: Adding /tmp/venv-QYyr/bin to PATH
+ PATH=/tmp/venv-QYyr/bin:/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
+ return 0
+ openstack --os-cloud vex limits show --absolute
+--------------------------+---------+
| Name                     |   Value |
+--------------------------+---------+
| maxTotalInstances        |      -1 |
| maxTotalCores            |     450 |
| maxTotalRAMSize          | 1000000 |
| maxServerMeta            |     128 |
| maxImageMeta             |     128 |
| maxPersonality           |       5 |
| maxPersonalitySize       |   10240 |
| maxTotalKeypairs         |     100 |
| maxServerGroups          |      10 |
| maxServerGroupMembers    |      10 |
| maxTotalFloatingIps      |      -1 |
| maxSecurityGroups        |      -1 |
| maxSecurityGroupRules    |      -1 |
| totalRAMUsed             |  745472 |
| totalCoresUsed           |     182 |
| totalInstancesUsed       |      63 |
| totalFloatingIpsUsed     |       0 |
| totalSecurityGroupsUsed  |       0 |
| totalServerGroupsUsed    |       0 |
| maxTotalVolumes          |      -1 |
| maxTotalSnapshots        |      10 |
| maxTotalVolumeGigabytes  |    4096 |
| maxTotalBackups          |      10 |
| maxTotalBackupGigabytes  |    1000 |
| totalVolumesUsed         |       3 |
| totalGigabytesUsed       |      60 |
| totalSnapshotsUsed       |       0 |
| totalBackupsUsed         |       0 |
| totalBackupGigabytesUsed |       0 |
+--------------------------+---------+
+ pushd /opt/ciman/openstack-hot
/opt/ciman/openstack-hot /w/workspace/controller-csit-3node-clustering-tell-all-calcium
+ lftools openstack --os-cloud vex stack create releng-controller-csit-3node-clustering-tell-all-calcium-76 csit-2-instance-type.yaml /w/workspace/controller-csit-3node-clustering-tell-all-calcium/stack-parameters.yaml
Creating stack releng-controller-csit-3node-clustering-tell-all-calcium-76
Waiting to initialize infrastructure...
Stack initialization successful.
------------------------------------
Stack Details
------------------------------------
{'added': None,
 'capabilities': [],
 'created_at': '2025-02-22T23:43:12Z',
 'deleted': None,
 'deleted_at': None,
 'description': 'No description',
 'environment': None,
 'environment_files': None,
 'files': None,
 'files_container': None,
 'id': 'b2936e65-f3ef-4ed3-ab3f-293059ddc35f',
 'is_rollback_disabled': True,
 'links': [{'href': 'https://orchestration.public.mtl1.vexxhost.net/v1/12c36e260d8e4bb2913965203b1b491f/stacks/releng-controller-csit-3node-clustering-tell-all-calcium-76/b2936e65-f3ef-4ed3-ab3f-293059ddc35f',
            'rel': 'self'}],
 'location': Munch({'cloud': 'vex', 'region_name': 'ca-ymq-1', 'zone': None, 'project': Munch({'id': '12c36e260d8e4bb2913965203b1b491f', 'name': '61975f2c-7c17-4d69-82fa-c3ae420ad6fd', 'domain_id': None, 'domain_name': 'Default'})}),
 'name': 'releng-controller-csit-3node-clustering-tell-all-calcium-76',
 'notification_topics': [],
 'outputs': [{'description': 'IP addresses of the 2nd vm types',
              'output_key': 'vm_1_ips',
              'output_value': []},
             {'description': 'IP addresses of the 1st vm types',
              'output_key': 'vm_0_ips',
              'output_value': ['10.30.171.225',
                               '10.30.170.179',
                               '10.30.171.166']}],
 'owner_id': ****,
 'parameters': {'OS::project_id': '12c36e260d8e4bb2913965203b1b491f',
                'OS::stack_id': 'b2936e65-f3ef-4ed3-ab3f-293059ddc35f',
                'OS::stack_name': 'releng-controller-csit-3node-clustering-tell-all-calcium-76',
                'job_name': '28865-76',
                'silo': 'releng',
                'vm_0_count': '3',
                'vm_0_flavor': 'v3-standard-4',
                'vm_0_image': 'ZZCI - Ubuntu 20.04 - builder - x86_64 - '
                              '20250101-160150.998',
                'vm_1_count': '0',
                'vm_1_flavor': 'v3-standard-2',
                'vm_1_image': 'ZZCI - Ubuntu 22.04 - mininet-ovs-217 - x86_64 '
                              '- 20250201-060151.911'},
 'parent_id': None,
 'replaced': None,
 'status': 'CREATE_COMPLETE',
 'status_reason': 'Stack CREATE completed successfully',
 'tags': [],
 'template': None,
 'template_description': 'No description',
 'template_url': None,
 'timeout_mins': 15,
 'unchanged': None,
 'updated': None,
 'updated_at': None,
 'user_project_id': '7e5b94ebfac74c8b85ee676b3bb0e6f9'}
------------------------------------
+ popd
/w/workspace/controller-csit-3node-clustering-tell-all-calcium
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash -l /tmp/jenkins16912714394412855883.sh
---> Copy SSH public keys to CSIT lab
Setup pyenv:
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-tell-all-calcium/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-QYyr from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes python-heatclient python-openstackclient 
lf-activate-venv(): INFO: Adding /tmp/venv-QYyr/bin to PATH
SSH not responding on 10.30.170.179. Retrying in 10 seconds...
SSH not responding on 10.30.171.166. Retrying in 10 seconds...
SSH not responding on 10.30.171.225. Retrying in 10 seconds...
Ping to 10.30.170.179 successful.
Ping to 10.30.171.166 successful.
Ping to 10.30.171.225 successful.
SSH not responding on 10.30.170.179. Retrying in 10 seconds...
SSH not responding on 10.30.171.166. Retrying in 10 seconds...
SSH not responding on 10.30.171.225. Retrying in 10 seconds...
Ping to 10.30.171.166 successful.
Ping to 10.30.170.179 successful.
Ping to 10.30.171.225 successful.
Warning: Permanently added '10.30.171.166' (ECDSA) to the list of known hosts.
Warning: Permanently added '10.30.170.179' (ECDSA) to the list of known hosts.
Warning: Permanently added '10.30.171.225' (ECDSA) to the list of known hosts.
 releng-28865-76-0-builder-2
Successfully copied public keys to slave 10.30.171.166
 releng-28865-76-0-builder-0
 releng-28865-76-0-builder-1
Successfully copied public keys to slave 10.30.170.179
Successfully copied public keys to slave 10.30.171.225
Process 6494 ready.
Process 6495 ready.
Process 6496 ready.
SSH ready on all stack servers.
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash -l /tmp/jenkins3053259791903405039.sh
Setup pyenv:
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-tell-all-calcium/.python-version)
lf-activate-venv(): INFO: Creating python3 venv at /tmp/venv-zNj0
lf-activate-venv(): INFO: Save venv in file: /w/workspace/controller-csit-3node-clustering-tell-all-calcium/.robot_venv
lf-activate-venv(): INFO: Installing: setuptools wheel 
lf-activate-venv(): INFO: Adding /tmp/venv-zNj0/bin to PATH
+ echo 'Installing Python Requirements'
Installing Python Requirements
+ cat
+ python -m pip install -r requirements.txt
Looking in indexes: https://nexus3.opendaylight.org/repository/PyPi/simple
Collecting docker-py (from -r requirements.txt (line 1))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/docker-py/1.10.6/docker_py-1.10.6-py2.py3-none-any.whl (50 kB)
Collecting ipaddr (from -r requirements.txt (line 2))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/ipaddr/2.2.0/ipaddr-2.2.0.tar.gz (26 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting netaddr (from -r requirements.txt (line 3))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/netaddr/1.3.0/netaddr-1.3.0-py3-none-any.whl (2.3 MB)
Collecting netifaces (from -r requirements.txt (line 4))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/netifaces/0.11.0/netifaces-0.11.0.tar.gz (30 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting pyhocon (from -r requirements.txt (line 5))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/pyhocon/0.3.61/pyhocon-0.3.61-py3-none-any.whl (25 kB)
Collecting requests (from -r requirements.txt (line 6))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/requests/2.32.3/requests-2.32.3-py3-none-any.whl (64 kB)
Collecting robotframework (from -r requirements.txt (line 7))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework/7.2.2/robotframework-7.2.2-py3-none-any.whl (777 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 777.7/777.7 kB 26.0 MB/s eta 0:00:00
Collecting robotframework-httplibrary (from -r requirements.txt (line 8))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-httplibrary/0.4.2/robotframework-httplibrary-0.4.2.tar.gz (9.1 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting robotframework-requests==0.9.7 (from -r requirements.txt (line 9))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-requests/0.9.7/robotframework_requests-0.9.7-py3-none-any.whl (21 kB)
Collecting robotframework-selenium2library (from -r requirements.txt (line 10))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-selenium2library/3.0.0/robotframework_selenium2library-3.0.0-py2.py3-none-any.whl (6.2 kB)
Collecting robotframework-sshlibrary==3.8.0 (from -r requirements.txt (line 11))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-sshlibrary/3.8.0/robotframework-sshlibrary-3.8.0.tar.gz (51 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting scapy (from -r requirements.txt (line 12))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/scapy/2.6.1/scapy-2.6.1-py3-none-any.whl (2.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.4/2.4 MB 45.6 MB/s eta 0:00:00
Collecting jsonpath-rw (from -r requirements.txt (line 15))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/jsonpath-rw/1.4.0/jsonpath-rw-1.4.0.tar.gz (13 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting elasticsearch (from -r requirements.txt (line 18))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elasticsearch/8.17.1/elasticsearch-8.17.1-py3-none-any.whl (653 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 654.0/654.0 kB 33.0 MB/s eta 0:00:00
Collecting elasticsearch-dsl (from -r requirements.txt (line 19))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elasticsearch-dsl/8.17.1/elasticsearch_dsl-8.17.1-py3-none-any.whl (158 kB)
Collecting pyangbind (from -r requirements.txt (line 22))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/pyangbind/0.8.6/pyangbind-0.8.6-py3-none-any.whl (52 kB)
Collecting isodate (from -r requirements.txt (line 25))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/isodate/0.7.2/isodate-0.7.2-py3-none-any.whl (22 kB)
Collecting jmespath (from -r requirements.txt (line 28))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/jmespath/1.0.1/jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting jsonpatch (from -r requirements.txt (line 31))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/jsonpatch/1.33/jsonpatch-1.33-py2.py3-none-any.whl (12 kB)
Collecting paramiko>=1.15.3 (from robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/paramiko/3.5.1/paramiko-3.5.1-py3-none-any.whl (227 kB)
Collecting scp>=0.13.0 (from robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/scp/0.15.0/scp-0.15.0-py2.py3-none-any.whl (8.8 kB)
Collecting docker-pycreds>=0.2.1 (from docker-py->-r requirements.txt (line 1))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/docker-pycreds/0.4.0/docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)
Collecting six>=1.4.0 (from docker-py->-r requirements.txt (line 1))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/six/1.17.0/six-1.17.0-py2.py3-none-any.whl (11 kB)
Collecting websocket-client>=0.32.0 (from docker-py->-r requirements.txt (line 1))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/websocket-client/1.8.0/websocket_client-1.8.0-py3-none-any.whl (58 kB)
Collecting pyparsing<4,>=2 (from pyhocon->-r requirements.txt (line 5))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/pyparsing/3.2.1/pyparsing-3.2.1-py3-none-any.whl (107 kB)
Collecting charset-normalizer<4,>=2 (from requests->-r requirements.txt (line 6))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/charset-normalizer/3.4.1/charset_normalizer-3.4.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (143 kB)
Collecting idna<4,>=2.5 (from requests->-r requirements.txt (line 6))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/idna/3.10/idna-3.10-py3-none-any.whl (70 kB)
Collecting urllib3<3,>=1.21.1 (from requests->-r requirements.txt (line 6))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/urllib3/2.3.0/urllib3-2.3.0-py3-none-any.whl (128 kB)
Collecting certifi>=2017.4.17 (from requests->-r requirements.txt (line 6))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/certifi/2025.1.31/certifi-2025.1.31-py3-none-any.whl (166 kB)
Collecting webtest>=2.0 (from robotframework-httplibrary->-r requirements.txt (line 8))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/webtest/3.0.4/WebTest-3.0.4-py3-none-any.whl (32 kB)
Collecting jsonpointer (from robotframework-httplibrary->-r requirements.txt (line 8))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/jsonpointer/3.0.0/jsonpointer-3.0.0-py2.py3-none-any.whl (7.6 kB)
Collecting robotframework-seleniumlibrary>=3.0.0 (from robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-seleniumlibrary/6.7.0/robotframework_seleniumlibrary-6.7.0-py2.py3-none-any.whl (104 kB)
Collecting ply (from jsonpath-rw->-r requirements.txt (line 15))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/ply/3.11/ply-3.11-py2.py3-none-any.whl (49 kB)
Collecting decorator (from jsonpath-rw->-r requirements.txt (line 15))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/decorator/5.2.0/decorator-5.2.0-py3-none-any.whl (9.1 kB)
Collecting elastic-transport<9,>=8.15.1 (from elasticsearch->-r requirements.txt (line 18))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elastic-transport/8.17.0/elastic_transport-8.17.0-py3-none-any.whl (64 kB)
Collecting python-dateutil (from elasticsearch-dsl->-r requirements.txt (line 19))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/python-dateutil/2.9.0.post0/python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Collecting typing-extensions (from elasticsearch-dsl->-r requirements.txt (line 19))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/typing-extensions/4.12.2/typing_extensions-4.12.2-py3-none-any.whl (37 kB)
Collecting pyang (from pyangbind->-r requirements.txt (line 22))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/pyang/2.6.1/pyang-2.6.1-py2.py3-none-any.whl (594 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 594.7/594.7 kB 24.3 MB/s eta 0:00:00
Collecting lxml (from pyangbind->-r requirements.txt (line 22))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/lxml/5.3.1/lxml-5.3.1-cp311-cp311-manylinux_2_28_x86_64.whl (5.0 MB)
Collecting regex (from pyangbind->-r requirements.txt (line 22))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/regex/2024.11.6/regex-2024.11.6-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (792 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 792.7/792.7 kB 32.1 MB/s eta 0:00:00
Collecting enum34 (from pyangbind->-r requirements.txt (line 22))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/enum34/1.1.10/enum34-1.1.10-py3-none-any.whl (11 kB)
Collecting bcrypt>=3.2 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/bcrypt/4.2.1/bcrypt-4.2.1-cp39-abi3-manylinux_2_28_x86_64.whl (278 kB)
Collecting cryptography>=3.3 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/cryptography/44.0.1/cryptography-44.0.1-cp39-abi3-manylinux_2_28_x86_64.whl (4.2 MB)
Collecting pynacl>=1.5 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/pynacl/1.5.0/PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (856 kB)
Collecting selenium>=4.3.0 (from robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/selenium/4.29.0/selenium-4.29.0-py3-none-any.whl (9.5 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.5/9.5 MB 58.0 MB/s eta 0:00:00
Collecting robotframework-pythonlibcore>=4.4.1 (from robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-pythonlibcore/4.4.1/robotframework_pythonlibcore-4.4.1-py2.py3-none-any.whl (12 kB)
Collecting click>=8.1.7 (from robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/click/8.1.8/click-8.1.8-py3-none-any.whl (98 kB)
Collecting WebOb>=1.2 (from webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/webob/1.8.9/WebOb-1.8.9-py2.py3-none-any.whl (115 kB)
Collecting waitress>=3.0.2 (from webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/waitress/3.0.2/waitress-3.0.2-py3-none-any.whl (56 kB)
Collecting beautifulsoup4 (from webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/beautifulsoup4/4.13.3/beautifulsoup4-4.13.3-py3-none-any.whl (186 kB)
Collecting cffi>=1.12 (from cryptography>=3.3->paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/cffi/1.17.1/cffi-1.17.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (467 kB)
Collecting trio~=0.17 (from selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/trio/0.29.0/trio-0.29.0-py3-none-any.whl (492 kB)
Collecting trio-websocket~=0.9 (from selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/trio-websocket/0.12.1/trio_websocket-0.12.1-py3-none-any.whl (21 kB)
Collecting soupsieve>1.2 (from beautifulsoup4->webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/soupsieve/2.6/soupsieve-2.6-py3-none-any.whl (36 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=3.3->paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/pycparser/2.22/pycparser-2.22-py3-none-any.whl (117 kB)
Collecting attrs>=23.2.0 (from trio~=0.17->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/attrs/25.1.0/attrs-25.1.0-py3-none-any.whl (63 kB)
Collecting sortedcontainers (from trio~=0.17->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/sortedcontainers/2.4.0/sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting outcome (from trio~=0.17->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/outcome/1.3.0.post0/outcome-1.3.0.post0-py2.py3-none-any.whl (10 kB)
Collecting sniffio>=1.3.0 (from trio~=0.17->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/sniffio/1.3.1/sniffio-1.3.1-py3-none-any.whl (10 kB)
Collecting wsproto>=0.14 (from trio-websocket~=0.9->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/wsproto/1.2.0/wsproto-1.2.0-py3-none-any.whl (24 kB)
Collecting pysocks!=1.5.7,<2.0,>=1.5.6 (from urllib3[socks]<3,>=1.26->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/pysocks/1.7.1/PySocks-1.7.1-py3-none-any.whl (16 kB)
Collecting h11<1,>=0.9.0 (from wsproto>=0.14->trio-websocket~=0.9->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/h11/0.14.0/h11-0.14.0-py3-none-any.whl (58 kB)
Building wheels for collected packages: robotframework-sshlibrary, ipaddr, netifaces, robotframework-httplibrary, jsonpath-rw
  Building wheel for robotframework-sshlibrary (setup.py): started
  Building wheel for robotframework-sshlibrary (setup.py): finished with status 'done'
  Created wheel for robotframework-sshlibrary: filename=robotframework_sshlibrary-3.8.0-py3-none-any.whl size=55176 sha256=0a6a7b22c765a94367f03098aa22c0c750df5f3cf9db420b240790bcb2b9a99c
  Stored in directory: /home/jenkins/.cache/pip/wheels/f7/c9/b3/a977b7bcc410d45ae27d240df3d00a12585509180e373ecccc
  Building wheel for ipaddr (setup.py): started
  Building wheel for ipaddr (setup.py): finished with status 'done'
  Created wheel for ipaddr: filename=ipaddr-2.2.0-py3-none-any.whl size=18322 sha256=d1503235afd2f5698fbb4f6f5d1defc5bbfb6bcef3f9c39161dac085ef8d9719
  Stored in directory: /home/jenkins/.cache/pip/wheels/dc/6c/04/da2d847fa8d45c59af3e1d83e2acc29cb8adcbaf04c0898dbf
  Building wheel for netifaces (setup.py): started
  Building wheel for netifaces (setup.py): finished with status 'done'
  Created wheel for netifaces: filename=netifaces-0.11.0-cp311-cp311-linux_x86_64.whl size=41039 sha256=7ceb31682739d4e803cf30f1aaad7f6eb7b651338f158f3216b501bbe05d5cd0
  Stored in directory: /home/jenkins/.cache/pip/wheels/f8/18/88/e61d54b995bea304bdb1d040a92b72228a1bf72ca2a3eba7c9
  Building wheel for robotframework-httplibrary (setup.py): started
  Building wheel for robotframework-httplibrary (setup.py): finished with status 'done'
  Created wheel for robotframework-httplibrary: filename=robotframework_httplibrary-0.4.2-py3-none-any.whl size=10013 sha256=016f7835b206bfb8334d3de4d9b03600d23e3625d09512e79ca2cbbf0cb47b8b
  Stored in directory: /home/jenkins/.cache/pip/wheels/aa/bc/0d/9a20dd51effef392aae2733cb4c7b66c6fa29fca33d88b57ed
  Building wheel for jsonpath-rw (setup.py): started
  Building wheel for jsonpath-rw (setup.py): finished with status 'done'
  Created wheel for jsonpath-rw: filename=jsonpath_rw-1.4.0-py3-none-any.whl size=15176 sha256=590e62aa70b55f516541257d1c636413d21abc61336f9da8f0a79a15e5204ccf
  Stored in directory: /home/jenkins/.cache/pip/wheels/f1/54/63/9a8da38cefae13755097b36cc852decc25d8ef69c37d58d4eb
Successfully built robotframework-sshlibrary ipaddr netifaces robotframework-httplibrary jsonpath-rw
Installing collected packages: sortedcontainers, ply, netifaces, ipaddr, enum34, websocket-client, WebOb, waitress, urllib3, typing-extensions, soupsieve, sniffio, six, scapy, robotframework-pythonlibcore, robotframework, regex, pysocks, pyparsing, pycparser, netaddr, lxml, jsonpointer, jmespath, isodate, idna, h11, decorator, click, charset-normalizer, certifi, bcrypt, attrs, wsproto, requests, python-dateutil, pyhocon, pyang, outcome, jsonpath-rw, jsonpatch, elastic-transport, docker-pycreds, cffi, beautifulsoup4, webtest, trio, robotframework-requests, pynacl, pyangbind, elasticsearch, docker-py, cryptography, trio-websocket, robotframework-httplibrary, paramiko, elasticsearch-dsl, selenium, scp, robotframework-sshlibrary, robotframework-seleniumlibrary, robotframework-selenium2library
Successfully installed WebOb-1.8.9 attrs-25.1.0 bcrypt-4.2.1 beautifulsoup4-4.13.3 certifi-2025.1.31 cffi-1.17.1 charset-normalizer-3.4.1 click-8.1.8 cryptography-44.0.1 decorator-5.2.0 docker-py-1.10.6 docker-pycreds-0.4.0 elastic-transport-8.17.0 elasticsearch-8.17.1 elasticsearch-dsl-8.17.1 enum34-1.1.10 h11-0.14.0 idna-3.10 ipaddr-2.2.0 isodate-0.7.2 jmespath-1.0.1 jsonpatch-1.33 jsonpath-rw-1.4.0 jsonpointer-3.0.0 lxml-5.3.1 netaddr-1.3.0 netifaces-0.11.0 outcome-1.3.0.post0 paramiko-3.5.1 ply-3.11 pyang-2.6.1 pyangbind-0.8.6 pycparser-2.22 pyhocon-0.3.61 pynacl-1.5.0 pyparsing-3.2.1 pysocks-1.7.1 python-dateutil-2.9.0.post0 regex-2024.11.6 requests-2.32.3 robotframework-7.2.2 robotframework-httplibrary-0.4.2 robotframework-pythonlibcore-4.4.1 robotframework-requests-0.9.7 robotframework-selenium2library-3.0.0 robotframework-seleniumlibrary-6.7.0 robotframework-sshlibrary-3.8.0 scapy-2.6.1 scp-0.15.0 selenium-4.29.0 six-1.17.0 sniffio-1.3.1 sortedcontainers-2.4.0 soupsieve-2.6 trio-0.29.0 trio-websocket-0.12.1 typing-extensions-4.12.2 urllib3-2.3.0 waitress-3.0.2 websocket-client-1.8.0 webtest-3.0.4 wsproto-1.2.0
+ pip freeze
attrs==25.1.0
bcrypt==4.2.1
beautifulsoup4==4.13.3
certifi==2025.1.31
cffi==1.17.1
charset-normalizer==3.4.1
click==8.1.8
cryptography==44.0.1
decorator==5.2.0
distlib==0.3.9
docker-py==1.10.6
docker-pycreds==0.4.0
elastic-transport==8.17.0
elasticsearch==8.17.1
elasticsearch-dsl==8.17.1
enum34==1.1.10
filelock==3.17.0
h11==0.14.0
idna==3.10
ipaddr==2.2.0
isodate==0.7.2
jmespath==1.0.1
jsonpatch==1.33
jsonpath-rw==1.4.0
jsonpointer==3.0.0
lxml==5.3.1
netaddr==1.3.0
netifaces==0.11.0
outcome==1.3.0.post0
paramiko==3.5.1
platformdirs==4.3.6
ply==3.11
pyang==2.6.1
pyangbind==0.8.6
pycparser==2.22
pyhocon==0.3.61
PyNaCl==1.5.0
pyparsing==3.2.1
PySocks==1.7.1
python-dateutil==2.9.0.post0
regex==2024.11.6
requests==2.32.3
robotframework==7.2.2
robotframework-httplibrary==0.4.2
robotframework-pythonlibcore==4.4.1
robotframework-requests==0.9.7
robotframework-selenium2library==3.0.0
robotframework-seleniumlibrary==6.7.0
robotframework-sshlibrary==3.8.0
scapy==2.6.1
scp==0.15.0
selenium==4.29.0
six==1.17.0
sniffio==1.3.1
sortedcontainers==2.4.0
soupsieve==2.6
trio==0.29.0
trio-websocket==0.12.1
typing_extensions==4.12.2
urllib3==2.3.0
virtualenv==20.29.2
waitress==3.0.2
WebOb==1.8.9
websocket-client==1.8.0
WebTest==3.0.4
wsproto==1.2.0
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties file path 'env.properties'
[EnvInject] - Variables injected successfully.
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash -l /tmp/jenkins6315792549982766478.sh
Setup pyenv:
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-tell-all-calcium/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-QYyr from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing: python-heatclient python-openstackclient yq 
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
lftools 0.37.10 requires urllib3<2.1.0, but you have urllib3 2.3.0 which is incompatible.
lf-activate-venv(): INFO: Adding /tmp/venv-QYyr/bin to PATH
+ ODL_SYSTEM=()
+ TOOLS_SYSTEM=()
+ OPENSTACK_SYSTEM=()
+ OPENSTACK_CONTROLLERS=()
+ mapfile -t ADDR
++ openstack stack show -f json -c outputs releng-controller-csit-3node-clustering-tell-all-calcium-76
++ jq -r '.outputs[] | select(.output_key | match("^vm_[0-9]+_ips$")) | .output_value | .[]'
+ for i in "${ADDR[@]}"
++ ssh 10.30.171.225 hostname -s
Warning: Permanently added '10.30.171.225' (ECDSA) to the list of known hosts.
+ REMHOST=releng-28865-76-0-builder-0
+ case ${REMHOST} in
+ ODL_SYSTEM=("${ODL_SYSTEM[@]}" "${i}")
+ for i in "${ADDR[@]}"
++ ssh 10.30.170.179 hostname -s
Warning: Permanently added '10.30.170.179' (ECDSA) to the list of known hosts.
+ REMHOST=releng-28865-76-0-builder-1
+ case ${REMHOST} in
+ ODL_SYSTEM=("${ODL_SYSTEM[@]}" "${i}")
+ for i in "${ADDR[@]}"
++ ssh 10.30.171.166 hostname -s
Warning: Permanently added '10.30.171.166' (ECDSA) to the list of known hosts.
+ REMHOST=releng-28865-76-0-builder-2
+ case ${REMHOST} in
+ ODL_SYSTEM=("${ODL_SYSTEM[@]}" "${i}")
+ echo NUM_ODL_SYSTEM=3
+ echo NUM_TOOLS_SYSTEM=0
+ '[' '' == yes ']'
+ NUM_OPENSTACK_SYSTEM=0
+ echo NUM_OPENSTACK_SYSTEM=0
+ '[' 0 -eq 2 ']'
+ echo ODL_SYSTEM_IP=10.30.171.225
++ seq 0 2
+ for i in $(seq 0 $(( ${#ODL_SYSTEM[@]} - 1 )))
+ echo ODL_SYSTEM_1_IP=10.30.171.225
+ for i in $(seq 0 $(( ${#ODL_SYSTEM[@]} - 1 )))
+ echo ODL_SYSTEM_2_IP=10.30.170.179
+ for i in $(seq 0 $(( ${#ODL_SYSTEM[@]} - 1 )))
+ echo ODL_SYSTEM_3_IP=10.30.171.166
+ echo TOOLS_SYSTEM_IP=
++ seq 0 -1
+ openstack_index=0
+ NUM_OPENSTACK_CONTROL_NODES=1
+ echo NUM_OPENSTACK_CONTROL_NODES=1
++ seq 0 0
+ for i in $(seq 0 $((NUM_OPENSTACK_CONTROL_NODES - 1)))
+ echo OPENSTACK_CONTROL_NODE_1_IP=
+ NUM_OPENSTACK_COMPUTE_NODES=-1
+ echo NUM_OPENSTACK_COMPUTE_NODES=-1
+ '[' -1 -ge 2 ']'
++ seq 0 -2
+ NUM_OPENSTACK_HAPROXY_NODES=0
+ echo NUM_OPENSTACK_HAPROXY_NODES=0
++ seq 0 -1
+ echo 'Contents of slave_addresses.txt:'
Contents of slave_addresses.txt:
+ cat slave_addresses.txt
NUM_ODL_SYSTEM=3
NUM_TOOLS_SYSTEM=0
NUM_OPENSTACK_SYSTEM=0
ODL_SYSTEM_IP=10.30.171.225
ODL_SYSTEM_1_IP=10.30.171.225
ODL_SYSTEM_2_IP=10.30.170.179
ODL_SYSTEM_3_IP=10.30.171.166
TOOLS_SYSTEM_IP=
NUM_OPENSTACK_CONTROL_NODES=1
OPENSTACK_CONTROL_NODE_1_IP=
NUM_OPENSTACK_COMPUTE_NODES=-1
NUM_OPENSTACK_HAPROXY_NODES=0
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties file path 'slave_addresses.txt'
[EnvInject] - Variables injected successfully.
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/sh /tmp/jenkins9106441686012021565.sh
Preparing for JRE Version 17
Karaf artifact is karaf
Karaf project is integration
Java home is /usr/lib/jvm/java-17-openjdk-amd64
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties file path 'set_variables.env'
[EnvInject] - Variables injected successfully.
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins6576365139364145140.sh
2025-02-22 23:44:52 URL:https://git.opendaylight.org/gerrit/gitweb?p=integration/distribution.git;a=blob_plain;f=pom.xml;hb=stable/calcium [2619/2619] -> "pom.xml" [1]
Bundle version is 0.20.4-SNAPSHOT
--2025-02-22 23:44:52--  https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.4-SNAPSHOT/maven-metadata.xml
Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6
Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1410 (1.4K) [application/xml]
Saving to: ‘maven-metadata.xml’

     0K .                                                     100% 53.1M=0s

2025-02-22 23:44:52 (53.1 MB/s) - ‘maven-metadata.xml’ saved [1410/1410]

<?xml version="1.0" encoding="UTF-8"?>
<metadata modelVersion="1.1.0">
  <groupId>org.opendaylight.integration</groupId>
  <artifactId>karaf</artifactId>
  <version>0.20.4-SNAPSHOT</version>
  <versioning>
    <snapshot>
      <timestamp>20250222.204906</timestamp>
      <buildNumber>381</buildNumber>
    </snapshot>
    <lastUpdated>20250222204906</lastUpdated>
    <snapshotVersions>
      <snapshotVersion>
        <extension>pom</extension>
        <value>0.20.4-20250222.204906-381</value>
        <updated>20250222204906</updated>
      </snapshotVersion>
      <snapshotVersion>
        <extension>tar.gz</extension>
        <value>0.20.4-20250222.204906-381</value>
        <updated>20250222204906</updated>
      </snapshotVersion>
      <snapshotVersion>
        <extension>zip</extension>
        <value>0.20.4-20250222.204906-381</value>
        <updated>20250222204906</updated>
      </snapshotVersion>
      <snapshotVersion>
        <classifier>cyclonedx</classifier>
        <extension>xml</extension>
        <value>0.20.4-20250222.204906-381</value>
        <updated>20250222204906</updated>
      </snapshotVersion>
      <snapshotVersion>
        <classifier>cyclonedx</classifier>
        <extension>json</extension>
        <value>0.20.4-20250222.204906-381</value>
        <updated>20250222204906</updated>
      </snapshotVersion>
    </snapshotVersions>
  </versioning>
</metadata>
Nexus timestamp is 0.20.4-20250222.204906-381
Distribution bundle URL is https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.4-SNAPSHOT/karaf-0.20.4-20250222.204906-381.zip
Distribution bundle is karaf-0.20.4-20250222.204906-381.zip
Distribution bundle version is 0.20.4-SNAPSHOT
Distribution folder is karaf-0.20.4-SNAPSHOT
Nexus prefix is https://nexus.opendaylight.org
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties file path 'detect_variables.env'
[EnvInject] - Variables injected successfully.
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash -l /tmp/jenkins18366000595003967903.sh
Setup pyenv:
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-tell-all-calcium/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-QYyr from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing: python-heatclient python-openstackclient 
lf-activate-venv(): INFO: Adding /tmp/venv-QYyr/bin to PATH
Copying common-functions.sh to /tmp
Copying common-functions.sh to 10.30.171.225:/tmp
Warning: Permanently added '10.30.171.225' (ECDSA) to the list of known hosts.
Copying common-functions.sh to 10.30.170.179:/tmp
Warning: Permanently added '10.30.170.179' (ECDSA) to the list of known hosts.
Copying common-functions.sh to 10.30.171.166:/tmp
Warning: Permanently added '10.30.171.166' (ECDSA) to the list of known hosts.
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins4597404242915310811.sh
common-functions.sh is being sourced
common-functions environment:
MAVENCONF: /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
ACTUALFEATURES: 
FEATURESCONF: /tmp/karaf-0.20.4-SNAPSHOT/etc/org.apache.karaf.features.cfg
CUSTOMPROP: /tmp/karaf-0.20.4-SNAPSHOT/etc/custom.properties
LOGCONF: /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
MEMCONF: /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
CONTROLLERMEM: 2048m
AKKACONF: /tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/akka.conf
MODULESCONF: /tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/modules.conf
MODULESHARDSCONF: /tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/module-shards.conf
SUITES: 

#################################################
##         Configure Cluster and Start         ##
#################################################
ACTUALFEATURES: odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app
SPACE_SEPARATED_FEATURES: odl-integration-compatible-with-all odl-jolokia odl-restconf odl-clustering-test-app
Locating script plan to use...
script plan exists!!!
Changing the script plan path...
# Place the scripts in run order:
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/scripts/car_people_shard.sh
Executing /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/scripts/car_people_shard.sh...
Add car-people shards file
Copy shard config to member-1 with IP address 10.30.171.225
Warning: Permanently added '10.30.171.225' (ECDSA) to the list of known hosts.
Copy shard config to member-2 with IP address 10.30.170.179
Warning: Permanently added '10.30.170.179' (ECDSA) to the list of known hosts.
Copy shard config to member-3 with IP address 10.30.171.166
Warning: Permanently added '10.30.171.166' (ECDSA) to the list of known hosts.
Finished running script plans
Configuring member-1 with IP address 10.30.171.225
Warning: Permanently added '10.30.171.225' (ECDSA) to the list of known hosts.
Warning: Permanently added '10.30.171.225' (ECDSA) to the list of known hosts.
common-functions.sh is being sourced
+ source /tmp/common-functions.sh karaf-0.20.4-SNAPSHOT
++ [[ /tmp/common-functions.sh == \/\t\m\p\/\c\o\n\f\i\g\u\r\a\t\i\o\n\-\s\c\r\i\p\t\.\s\h ]]
++ echo 'common-functions.sh is being sourced'
++ BUNDLEFOLDER=karaf-0.20.4-SNAPSHOT
++ export MAVENCONF=/tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
++ MAVENCONF=/tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
++ export FEATURESCONF=/tmp/karaf-0.20.4-SNAPSHOT/etc/org.apache.karaf.features.cfg
++ FEATURESCONF=/tmp/karaf-0.20.4-SNAPSHOT/etc/org.apache.karaf.features.cfg
++ export CUSTOMPROP=/tmp/karaf-0.20.4-SNAPSHOT/etc/custom.properties
++ CUSTOMPROP=/tmp/karaf-0.20.4-SNAPSHOT/etc/custom.properties
++ export LOGCONF=/tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
++ LOGCONF=/tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
++ export MEMCONF=/tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
++ MEMCONF=/tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
++ export CONTROLLERMEM=
++ CONTROLLERMEM=
++ export AKKACONF=/tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/akka.conf
++ AKKACONF=/tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/akka.conf
++ export MODULESCONF=/tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/modules.conf
++ MODULESCONF=/tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/modules.conf
++ export MODULESHARDSCONF=/tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/module-shards.conf
++ MODULESHARDSCONF=/tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/module-shards.conf
++ print_common_env
++ cat
common-functions environment:
MAVENCONF: /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
ACTUALFEATURES: 
FEATURESCONF: /tmp/karaf-0.20.4-SNAPSHOT/etc/org.apache.karaf.features.cfg
CUSTOMPROP: /tmp/karaf-0.20.4-SNAPSHOT/etc/custom.properties
LOGCONF: /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
MEMCONF: /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
CONTROLLERMEM: 
AKKACONF: /tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/akka.conf
MODULESCONF: /tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/modules.conf
MODULESHARDSCONF: /tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/module-shards.conf
SUITES: 

++ SSH='ssh -t -t'
++ extra_services_cntl='     dnsmasq.service     httpd.service     libvirtd.service     openvswitch.service     ovs-vswitchd.service     ovsdb-server.service     rabbitmq-server.service '
++ extra_services_cmp='     libvirtd.service     openvswitch.service     ovs-vswitchd.service     ovsdb-server.service '
Changing to /tmp
Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.4-SNAPSHOT/karaf-0.20.4-20250222.204906-381.zip
+ echo 'Changing to /tmp'
+ cd /tmp
+ echo 'Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.4-SNAPSHOT/karaf-0.20.4-20250222.204906-381.zip'
+ wget --progress=dot:mega https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.4-SNAPSHOT/karaf-0.20.4-20250222.204906-381.zip
--2025-02-22 23:45:01--  https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.4-SNAPSHOT/karaf-0.20.4-20250222.204906-381.zip
Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6
Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 234245297 (223M) [application/zip]
Saving to: ‘karaf-0.20.4-20250222.204906-381.zip’

     0K ........ ........ ........ ........ ........ ........  1% 65.5M 3s
  3072K ........ ........ ........ ........ ........ ........  2%  117M 3s
  6144K ........ ........ ........ ........ ........ ........  4%  137M 2s
  9216K ........ ........ ........ ........ ........ ........  5%  134M 2s
 12288K ........ ........ ........ ........ ........ ........  6%  166M 2s
 15360K ........ ........ ........ ........ ........ ........  8%  186M 2s
 18432K ........ ........ ........ ........ ........ ........  9%  199M 2s
 21504K ........ ........ ........ ........ ........ ........ 10%  242M 1s
 24576K ........ ........ ........ ........ ........ ........ 12%  239M 1s
 27648K ........ ........ ........ ........ ........ ........ 13%  248M 1s
 30720K ........ ........ ........ ........ ........ ........ 14%  239M 1s
 33792K ........ ........ ........ ........ ........ ........ 16%  259M 1s
 36864K ........ ........ ........ ........ ........ ........ 17%  240M 1s
 39936K ........ ........ ........ ........ ........ ........ 18%  413M 1s
 43008K ........ ........ ........ ........ ........ ........ 20%  348M 1s
 46080K ........ ........ ........ ........ ........ ........ 21%  325M 1s
 49152K ........ ........ ........ ........ ........ ........ 22%  302M 1s
 52224K ........ ........ ........ ........ ........ ........ 24%  276M 1s
 55296K ........ ........ ........ ........ ........ ........ 25%  225M 1s
 58368K ........ ........ ........ ........ ........ ........ 26%  310M 1s
 61440K ........ ........ ........ ........ ........ ........ 28%  269M 1s
 64512K ........ ........ ........ ........ ........ ........ 29%  354M 1s
 67584K ........ ........ ........ ........ ........ ........ 30%  258M 1s
 70656K ........ ........ ........ ........ ........ ........ 32%  248M 1s
 73728K ........ ........ ........ ........ ........ ........ 33%  324M 1s
 76800K ........ ........ ........ ........ ........ ........ 34%  355M 1s
 79872K ........ ........ ........ ........ ........ ........ 36%  254M 1s
 82944K ........ ........ ........ ........ ........ ........ 37%  270M 1s
 86016K ........ ........ ........ ........ ........ ........ 38%  359M 1s
 89088K ........ ........ ........ ........ ........ ........ 40%  291M 1s
 92160K ........ ........ ........ ........ ........ ........ 41%  254M 1s
 95232K ........ ........ ........ ........ ........ ........ 42%  343M 1s
 98304K ........ ........ ........ ........ ........ ........ 44%  341M 1s
101376K ........ ........ ........ ........ ........ ........ 45%  365M 1s
104448K ........ ........ ........ ........ ........ ........ 47%  370M 1s
107520K ........ ........ ........ ........ ........ ........ 48%  292M 0s
110592K ........ ........ ........ ........ ........ ........ 49%  348M 0s
113664K ........ ........ ........ ........ ........ ........ 51%  304M 0s
116736K ........ ........ ........ ........ ........ ........ 52%  370M 0s
119808K ........ ........ ........ ........ ........ ........ 53%  360M 0s
122880K ........ ........ ........ ........ ........ ........ 55%  371M 0s
125952K ........ ........ ........ ........ ........ ........ 56%  369M 0s
129024K ........ ........ ........ ........ ........ ........ 57%  386M 0s
132096K ........ ........ ........ ........ ........ ........ 59%  265M 0s
135168K ........ ........ ........ ........ ........ ........ 60%  323M 0s
138240K ........ ........ ........ ........ ........ ........ 61%  363M 0s
141312K ........ ........ ........ ........ ........ ........ 63%  398M 0s
144384K ........ ........ ........ ........ ........ ........ 64%  352M 0s
147456K ........ ........ ........ ........ ........ ........ 65%  379M 0s
150528K ........ ........ ........ ........ ........ ........ 67%  360M 0s
153600K ........ ........ ........ ........ ........ ........ 68%  368M 0s
156672K ........ ........ ........ ........ ........ ........ 69%  349M 0s
159744K ........ ........ ........ ........ ........ ........ 71%  379M 0s
162816K ........ ........ ........ ........ ........ ........ 72%  277M 0s
165888K ........ ........ ........ ........ ........ ........ 73%  330M 0s
168960K ........ ........ ........ ........ ........ ........ 75%  313M 0s
172032K ........ ........ ........ ........ ........ ........ 76%  372M 0s
175104K ........ ........ ........ ........ ........ ........ 77%  347M 0s
178176K ........ ........ ........ ........ ........ ........ 79%  378M 0s
181248K ........ ........ ........ ........ ........ ........ 80%  359M 0s
184320K ........ ........ ........ ........ ........ ........ 81%  367M 0s
187392K ........ ........ ........ ........ ........ ........ 83%  370M 0s
190464K ........ ........ ........ ........ ........ ........ 84%  278M 0s
193536K ........ ........ ........ ........ ........ ........ 85%  307M 0s
196608K ........ ........ ........ ........ ........ ........ 87%  313M 0s
199680K ........ ........ ........ ........ ........ ........ 88%  442M 0s
202752K ........ ........ ........ ........ ........ ........ 89%  387M 0s
205824K ........ ........ ........ ........ ........ ........ 91%  273M 0s
208896K ........ ........ ........ ........ ........ ........ 92%  360M 0s
211968K ........ ........ ........ ........ ........ ........ 94%  361M 0s
215040K ........ ........ ........ ........ ........ ........ 95%  374M 0s
218112K ........ ........ ........ ........ ........ ........ 96%  355M 0s
221184K ........ ........ ........ ........ ........ ........ 98%  370M 0s
224256K ........ ........ ........ ........ ........ ........ 99%  363M 0s
227328K ........ ........ ......                             100%  398M=0.8s

2025-02-22 23:45:02 (280 MB/s) - ‘karaf-0.20.4-20250222.204906-381.zip’ saved [234245297/234245297]

Extracting the new controller...
+ echo 'Extracting the new controller...'
+ unzip -q karaf-0.20.4-20250222.204906-381.zip
Adding external repositories...
+ echo 'Adding external repositories...'
+ sed -ie 's%org.ops4j.pax.url.mvn.repositories=%org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases%g' /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
+ cat /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
################################################################################
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#
################################################################################

#
# If set to true, the following property will not allow any certificate to be used
# when accessing Maven repositories through SSL
#
#org.ops4j.pax.url.mvn.certificateCheck=

#
# Path to the local Maven settings file.
# The repositories defined in this file will be automatically added to the list
# of default repositories if the 'org.ops4j.pax.url.mvn.repositories' property
# below is not set.
# The following locations are checked for the existence of the settings.xml file
#   * 1. looks for the specified url
#   * 2. if not found looks for ${user.home}/.m2/settings.xml
#   * 3. if not found looks for ${maven.home}/conf/settings.xml
#   * 4. if not found looks for ${M2_HOME}/conf/settings.xml
#
#org.ops4j.pax.url.mvn.settings=

#
# Path to the local Maven repository which is used to avoid downloading
# artifacts when they already exist locally.
# The value of this property will be extracted from the settings.xml file
# above, or defaulted to:
#     System.getProperty( "user.home" ) + "/.m2/repository"
#
org.ops4j.pax.url.mvn.localRepository=${karaf.home}/${karaf.default.repository}

#
# Default this to false. It's just weird to use undocumented repos
#
org.ops4j.pax.url.mvn.useFallbackRepositories=false

#
# Uncomment if you don't wanna use the proxy settings
# from the Maven conf/settings.xml file
#
# org.ops4j.pax.url.mvn.proxySupport=false

#
# Comma separated list of repositories scanned when resolving an artifact.
# Those repositories will be checked before iterating through the
#    below list of repositories and even before the local repository
# A repository url can be appended with zero or more of the following flags:
#    @snapshots  : the repository contains snaphots
#    @noreleases : the repository does not contain any released artifacts
#
# The following property value will add the system folder as a repo.
#
org.ops4j.pax.url.mvn.defaultRepositories=\
    file:${karaf.home}/${karaf.default.repository}@id=system.repository@snapshots,\
    file:${karaf.data}/kar@id=kar.repository@multi@snapshots,\
    file:${karaf.base}/${karaf.default.repository}@id=child.system.repository@snapshots

# Use the default local repo (e.g.~/.m2/repository) as a "remote" repo
#org.ops4j.pax.url.mvn.defaultLocalRepoAsRemote=false

#
# Comma separated list of repositories scanned when resolving an artifact.
# The default list includes the following repositories:
#    http://repo1.maven.org/maven2@id=central
#    http://repository.springsource.com/maven/bundles/release@id=spring.ebr
#    http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external
#    http://zodiac.springsource.com/maven/bundles/release@id=gemini
#    http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases
#    https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases
#    https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases
# To add repositories to the default ones, prepend '+' to the list of repositories
# to add.
# A repository url can be appended with zero or more of the following flags:
#    @snapshots  : the repository contains snapshots
#    @noreleases : the repository does not contain any released artifacts
#    @id=repository.id : the id for the repository, just like in the settings.xml this is optional but recommended
#
org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases

### ^^^ No remote repositories. This is the only ODL change compared to Karaf defaults.Configuring the startup features...
+ [[ True == \T\r\u\e ]]
+ echo 'Configuring the startup features...'
+ sed -ie 's/\(featuresBoot=\|featuresBoot =\)/featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app,/g' /tmp/karaf-0.20.4-SNAPSHOT/etc/org.apache.karaf.features.cfg
+ FEATURE_TEST_STRING=features-test
+ FEATURE_TEST_VERSION=0.20.4-SNAPSHOT
+ KARAF_VERSION=karaf4
+ [[ integration == \i\n\t\e\g\r\a\t\i\o\n ]]
+ sed -ie 's%\(featuresRepositories=\|featuresRepositories =\)%featuresRepositories = mvn:org.opendaylight.integration/features-test/0.20.4-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features,%g' /tmp/karaf-0.20.4-SNAPSHOT/etc/org.apache.karaf.features.cfg
+ [[ ! -z '' ]]
+ cat /tmp/karaf-0.20.4-SNAPSHOT/etc/org.apache.karaf.features.cfg
################################################################################
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#
################################################################################

#
# Comma separated list of features repositories to register by default
#
featuresRepositories = mvn:org.opendaylight.integration/features-test/0.20.4-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features, file:${karaf.etc}/1770ed62-ed0a-4998-9f53-555fbe661aac.xml

#
# Comma separated list of features to install at startup
#
featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app, 185dc51d-e1fe-4d2d-bb8f-35ce328d82cf

#
# Resource repositories (OBR) that the features resolver can use
# to resolve requirements/capabilities
#
# The format of the resourceRepositories is 
# resourceRepositories=[xml:url|json:url],...
# for Instance:
#
#resourceRepositories=xml:http://host/path/to/index.xml
# or
#resourceRepositories=json:http://host/path/to/index.json
#

#
# Defines if the boot features are started in asynchronous mode (in a dedicated thread)
#
featuresBootAsynchronous=false

#
# Service requirements enforcement
#
# By default, the feature resolver checks the service requirements/capabilities of
# bundles for new features (xml schema >= 1.3.0) in order to automatically installs
# the required bundles.
# The following flag can have those values:
#   - disable: service requirements are completely ignored
#   - default: service requirements are ignored for old features
#   - enforce: service requirements are always verified
#
#serviceRequirements=default

#
# Store cfg file for config element in feature
#
#configCfgStore=true

#
# Define if the feature service automatically refresh bundles
#
autoRefresh=true

#
# Configuration of features processing mechanism (overrides, blacklisting, modification of features)
# XML file defines instructions related to features processing
# versions.properties may declare properties to resolve placeholders in XML file
# both files are relative to ${karaf.etc}
#
#featureProcessing=org.apache.karaf.features.xml
#featureProcessingVersions=versions.properties
+ configure_karaf_log karaf4 ''
+ local -r karaf_version=karaf4
+ local -r controllerdebugmap=
+ local logapi=log4j
+ grep log4j2 /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
log4j2.rootLogger.level = INFO
#log4j2.rootLogger.type = asyncRoot
#log4j2.rootLogger.includeLocation = false
log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile
log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi
log4j2.rootLogger.appenderRef.Console.ref = Console
log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter
log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF}
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL
log4j2.logger.spifly.name = org.apache.aries.spifly
log4j2.logger.spifly.level = WARN
log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit
log4j2.logger.audit.level = INFO
log4j2.logger.audit.additivity = false
log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile
# Console appender not used by default (see log4j2.rootLogger.appenderRefs)
log4j2.appender.console.type = Console
log4j2.appender.console.name = Console
log4j2.appender.console.layout.type = PatternLayout
log4j2.appender.console.layout.pattern = ${log4j2.pattern}
log4j2.appender.rolling.type = RollingRandomAccessFile
log4j2.appender.rolling.name = RollingFile
log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i
#log4j2.appender.rolling.immediateFlush = false
log4j2.appender.rolling.append = true
log4j2.appender.rolling.layout.type = PatternLayout
log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
log4j2.appender.rolling.policies.type = Policies
log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.rolling.policies.size.size = 64MB
log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
log4j2.appender.rolling.strategy.max = 7
log4j2.appender.audit.type = RollingRandomAccessFile
log4j2.appender.audit.name = AuditRollingFile
log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log
log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i
log4j2.appender.audit.append = true
log4j2.appender.audit.layout.type = PatternLayout
log4j2.appender.audit.layout.pattern = ${log4j2.pattern}
log4j2.appender.audit.policies.type = Policies
log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.audit.policies.size.size = 8MB
log4j2.appender.audit.strategy.type = DefaultRolloverStrategy
log4j2.appender.audit.strategy.max = 7
log4j2.appender.osgi.type = PaxOsgi
log4j2.appender.osgi.name = PaxOsgi
log4j2.appender.osgi.filter = *
#log4j2.logger.aether.name = shaded.org.eclipse.aether
#log4j2.logger.aether.level = TRACE
#log4j2.logger.http-headers.name = shaded.org.apache.http.headers
#log4j2.logger.http-headers.level = DEBUG
#log4j2.logger.maven.name = org.ops4j.pax.url.mvn
#log4j2.logger.maven.level = TRACE
+ logapi=log4j2
+ echo 'Configuring the karaf log... karaf_version: karaf4, logapi: log4j2'
Configuring the karaf log... karaf_version: karaf4, logapi: log4j2
+ '[' log4j2 == log4j2 ']'
+ sed -ie 's/log4j2.appender.rolling.policies.size.size = 64MB/log4j2.appender.rolling.policies.size.size = 1GB/g' /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
controllerdebugmap: 
+ orgmodule=org.opendaylight.yangtools.yang.parser.repo.YangTextSchemaContextResolver
+ orgmodule_=org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver
+ echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN'
+ echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN'
+ unset IFS
+ echo 'controllerdebugmap: '
+ '[' -n '' ']'
+ echo 'cat /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg'
cat /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
+ cat /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
################################################################################
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#
################################################################################

# Common pattern layout for appenders
log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n

# Root logger
log4j2.rootLogger.level = INFO
# uncomment to use asynchronous loggers, which require mvn:com.lmax/disruptor/3.3.2 library
#log4j2.rootLogger.type = asyncRoot
#log4j2.rootLogger.includeLocation = false
log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile
log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi
log4j2.rootLogger.appenderRef.Console.ref = Console
log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter
log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF}

# Filters for logs marked by org.opendaylight.odlparent.Markers
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL

# Loggers configuration

# Spifly logger
log4j2.logger.spifly.name = org.apache.aries.spifly
log4j2.logger.spifly.level = WARN

# Security audit logger
log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit
log4j2.logger.audit.level = INFO
log4j2.logger.audit.additivity = false
log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile

# Appenders configuration

# Console appender not used by default (see log4j2.rootLogger.appenderRefs)
log4j2.appender.console.type = Console
log4j2.appender.console.name = Console
log4j2.appender.console.layout.type = PatternLayout
log4j2.appender.console.layout.pattern = ${log4j2.pattern}

# Rolling file appender
log4j2.appender.rolling.type = RollingRandomAccessFile
log4j2.appender.rolling.name = RollingFile
log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i
# uncomment to not force a disk flush
#log4j2.appender.rolling.immediateFlush = false
log4j2.appender.rolling.append = true
log4j2.appender.rolling.layout.type = PatternLayout
log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
log4j2.appender.rolling.policies.type = Policies
log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.rolling.policies.size.size = 1GB
log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
log4j2.appender.rolling.strategy.max = 7

# Audit file appender
log4j2.appender.audit.type = RollingRandomAccessFile
log4j2.appender.audit.name = AuditRollingFile
log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log
log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i
log4j2.appender.audit.append = true
log4j2.appender.audit.layout.type = PatternLayout
log4j2.appender.audit.layout.pattern = ${log4j2.pattern}
log4j2.appender.audit.policies.type = Policies
log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.audit.policies.size.size = 8MB
log4j2.appender.audit.strategy.type = DefaultRolloverStrategy
log4j2.appender.audit.strategy.max = 7

# OSGi appender
log4j2.appender.osgi.type = PaxOsgi
log4j2.appender.osgi.name = PaxOsgi
log4j2.appender.osgi.filter = *

# help with identification of maven-related problems with pax-url-aether
#log4j2.logger.aether.name = shaded.org.eclipse.aether
#log4j2.logger.aether.level = TRACE
#log4j2.logger.http-headers.name = shaded.org.apache.http.headers
#log4j2.logger.http-headers.level = DEBUG
#log4j2.logger.maven.name = org.ops4j.pax.url.mvn
#log4j2.logger.maven.level = TRACE
log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN
log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN
Configure
    java home: /usr/lib/jvm/java-17-openjdk-amd64
    max memory: 3072m
    memconf: /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
+ set_java_vars /usr/lib/jvm/java-17-openjdk-amd64 3072m /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
+ local -r java_home=/usr/lib/jvm/java-17-openjdk-amd64
+ local -r controllermem=3072m
+ local -r memconf=/tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
+ echo Configure
+ echo '    java home: /usr/lib/jvm/java-17-openjdk-amd64'
+ echo '    max memory: 3072m'
+ echo '    memconf: /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv'
+ sed -ie 's%^# export JAVA_HOME%export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-17-openjdk-amd64}%g' /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
+ sed -ie 's/JAVA_MAX_MEM="2048m"/JAVA_MAX_MEM=3072m/g' /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
cat /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
+ echo 'cat /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv'
+ cat /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
#!/bin/sh
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#

#
# handle specific scripts; the SCRIPT_NAME is exactly the name of the Karaf
# script: client, instance, shell, start, status, stop, karaf
#
# if [ "${KARAF_SCRIPT}" == "SCRIPT_NAME" ]; then
#   Actions go here...
# fi

#
# general settings which should be applied for all scripts go here; please keep
# in mind that it is possible that scripts might be executed more than once, e.g.
# in example of the start script where the start script is executed first and the
# karaf script afterwards.
#

#
# The following section shows the possible configuration options for the default 
# karaf scripts
#
export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-17-openjdk-amd64} # Location of Java installation
# export JAVA_OPTS # Generic JVM options, for instance, where you can pass the memory configuration
# export JAVA_NON_DEBUG_OPTS # Additional non-debug JVM options
# export EXTRA_JAVA_OPTS # Additional JVM options
# export KARAF_HOME # Karaf home folder
# export KARAF_DATA # Karaf data folder
# export KARAF_BASE # Karaf base folder
# export KARAF_ETC  # Karaf etc  folder
# export KARAF_LOG  # Karaf log  folder
# export KARAF_SYSTEM_OPTS # First citizen Karaf options
# export KARAF_OPTS # Additional available Karaf options
# export KARAF_DEBUG # Enable debug mode
# export KARAF_REDIRECT # Enable/set the std/err redirection when using bin/start
# export KARAF_NOROOT # Prevent execution as root if set to true
Set Java version
+ echo 'Set Java version'
+ sudo /usr/sbin/alternatives --install /usr/bin/java java /usr/lib/jvm/java-17-openjdk-amd64/bin/java 1
sudo: /usr/sbin/alternatives: command not found
+ sudo /usr/sbin/alternatives --set java /usr/lib/jvm/java-17-openjdk-amd64/bin/java
sudo: /usr/sbin/alternatives: command not found
JDK default version ...
+ echo 'JDK default version ...'
+ java -version
openjdk version "11.0.25" 2024-10-15
OpenJDK Runtime Environment (build 11.0.25+9-post-Ubuntu-1ubuntu120.04)
OpenJDK 64-Bit Server VM (build 11.0.25+9-post-Ubuntu-1ubuntu120.04, mixed mode, sharing)
Set JAVA_HOME
+ echo 'Set JAVA_HOME'
+ export JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64
++ readlink -e /usr/lib/jvm/java-17-openjdk-amd64/bin/java
Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-17-openjdk-amd64/bin/java
Listing all open ports on controller system...
+ JAVA_RESOLVED=/usr/lib/jvm/java-17-openjdk-amd64/bin/java
+ echo 'Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-17-openjdk-amd64/bin/java'
+ echo 'Listing all open ports on controller system...'
+ netstat -pnatu
/tmp/configuration-script.sh: line 40: netstat: command not found
Custom shard config exists!!!
+ '[' -f /tmp/custom_shard_config.txt ']'
+ echo 'Custom shard config exists!!!'
Copying the shard config...
+ echo 'Copying the shard config...'
+ cp /tmp/custom_shard_config.txt /tmp/karaf-0.20.4-SNAPSHOT/bin/
Configuring cluster
+ echo 'Configuring cluster'
+ /tmp/karaf-0.20.4-SNAPSHOT/bin/configure_cluster.sh 1 10.30.171.225 10.30.170.179 10.30.171.166
################################################
##             Configure Cluster              ##
################################################
 NOTE: Cluster configuration files not found. Copying from
 /tmp/karaf-0.20.4-SNAPSHOT/system/org/opendaylight/controller/sal-clustering-config/9.0.6
Configuring unique name in akka.conf
Configuring hostname in akka.conf
Configuring data and rpc seed nodes in akka.conf
modules = [
	
	{
		name = "inventory"
		namespace = "urn:opendaylight:inventory"
		shard-strategy = "module"
	},
	{
		name = "topology"
		namespace = "urn:TBD:params:xml:ns:yang:network-topology"
		shard-strategy = "module"
	},
	{
		name = "toaster"
		namespace = "http://netconfcentral.org/ns/toaster"
		shard-strategy = "module"
	},
	{
		name = "car"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car"
		shard-strategy = "module"
	},
	{
		name = "people"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people"
		shard-strategy = "module"
	},
	{
		name = "car-people"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people"
		shard-strategy = "module"
	}
]
Configuring replication type in module-shards.conf
################################################
##   NOTE: Manually restart controller to     ##
##         apply configuration.               ##
################################################
Dump akka.conf
+ echo 'Dump akka.conf'
+ cat /tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/akka.conf

odl-cluster-data {
  akka {
    remote {
      artery {
        enabled = on
        transport = tcp
        canonical.hostname = "10.30.171.225"
        canonical.port = 2550
      }
    }

    cluster {
      # Using artery.
      seed-nodes = ["akka://opendaylight-cluster-data@10.30.171.225:2550",
				"akka://opendaylight-cluster-data@10.30.170.179:2550",
				"akka://opendaylight-cluster-data@10.30.171.166:2550"]

      roles = ["member-1"]

      # when under load we might trip a false positive on the failure detector
      # failure-detector {
        # heartbeat-interval = 4 s
        # acceptable-heartbeat-pause = 16s
      # }
    }

    persistence {
      # By default the snapshots/journal directories live in KARAF_HOME. You can choose to put it somewhere else by
      # modifying the following two properties. The directory location specified may be a relative or absolute path. 
      # The relative path is always relative to KARAF_HOME.

      # snapshot-store.local.dir = "target/snapshots"

      # Use lz4 compression for LocalSnapshotStore snapshots
      snapshot-store.local.use-lz4-compression = false
      # Size of blocks for lz4 compression: 64KB, 256KB, 1MB or 4MB
      snapshot-store.local.lz4-blocksize = 256KB
    }
    disable-default-actor-system-quarantined-event-handling = "false"
  }
}
Dump modules.conf
+ echo 'Dump modules.conf'
+ cat /tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/modules.conf
modules = [
	
	{
		name = "inventory"
		namespace = "urn:opendaylight:inventory"
		shard-strategy = "module"
	},
	{
		name = "topology"
		namespace = "urn:TBD:params:xml:ns:yang:network-topology"
		shard-strategy = "module"
	},
	{
		name = "toaster"
		namespace = "http://netconfcentral.org/ns/toaster"
		shard-strategy = "module"
	},
	{
		name = "car"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car"
		shard-strategy = "module"
	},
	{
		name = "people"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people"
		shard-strategy = "module"
	},
	{
		name = "car-people"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people"
		shard-strategy = "module"
	}
]
Dump module-shards.conf
+ echo 'Dump module-shards.conf'
+ cat /tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/module-shards.conf
module-shards = [
	{
		name = "default"
		shards = [
			{
				name = "default"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "inventory"
		shards = [
			{
				name="inventory"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "topology"
		shards = [
			{
				name="topology"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "toaster"
		shards = [
			{
				name="toaster"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "car"
		shards = [
			{
				name="car"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "people"
		shards = [
			{
				name="people"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "car-people"
		shards = [
			{
				name="car-people"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	}
]
Configuring member-2 with IP address 10.30.170.179
Warning: Permanently added '10.30.170.179' (ECDSA) to the list of known hosts.
Warning: Permanently added '10.30.170.179' (ECDSA) to the list of known hosts.
+ source /tmp/common-functions.sh karaf-0.20.4-SNAPSHOT
common-functions.sh is being sourced
++ [[ /tmp/common-functions.sh == \/\t\m\p\/\c\o\n\f\i\g\u\r\a\t\i\o\n\-\s\c\r\i\p\t\.\s\h ]]
++ echo 'common-functions.sh is being sourced'
++ BUNDLEFOLDER=karaf-0.20.4-SNAPSHOT
++ export MAVENCONF=/tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
++ MAVENCONF=/tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
++ export FEATURESCONF=/tmp/karaf-0.20.4-SNAPSHOT/etc/org.apache.karaf.features.cfg
++ FEATURESCONF=/tmp/karaf-0.20.4-SNAPSHOT/etc/org.apache.karaf.features.cfg
++ export CUSTOMPROP=/tmp/karaf-0.20.4-SNAPSHOT/etc/custom.properties
++ CUSTOMPROP=/tmp/karaf-0.20.4-SNAPSHOT/etc/custom.properties
++ export LOGCONF=/tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
++ LOGCONF=/tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
++ export MEMCONF=/tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
++ MEMCONF=/tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
++ export CONTROLLERMEM=
++ CONTROLLERMEM=
++ export AKKACONF=/tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/akka.conf
++ AKKACONF=/tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/akka.conf
++ export MODULESCONF=/tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/modules.conf
++ MODULESCONF=/tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/modules.conf
++ export MODULESHARDSCONF=/tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/module-shards.conf
++ MODULESHARDSCONF=/tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/module-shards.conf
++ print_common_env
++ cat
common-functions environment:
MAVENCONF: /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
ACTUALFEATURES: 
FEATURESCONF: /tmp/karaf-0.20.4-SNAPSHOT/etc/org.apache.karaf.features.cfg
CUSTOMPROP: /tmp/karaf-0.20.4-SNAPSHOT/etc/custom.properties
LOGCONF: /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
MEMCONF: /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
CONTROLLERMEM: 
AKKACONF: /tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/akka.conf
MODULESCONF: /tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/modules.conf
MODULESHARDSCONF: /tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/module-shards.conf
SUITES: 

++ SSH='ssh -t -t'
++ extra_services_cntl='     dnsmasq.service     httpd.service     libvirtd.service     openvswitch.service     ovs-vswitchd.service     ovsdb-server.service     rabbitmq-server.service '
++ extra_services_cmp='     libvirtd.service     openvswitch.service     ovs-vswitchd.service     ovsdb-server.service '
Changing to /tmp
Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.4-SNAPSHOT/karaf-0.20.4-20250222.204906-381.zip
+ echo 'Changing to /tmp'
+ cd /tmp
+ echo 'Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.4-SNAPSHOT/karaf-0.20.4-20250222.204906-381.zip'
+ wget --progress=dot:mega https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.4-SNAPSHOT/karaf-0.20.4-20250222.204906-381.zip
--2025-02-22 23:45:05--  https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.4-SNAPSHOT/karaf-0.20.4-20250222.204906-381.zip
Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6
Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 234245297 (223M) [application/zip]
Saving to: ‘karaf-0.20.4-20250222.204906-381.zip’

     0K ........ ........ ........ ........ ........ ........  1% 59.7M 4s
  3072K ........ ........ ........ ........ ........ ........  2% 78.9M 3s
  6144K ........ ........ ........ ........ ........ ........  4%  152M 3s
  9216K ........ ........ ........ ........ ........ ........  5%  148M 2s
 12288K ........ ........ ........ ........ ........ ........  6%  220M 2s
 15360K ........ ........ ........ ........ ........ ........  8%  192M 2s
 18432K ........ ........ ........ ........ ........ ........  9%  172M 2s
 21504K ........ ........ ........ ........ ........ ........ 10%  241M 2s
 24576K ........ ........ ........ ........ ........ ........ 12%  227M 1s
 27648K ........ ........ ........ ........ ........ ........ 13%  269M 1s
 30720K ........ ........ ........ ........ ........ ........ 14%  272M 1s
 33792K ........ ........ ........ ........ ........ ........ 16%  319M 1s
 36864K ........ ........ ........ ........ ........ ........ 17%  304M 1s
 39936K ........ ........ ........ ........ ........ ........ 18%  362M 1s
 43008K ........ ........ ........ ........ ........ ........ 20%  209M 1s
 46080K ........ ........ ........ ........ ........ ........ 21%  309M 1s
 49152K ........ ........ ........ ........ ........ ........ 22%  348M 1s
 52224K ........ ........ ........ ........ ........ ........ 24%  364M 1s
 55296K ........ ........ ........ ........ ........ ........ 25%  346M 1s
 58368K ........ ........ ........ ........ ........ ........ 26%  358M 1s
 61440K ........ ........ ........ ........ ........ ........ 28%  354M 1s
 64512K ........ ........ ........ ........ ........ ........ 29%  345M 1s
 67584K ........ ........ ........ ........ ........ ........ 30%  379M 1s
 70656K ........ ........ ........ ........ ........ ........ 32%  365M 1s
 73728K ........ ........ ........ ........ ........ ........ 33%  336M 1s
 76800K ........ ........ ........ ........ ........ ........ 34%  320M 1s
 79872K ........ ........ ........ ........ ........ ........ 36%  371M 1s
 82944K ........ ........ ........ ........ ........ ........ 37%  341M 1s
 86016K ........ ........ ........ ........ ........ ........ 38%  362M 1s
 89088K ........ ........ ........ ........ ........ ........ 40%  372M 1s
 92160K ........ ........ ........ ........ ........ ........ 41%  379M 1s
 95232K ........ ........ ........ ........ ........ ........ 42%  369M 1s
 98304K ........ ........ ........ ........ ........ ........ 44%  377M 1s
101376K ........ ........ ........ ........ ........ ........ 45%  378M 1s
104448K ........ ........ ........ ........ ........ ........ 47%  375M 0s
107520K ........ ........ ........ ........ ........ ........ 48%  245M 0s
110592K ........ ........ ........ ........ ........ ........ 49%  322M 0s
113664K ........ ........ ........ ........ ........ ........ 51%  215M 0s
116736K ........ ........ ........ ........ ........ ........ 52%  262M 0s
119808K ........ ........ ........ ........ ........ ........ 53%  301M 0s
122880K ........ ........ ........ ........ ........ ........ 55%  312M 0s
125952K ........ ........ ........ ........ ........ ........ 56%  238M 0s
129024K ........ ........ ........ ........ ........ ........ 57%  283M 0s
132096K ........ ........ ........ ........ ........ ........ 59%  233M 0s
135168K ........ ........ ........ ........ ........ ........ 60%  249M 0s
138240K ........ ........ ........ ........ ........ ........ 61%  235M 0s
141312K ........ ........ ........ ........ ........ ........ 63%  261M 0s
144384K ........ ........ ........ ........ ........ ........ 64%  239M 0s
147456K ........ ........ ........ ........ ........ ........ 65%  277M 0s
150528K ........ ........ ........ ........ ........ ........ 67%  273M 0s
153600K ........ ........ ........ ........ ........ ........ 68%  228M 0s
156672K ........ ........ ........ ........ ........ ........ 69%  245M 0s
159744K ........ ........ ........ ........ ........ ........ 71%  233M 0s
162816K ........ ........ ........ ........ ........ ........ 72%  206M 0s
165888K ........ ........ ........ ........ ........ ........ 73%  248M 0s
168960K ........ ........ ........ ........ ........ ........ 75%  260M 0s
172032K ........ ........ ........ ........ ........ ........ 76%  291M 0s
175104K ........ ........ ........ ........ ........ ........ 77%  271M 0s
178176K ........ ........ ........ ........ ........ ........ 79%  262M 0s
181248K ........ ........ ........ ........ ........ ........ 80%  280M 0s
184320K ........ ........ ........ ........ ........ ........ 81%  302M 0s
187392K ........ ........ ........ ........ ........ ........ 83%  305M 0s
190464K ........ ........ ........ ........ ........ ........ 84%  296M 0s
193536K ........ ........ ........ ........ ........ ........ 85%  285M 0s
196608K ........ ........ ........ ........ ........ ........ 87%  294M 0s
199680K ........ ........ ........ ........ ........ ........ 88%  280M 0s
202752K ........ ........ ........ ........ ........ ........ 89%  278M 0s
205824K ........ ........ ........ ........ ........ ........ 91%  275M 0s
208896K ........ ........ ........ ........ ........ ........ 92%  282M 0s
211968K ........ ........ ........ ........ ........ ........ 94%  281M 0s
215040K ........ ........ ........ ........ ........ ........ 95%  275M 0s
218112K ........ ........ ........ ........ ........ ........ 96%  280M 0s
221184K ........ ........ ........ ........ ........ ........ 98%  277M 0s
224256K ........ ........ ........ ........ ........ ........ 99%  285M 0s
227328K ........ ........ ......                             100%  302M=0.9s

2025-02-22 23:45:06 (254 MB/s) - ‘karaf-0.20.4-20250222.204906-381.zip’ saved [234245297/234245297]

Extracting the new controller...
+ echo 'Extracting the new controller...'
+ unzip -q karaf-0.20.4-20250222.204906-381.zip
Adding external repositories...
+ echo 'Adding external repositories...'
+ sed -ie 's%org.ops4j.pax.url.mvn.repositories=%org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases%g' /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
+ cat /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
################################################################################
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#
################################################################################

#
# If set to true, the following property will not allow any certificate to be used
# when accessing Maven repositories through SSL
#
#org.ops4j.pax.url.mvn.certificateCheck=

#
# Path to the local Maven settings file.
# The repositories defined in this file will be automatically added to the list
# of default repositories if the 'org.ops4j.pax.url.mvn.repositories' property
# below is not set.
# The following locations are checked for the existence of the settings.xml file
#   * 1. looks for the specified url
#   * 2. if not found looks for ${user.home}/.m2/settings.xml
#   * 3. if not found looks for ${maven.home}/conf/settings.xml
#   * 4. if not found looks for ${M2_HOME}/conf/settings.xml
#
#org.ops4j.pax.url.mvn.settings=

#
# Path to the local Maven repository which is used to avoid downloading
# artifacts when they already exist locally.
# The value of this property will be extracted from the settings.xml file
# above, or defaulted to:
#     System.getProperty( "user.home" ) + "/.m2/repository"
#
org.ops4j.pax.url.mvn.localRepository=${karaf.home}/${karaf.default.repository}

#
# Default this to false. It's just weird to use undocumented repos
#
org.ops4j.pax.url.mvn.useFallbackRepositories=false

#
# Uncomment if you don't wanna use the proxy settings
# from the Maven conf/settings.xml file
#
# org.ops4j.pax.url.mvn.proxySupport=false

#
# Comma separated list of repositories scanned when resolving an artifact.
# Those repositories will be checked before iterating through the
#    below list of repositories and even before the local repository
# A repository url can be appended with zero or more of the following flags:
#    @snapshots  : the repository contains snaphots
#    @noreleases : the repository does not contain any released artifacts
#
# The following property value will add the system folder as a repo.
#
org.ops4j.pax.url.mvn.defaultRepositories=\
    file:${karaf.home}/${karaf.default.repository}@id=system.repository@snapshots,\
    file:${karaf.data}/kar@id=kar.repository@multi@snapshots,\
    file:${karaf.base}/${karaf.default.repository}@id=child.system.repository@snapshots

# Use the default local repo (e.g.~/.m2/repository) as a "remote" repo
#org.ops4j.pax.url.mvn.defaultLocalRepoAsRemote=false

#
# Comma separated list of repositories scanned when resolving an artifact.
# The default list includes the following repositories:
#    http://repo1.maven.org/maven2@id=central
#    http://repository.springsource.com/maven/bundles/release@id=spring.ebr
#    http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external
#    http://zodiac.springsource.com/maven/bundles/release@id=gemini
#    http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases
#    https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases
#    https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases
# To add repositories to the default ones, prepend '+' to the list of repositories
# to add.
# A repository url can be appended with zero or more of the following flags:
#    @snapshots  : the repository contains snapshots
#    @noreleases : the repository does not contain any released artifacts
#    @id=repository.id : the id for the repository, just like in the settings.xml this is optional but recommended
#
org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases

### ^^^ No remote repositories. This is the only ODL change compared to Karaf defaults.Configuring the startup features...
+ [[ True == \T\r\u\e ]]
+ echo 'Configuring the startup features...'
+ sed -ie 's/\(featuresBoot=\|featuresBoot =\)/featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app,/g' /tmp/karaf-0.20.4-SNAPSHOT/etc/org.apache.karaf.features.cfg
+ FEATURE_TEST_STRING=features-test
+ FEATURE_TEST_VERSION=0.20.4-SNAPSHOT
+ KARAF_VERSION=karaf4
+ [[ integration == \i\n\t\e\g\r\a\t\i\o\n ]]
+ sed -ie 's%\(featuresRepositories=\|featuresRepositories =\)%featuresRepositories = mvn:org.opendaylight.integration/features-test/0.20.4-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features,%g' /tmp/karaf-0.20.4-SNAPSHOT/etc/org.apache.karaf.features.cfg
+ [[ ! -z '' ]]
+ cat /tmp/karaf-0.20.4-SNAPSHOT/etc/org.apache.karaf.features.cfg
################################################################################
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#
################################################################################

#
# Comma separated list of features repositories to register by default
#
featuresRepositories = mvn:org.opendaylight.integration/features-test/0.20.4-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features, file:${karaf.etc}/1770ed62-ed0a-4998-9f53-555fbe661aac.xml

#
# Comma separated list of features to install at startup
#
featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app, 185dc51d-e1fe-4d2d-bb8f-35ce328d82cf

#
# Resource repositories (OBR) that the features resolver can use
# to resolve requirements/capabilities
#
# The format of the resourceRepositories is 
# resourceRepositories=[xml:url|json:url],...
# for Instance:
#
#resourceRepositories=xml:http://host/path/to/index.xml
# or
#resourceRepositories=json:http://host/path/to/index.json
#

#
# Defines if the boot features are started in asynchronous mode (in a dedicated thread)
#
featuresBootAsynchronous=false

#
# Service requirements enforcement
#
# By default, the feature resolver checks the service requirements/capabilities of
# bundles for new features (xml schema >= 1.3.0) in order to automatically installs
# the required bundles.
# The following flag can have those values:
#   - disable: service requirements are completely ignored
#   - default: service requirements are ignored for old features
#   - enforce: service requirements are always verified
#
#serviceRequirements=default

#
# Store cfg file for config element in feature
#
#configCfgStore=true

#
# Define if the feature service automatically refresh bundles
#
autoRefresh=true

#
# Configuration of features processing mechanism (overrides, blacklisting, modification of features)
# XML file defines instructions related to features processing
# versions.properties may declare properties to resolve placeholders in XML file
# both files are relative to ${karaf.etc}
#
#featureProcessing=org.apache.karaf.features.xml
#featureProcessingVersions=versions.properties
+ configure_karaf_log karaf4 ''
+ local -r karaf_version=karaf4
+ local -r controllerdebugmap=
+ local logapi=log4j
+ grep log4j2 /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
log4j2.rootLogger.level = INFO
#log4j2.rootLogger.type = asyncRoot
#log4j2.rootLogger.includeLocation = false
log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile
log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi
log4j2.rootLogger.appenderRef.Console.ref = Console
log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter
log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF}
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL
log4j2.logger.spifly.name = org.apache.aries.spifly
log4j2.logger.spifly.level = WARN
log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit
log4j2.logger.audit.level = INFO
log4j2.logger.audit.additivity = false
log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile
# Console appender not used by default (see log4j2.rootLogger.appenderRefs)
log4j2.appender.console.type = Console
log4j2.appender.console.name = Console
log4j2.appender.console.layout.type = PatternLayout
log4j2.appender.console.layout.pattern = ${log4j2.pattern}
log4j2.appender.rolling.type = RollingRandomAccessFile
log4j2.appender.rolling.name = RollingFile
log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i
#log4j2.appender.rolling.immediateFlush = false
log4j2.appender.rolling.append = true
log4j2.appender.rolling.layout.type = PatternLayout
log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
log4j2.appender.rolling.policies.type = Policies
log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.rolling.policies.size.size = 64MB
log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
log4j2.appender.rolling.strategy.max = 7
log4j2.appender.audit.type = RollingRandomAccessFile
log4j2.appender.audit.name = AuditRollingFile
log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log
log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i
log4j2.appender.audit.append = true
log4j2.appender.audit.layout.type = PatternLayout
log4j2.appender.audit.layout.pattern = ${log4j2.pattern}
log4j2.appender.audit.policies.type = Policies
log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.audit.policies.size.size = 8MB
log4j2.appender.audit.strategy.type = DefaultRolloverStrategy
log4j2.appender.audit.strategy.max = 7
log4j2.appender.osgi.type = PaxOsgi
log4j2.appender.osgi.name = PaxOsgi
log4j2.appender.osgi.filter = *
#log4j2.logger.aether.name = shaded.org.eclipse.aether
#log4j2.logger.aether.level = TRACE
#log4j2.logger.http-headers.name = shaded.org.apache.http.headers
#log4j2.logger.http-headers.level = DEBUG
#log4j2.logger.maven.name = org.ops4j.pax.url.mvn
#log4j2.logger.maven.level = TRACE
+ logapi=log4j2
+ echo 'Configuring the karaf log... karaf_version: karaf4, logapi: log4j2'
Configuring the karaf log... karaf_version: karaf4, logapi: log4j2
+ '[' log4j2 == log4j2 ']'
+ sed -ie 's/log4j2.appender.rolling.policies.size.size = 64MB/log4j2.appender.rolling.policies.size.size = 1GB/g' /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
+ orgmodule=org.opendaylight.yangtools.yang.parser.repo.YangTextSchemaContextResolver
+ orgmodule_=org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver
+ echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN'
controllerdebugmap: 
+ echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN'
+ unset IFS
+ echo 'controllerdebugmap: '
cat /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
+ '[' -n '' ']'
+ echo 'cat /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg'
+ cat /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
################################################################################
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#
################################################################################

# Common pattern layout for appenders
log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n

# Root logger
log4j2.rootLogger.level = INFO
# uncomment to use asynchronous loggers, which require mvn:com.lmax/disruptor/3.3.2 library
#log4j2.rootLogger.type = asyncRoot
#log4j2.rootLogger.includeLocation = false
log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile
log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi
log4j2.rootLogger.appenderRef.Console.ref = Console
log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter
log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF}

# Filters for logs marked by org.opendaylight.odlparent.Markers
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL

# Loggers configuration

# Spifly logger
log4j2.logger.spifly.name = org.apache.aries.spifly
log4j2.logger.spifly.level = WARN

# Security audit logger
log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit
log4j2.logger.audit.level = INFO
log4j2.logger.audit.additivity = false
log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile

# Appenders configuration

# Console appender not used by default (see log4j2.rootLogger.appenderRefs)
log4j2.appender.console.type = Console
log4j2.appender.console.name = Console
log4j2.appender.console.layout.type = PatternLayout
log4j2.appender.console.layout.pattern = ${log4j2.pattern}

# Rolling file appender
log4j2.appender.rolling.type = RollingRandomAccessFile
log4j2.appender.rolling.name = RollingFile
log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i
# uncomment to not force a disk flush
#log4j2.appender.rolling.immediateFlush = false
log4j2.appender.rolling.append = true
log4j2.appender.rolling.layout.type = PatternLayout
log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
log4j2.appender.rolling.policies.type = Policies
log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.rolling.policies.size.size = 1GB
log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
log4j2.appender.rolling.strategy.max = 7

# Audit file appender
log4j2.appender.audit.type = RollingRandomAccessFile
log4j2.appender.audit.name = AuditRollingFile
log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log
log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i
log4j2.appender.audit.append = true
log4j2.appender.audit.layout.type = PatternLayout
log4j2.appender.audit.layout.pattern = ${log4j2.pattern}
log4j2.appender.audit.policies.type = Policies
log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.audit.policies.size.size = 8MB
log4j2.appender.audit.strategy.type = DefaultRolloverStrategy
log4j2.appender.audit.strategy.max = 7

# OSGi appender
log4j2.appender.osgi.type = PaxOsgi
log4j2.appender.osgi.name = PaxOsgi
log4j2.appender.osgi.filter = *

# help with identification of maven-related problems with pax-url-aether
#log4j2.logger.aether.name = shaded.org.eclipse.aether
#log4j2.logger.aether.level = TRACE
#log4j2.logger.http-headers.name = shaded.org.apache.http.headers
#log4j2.logger.http-headers.level = DEBUG
#log4j2.logger.maven.name = org.ops4j.pax.url.mvn
#log4j2.logger.maven.level = TRACE
log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN
log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN
+ set_java_vars /usr/lib/jvm/java-17-openjdk-amd64 3072m /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
+ local -r java_home=/usr/lib/jvm/java-17-openjdk-amd64
+ local -r controllermem=3072m
Configure
+ local -r memconf=/tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
+ echo Configure
    java home: /usr/lib/jvm/java-17-openjdk-amd64
    max memory: 3072m
    memconf: /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
+ echo '    java home: /usr/lib/jvm/java-17-openjdk-amd64'
+ echo '    max memory: 3072m'
+ echo '    memconf: /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv'
+ sed -ie 's%^# export JAVA_HOME%export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-17-openjdk-amd64}%g' /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
+ sed -ie 's/JAVA_MAX_MEM="2048m"/JAVA_MAX_MEM=3072m/g' /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
cat /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
+ echo 'cat /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv'
+ cat /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
#!/bin/sh
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#

#
# handle specific scripts; the SCRIPT_NAME is exactly the name of the Karaf
# script: client, instance, shell, start, status, stop, karaf
#
# if [ "${KARAF_SCRIPT}" == "SCRIPT_NAME" ]; then
#   Actions go here...
# fi

#
# general settings which should be applied for all scripts go here; please keep
# in mind that it is possible that scripts might be executed more than once, e.g.
# in example of the start script where the start script is executed first and the
# karaf script afterwards.
#

#
# The following section shows the possible configuration options for the default 
# karaf scripts
#
export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-17-openjdk-amd64} # Location of Java installation
# export JAVA_OPTS # Generic JVM options, for instance, where you can pass the memory configuration
# export JAVA_NON_DEBUG_OPTS # Additional non-debug JVM options
# export EXTRA_JAVA_OPTS # Additional JVM options
# export KARAF_HOME # Karaf home folder
# export KARAF_DATA # Karaf data folder
# export KARAF_BASE # Karaf base folder
# export KARAF_ETC  # Karaf etc  folder
# export KARAF_LOG  # Karaf log  folder
# export KARAF_SYSTEM_OPTS # First citizen Karaf options
# export KARAF_OPTS # Additional available Karaf options
# export KARAF_DEBUG # Enable debug mode
# export KARAF_REDIRECT # Enable/set the std/err redirection when using bin/start
# export KARAF_NOROOT # Prevent execution as root if set to true
Set Java version
+ echo 'Set Java version'
+ sudo /usr/sbin/alternatives --install /usr/bin/java java /usr/lib/jvm/java-17-openjdk-amd64/bin/java 1
sudo: /usr/sbin/alternatives: command not found
+ sudo /usr/sbin/alternatives --set java /usr/lib/jvm/java-17-openjdk-amd64/bin/java
sudo: /usr/sbin/alternatives: command not found
JDK default version ...
+ echo 'JDK default version ...'
+ java -version
openjdk version "11.0.25" 2024-10-15
OpenJDK Runtime Environment (build 11.0.25+9-post-Ubuntu-1ubuntu120.04)
OpenJDK 64-Bit Server VM (build 11.0.25+9-post-Ubuntu-1ubuntu120.04, mixed mode, sharing)
Set JAVA_HOME
+ echo 'Set JAVA_HOME'
+ export JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64
++ readlink -e /usr/lib/jvm/java-17-openjdk-amd64/bin/java
+ JAVA_RESOLVED=/usr/lib/jvm/java-17-openjdk-amd64/bin/java
+ echo 'Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-17-openjdk-amd64/bin/java'
Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-17-openjdk-amd64/bin/java
Listing all open ports on controller system...
+ echo 'Listing all open ports on controller system...'
+ netstat -pnatu
/tmp/configuration-script.sh: line 40: netstat: command not found
Custom shard config exists!!!
Copying the shard config...
+ '[' -f /tmp/custom_shard_config.txt ']'
+ echo 'Custom shard config exists!!!'
+ echo 'Copying the shard config...'
+ cp /tmp/custom_shard_config.txt /tmp/karaf-0.20.4-SNAPSHOT/bin/
Configuring cluster
+ echo 'Configuring cluster'
+ /tmp/karaf-0.20.4-SNAPSHOT/bin/configure_cluster.sh 2 10.30.171.225 10.30.170.179 10.30.171.166
################################################
##             Configure Cluster              ##
################################################
 NOTE: Cluster configuration files not found. Copying from
 /tmp/karaf-0.20.4-SNAPSHOT/system/org/opendaylight/controller/sal-clustering-config/9.0.6
Configuring unique name in akka.conf
Configuring hostname in akka.conf
Configuring data and rpc seed nodes in akka.conf
modules = [
	
	{
		name = "inventory"
		namespace = "urn:opendaylight:inventory"
		shard-strategy = "module"
	},
	{
		name = "topology"
		namespace = "urn:TBD:params:xml:ns:yang:network-topology"
		shard-strategy = "module"
	},
	{
		name = "toaster"
		namespace = "http://netconfcentral.org/ns/toaster"
		shard-strategy = "module"
	},
	{
		name = "car"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car"
		shard-strategy = "module"
	},
	{
		name = "people"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people"
		shard-strategy = "module"
	},
	{
		name = "car-people"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people"
		shard-strategy = "module"
	}
]
Configuring replication type in module-shards.conf
################################################
##   NOTE: Manually restart controller to     ##
##         apply configuration.               ##
################################################
Dump akka.conf
+ echo 'Dump akka.conf'
+ cat /tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/akka.conf

odl-cluster-data {
  akka {
    remote {
      artery {
        enabled = on
        transport = tcp
        canonical.hostname = "10.30.170.179"
        canonical.port = 2550
      }
    }

    cluster {
      # Using artery.
      seed-nodes = ["akka://opendaylight-cluster-data@10.30.171.225:2550",
				"akka://opendaylight-cluster-data@10.30.170.179:2550",
				"akka://opendaylight-cluster-data@10.30.171.166:2550"]

      roles = ["member-2"]

      # when under load we might trip a false positive on the failure detector
      # failure-detector {
        # heartbeat-interval = 4 s
        # acceptable-heartbeat-pause = 16s
      # }
    }

    persistence {
      # By default the snapshots/journal directories live in KARAF_HOME. You can choose to put it somewhere else by
      # modifying the following two properties. The directory location specified may be a relative or absolute path. 
      # The relative path is always relative to KARAF_HOME.

      # snapshot-store.local.dir = "target/snapshots"

      # Use lz4 compression for LocalSnapshotStore snapshots
      snapshot-store.local.use-lz4-compression = false
      # Size of blocks for lz4 compression: 64KB, 256KB, 1MB or 4MB
      snapshot-store.local.lz4-blocksize = 256KB
    }
    disable-default-actor-system-quarantined-event-handling = "false"
  }
}
Dump modules.conf
+ echo 'Dump modules.conf'
+ cat /tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/modules.conf
modules = [
	
	{
		name = "inventory"
		namespace = "urn:opendaylight:inventory"
		shard-strategy = "module"
	},
	{
		name = "topology"
		namespace = "urn:TBD:params:xml:ns:yang:network-topology"
		shard-strategy = "module"
	},
	{
		name = "toaster"
		namespace = "http://netconfcentral.org/ns/toaster"
		shard-strategy = "module"
	},
	{
		name = "car"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car"
		shard-strategy = "module"
	},
	{
		name = "people"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people"
		shard-strategy = "module"
	},
	{
		name = "car-people"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people"
		shard-strategy = "module"
	}
]
Dump module-shards.conf
+ echo 'Dump module-shards.conf'
+ cat /tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/module-shards.conf
module-shards = [
	{
		name = "default"
		shards = [
			{
				name = "default"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "inventory"
		shards = [
			{
				name="inventory"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "topology"
		shards = [
			{
				name="topology"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "toaster"
		shards = [
			{
				name="toaster"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "car"
		shards = [
			{
				name="car"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "people"
		shards = [
			{
				name="people"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "car-people"
		shards = [
			{
				name="car-people"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	}
]
Configuring member-3 with IP address 10.30.171.166
Warning: Permanently added '10.30.171.166' (ECDSA) to the list of known hosts.
Warning: Permanently added '10.30.171.166' (ECDSA) to the list of known hosts.
+ source /tmp/common-functions.sh karaf-0.20.4-SNAPSHOT
++ [[ /tmp/common-functions.sh == \/\t\m\p\/\c\o\n\f\i\g\u\r\a\t\i\o\n\-\s\c\r\i\p\t\.\s\h ]]
++ echo 'common-functions.sh is being sourced'
common-functions.sh is being sourced
++ BUNDLEFOLDER=karaf-0.20.4-SNAPSHOT
++ export MAVENCONF=/tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
++ MAVENCONF=/tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
++ export FEATURESCONF=/tmp/karaf-0.20.4-SNAPSHOT/etc/org.apache.karaf.features.cfg
++ FEATURESCONF=/tmp/karaf-0.20.4-SNAPSHOT/etc/org.apache.karaf.features.cfg
++ export CUSTOMPROP=/tmp/karaf-0.20.4-SNAPSHOT/etc/custom.properties
++ CUSTOMPROP=/tmp/karaf-0.20.4-SNAPSHOT/etc/custom.properties
++ export LOGCONF=/tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
++ LOGCONF=/tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
++ export MEMCONF=/tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
++ MEMCONF=/tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
++ export CONTROLLERMEM=
++ CONTROLLERMEM=
++ export AKKACONF=/tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/akka.conf
++ AKKACONF=/tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/akka.conf
++ export MODULESCONF=/tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/modules.conf
++ MODULESCONF=/tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/modules.conf
++ export MODULESHARDSCONF=/tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/module-shards.conf
++ MODULESHARDSCONF=/tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/module-shards.conf
++ print_common_env
++ cat
common-functions environment:
MAVENCONF: /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
ACTUALFEATURES: 
FEATURESCONF: /tmp/karaf-0.20.4-SNAPSHOT/etc/org.apache.karaf.features.cfg
CUSTOMPROP: /tmp/karaf-0.20.4-SNAPSHOT/etc/custom.properties
LOGCONF: /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
MEMCONF: /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
CONTROLLERMEM: 
AKKACONF: /tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/akka.conf
MODULESCONF: /tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/modules.conf
MODULESHARDSCONF: /tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/module-shards.conf
SUITES: 

++ SSH='ssh -t -t'
++ extra_services_cntl='     dnsmasq.service     httpd.service     libvirtd.service     openvswitch.service     ovs-vswitchd.service     ovsdb-server.service     rabbitmq-server.service '
++ extra_services_cmp='     libvirtd.service     openvswitch.service     ovs-vswitchd.service     ovsdb-server.service '
Changing to /tmp
Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.4-SNAPSHOT/karaf-0.20.4-20250222.204906-381.zip
+ echo 'Changing to /tmp'
+ cd /tmp
+ echo 'Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.4-SNAPSHOT/karaf-0.20.4-20250222.204906-381.zip'
+ wget --progress=dot:mega https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.4-SNAPSHOT/karaf-0.20.4-20250222.204906-381.zip
--2025-02-22 23:45:08--  https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.4-SNAPSHOT/karaf-0.20.4-20250222.204906-381.zip
Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6
Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 234245297 (223M) [application/zip]
Saving to: ‘karaf-0.20.4-20250222.204906-381.zip’

     0K ........ ........ ........ ........ ........ ........  1% 64.0M 3s
  3072K ........ ........ ........ ........ ........ ........  2%  131M 3s
  6144K ........ ........ ........ ........ ........ ........  4%  156M 2s
  9216K ........ ........ ........ ........ ........ ........  5%  183M 2s
 12288K ........ ........ ........ ........ ........ ........  6%  204M 2s
 15360K ........ ........ ........ ........ ........ ........  8%  187M 2s
 18432K ........ ........ ........ ........ ........ ........  9%  208M 1s
 21504K ........ ........ ........ ........ ........ ........ 10%  231M 1s
 24576K ........ ........ ........ ........ ........ ........ 12%  251M 1s
 27648K ........ ........ ........ ........ ........ ........ 13%  221M 1s
 30720K ........ ........ ........ ........ ........ ........ 14%  249M 1s
 33792K ........ ........ ........ ........ ........ ........ 16%  240M 1s
 36864K ........ ........ ........ ........ ........ ........ 17%  249M 1s
 39936K ........ ........ ........ ........ ........ ........ 18%  257M 1s
 43008K ........ ........ ........ ........ ........ ........ 20%  205M 1s
 46080K ........ ........ ........ ........ ........ ........ 21%  173M 1s
 49152K ........ ........ ........ ........ ........ ........ 22%  245M 1s
 52224K ........ ........ ........ ........ ........ ........ 24%  248M 1s
 55296K ........ ........ ........ ........ ........ ........ 25%  236M 1s
 58368K ........ ........ ........ ........ ........ ........ 26%  257M 1s
 61440K ........ ........ ........ ........ ........ ........ 28%  215M 1s
 64512K ........ ........ ........ ........ ........ ........ 29%  180M 1s
 67584K ........ ........ ........ ........ ........ ........ 30%  192M 1s
 70656K ........ ........ ........ ........ ........ ........ 32%  211M 1s
 73728K ........ ........ ........ ........ ........ ........ 33%  201M 1s
 76800K ........ ........ ........ ........ ........ ........ 34%  154M 1s
 79872K ........ ........ ........ ........ ........ ........ 36%  178M 1s
 82944K ........ ........ ........ ........ ........ ........ 37%  218M 1s
 86016K ........ ........ ........ ........ ........ ........ 38%  208M 1s
 89088K ........ ........ ........ ........ ........ ........ 40%  229M 1s
 92160K ........ ........ ........ ........ ........ ........ 41%  241M 1s
 95232K ........ ........ ........ ........ ........ ........ 42%  243M 1s
 98304K ........ ........ ........ ........ ........ ........ 44%  239M 1s
101376K ........ ........ ........ ........ ........ ........ 45%  218M 1s
104448K ........ ........ ........ ........ ........ ........ 47%  171M 1s
107520K ........ ........ ........ ........ ........ ........ 48%  222M 1s
110592K ........ ........ ........ ........ ........ ........ 49%  203M 1s
113664K ........ ........ ........ ........ ........ ........ 51%  142M 1s
116736K ........ ........ ........ ........ ........ ........ 52%  154M 1s
119808K ........ ........ ........ ........ ........ ........ 53%  173M 1s
122880K ........ ........ ........ ........ ........ ........ 55%  143M 1s
125952K ........ ........ ........ ........ ........ ........ 56%  129M 1s
129024K ........ ........ ........ ........ ........ ........ 57%  123M 1s
132096K ........ ........ ........ ........ ........ ........ 59%  160M 0s
135168K ........ ........ ........ ........ ........ ........ 60%  203M 0s
138240K ........ ........ ........ ........ ........ ........ 61%  159M 0s
141312K ........ ........ ........ ........ ........ ........ 63%  216M 0s
144384K ........ ........ ........ ........ ........ ........ 64%  234M 0s
147456K ........ ........ ........ ........ ........ ........ 65%  244M 0s
150528K ........ ........ ........ ........ ........ ........ 67%  233M 0s
153600K ........ ........ ........ ........ ........ ........ 68%  245M 0s
156672K ........ ........ ........ ........ ........ ........ 69%  215M 0s
159744K ........ ........ ........ ........ ........ ........ 71%  266M 0s
162816K ........ ........ ........ ........ ........ ........ 72%  148M 0s
165888K ........ ........ ........ ........ ........ ........ 73%  231M 0s
168960K ........ ........ ........ ........ ........ ........ 75%  177M 0s
172032K ........ ........ ........ ........ ........ ........ 76%  176M 0s
175104K ........ ........ ........ ........ ........ ........ 77%  232M 0s
178176K ........ ........ ........ ........ ........ ........ 79%  219M 0s
181248K ........ ........ ........ ........ ........ ........ 80%  148M 0s
184320K ........ ........ ........ ........ ........ ........ 81%  227M 0s
187392K ........ ........ ........ ........ ........ ........ 83%  220M 0s
190464K ........ ........ ........ ........ ........ ........ 84%  257M 0s
193536K ........ ........ ........ ........ ........ ........ 85%  243M 0s
196608K ........ ........ ........ ........ ........ ........ 87%  218M 0s
199680K ........ ........ ........ ........ ........ ........ 88%  217M 0s
202752K ........ ........ ........ ........ ........ ........ 89%  158M 0s
205824K ........ ........ ........ ........ ........ ........ 91%  209M 0s
208896K ........ ........ ........ ........ ........ ........ 92%  210M 0s
211968K ........ ........ ........ ........ ........ ........ 94%  229M 0s
215040K ........ ........ ........ ........ ........ ........ 95%  227M 0s
218112K ........ ........ ........ ........ ........ ........ 96%  213M 0s
221184K ........ ........ ........ ........ ........ ........ 98%  179M 0s
224256K ........ ........ ........ ........ ........ ........ 99%  191M 0s
227328K ........ ........ ......                             100%  226M=1.2s

2025-02-22 23:45:10 (193 MB/s) - ‘karaf-0.20.4-20250222.204906-381.zip’ saved [234245297/234245297]

Extracting the new controller...
+ echo 'Extracting the new controller...'
+ unzip -q karaf-0.20.4-20250222.204906-381.zip
Adding external repositories...
+ echo 'Adding external repositories...'
+ sed -ie 's%org.ops4j.pax.url.mvn.repositories=%org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases%g' /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
+ cat /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
################################################################################
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#
################################################################################

#
# If set to true, the following property will not allow any certificate to be used
# when accessing Maven repositories through SSL
#
#org.ops4j.pax.url.mvn.certificateCheck=

#
# Path to the local Maven settings file.
# The repositories defined in this file will be automatically added to the list
# of default repositories if the 'org.ops4j.pax.url.mvn.repositories' property
# below is not set.
# The following locations are checked for the existence of the settings.xml file
#   * 1. looks for the specified url
#   * 2. if not found looks for ${user.home}/.m2/settings.xml
#   * 3. if not found looks for ${maven.home}/conf/settings.xml
#   * 4. if not found looks for ${M2_HOME}/conf/settings.xml
#
#org.ops4j.pax.url.mvn.settings=

#
# Path to the local Maven repository which is used to avoid downloading
# artifacts when they already exist locally.
# The value of this property will be extracted from the settings.xml file
# above, or defaulted to:
#     System.getProperty( "user.home" ) + "/.m2/repository"
#
org.ops4j.pax.url.mvn.localRepository=${karaf.home}/${karaf.default.repository}

#
# Default this to false. It's just weird to use undocumented repos
#
org.ops4j.pax.url.mvn.useFallbackRepositories=false

#
# Uncomment if you don't wanna use the proxy settings
# from the Maven conf/settings.xml file
#
# org.ops4j.pax.url.mvn.proxySupport=false

#
# Comma separated list of repositories scanned when resolving an artifact.
# Those repositories will be checked before iterating through the
#    below list of repositories and even before the local repository
# A repository url can be appended with zero or more of the following flags:
#    @snapshots  : the repository contains snaphots
#    @noreleases : the repository does not contain any released artifacts
#
# The following property value will add the system folder as a repo.
#
org.ops4j.pax.url.mvn.defaultRepositories=\
    file:${karaf.home}/${karaf.default.repository}@id=system.repository@snapshots,\
    file:${karaf.data}/kar@id=kar.repository@multi@snapshots,\
    file:${karaf.base}/${karaf.default.repository}@id=child.system.repository@snapshots

# Use the default local repo (e.g.~/.m2/repository) as a "remote" repo
#org.ops4j.pax.url.mvn.defaultLocalRepoAsRemote=false

#
# Comma separated list of repositories scanned when resolving an artifact.
# The default list includes the following repositories:
#    http://repo1.maven.org/maven2@id=central
#    http://repository.springsource.com/maven/bundles/release@id=spring.ebr
#    http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external
#    http://zodiac.springsource.com/maven/bundles/release@id=gemini
#    http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases
#    https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases
#    https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases
# To add repositories to the default ones, prepend '+' to the list of repositories
# to add.
# A repository url can be appended with zero or more of the following flags:
#    @snapshots  : the repository contains snapshots
#    @noreleases : the repository does not contain any released artifacts
#    @id=repository.id : the id for the repository, just like in the settings.xml this is optional but recommended
#
org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases

### ^^^ No remote repositories. This is the only ODL change compared to Karaf defaults.Configuring the startup features...
+ [[ True == \T\r\u\e ]]
+ echo 'Configuring the startup features...'
+ sed -ie 's/\(featuresBoot=\|featuresBoot =\)/featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app,/g' /tmp/karaf-0.20.4-SNAPSHOT/etc/org.apache.karaf.features.cfg
+ FEATURE_TEST_STRING=features-test
+ FEATURE_TEST_VERSION=0.20.4-SNAPSHOT
+ KARAF_VERSION=karaf4
+ [[ integration == \i\n\t\e\g\r\a\t\i\o\n ]]
+ sed -ie 's%\(featuresRepositories=\|featuresRepositories =\)%featuresRepositories = mvn:org.opendaylight.integration/features-test/0.20.4-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features,%g' /tmp/karaf-0.20.4-SNAPSHOT/etc/org.apache.karaf.features.cfg
+ [[ ! -z '' ]]
+ cat /tmp/karaf-0.20.4-SNAPSHOT/etc/org.apache.karaf.features.cfg
################################################################################
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#
################################################################################

#
# Comma separated list of features repositories to register by default
#
featuresRepositories = mvn:org.opendaylight.integration/features-test/0.20.4-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features, file:${karaf.etc}/1770ed62-ed0a-4998-9f53-555fbe661aac.xml

#
# Comma separated list of features to install at startup
#
featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app, 185dc51d-e1fe-4d2d-bb8f-35ce328d82cf

#
# Resource repositories (OBR) that the features resolver can use
# to resolve requirements/capabilities
#
# The format of the resourceRepositories is 
# resourceRepositories=[xml:url|json:url],...
# for Instance:
#
#resourceRepositories=xml:http://host/path/to/index.xml
# or
#resourceRepositories=json:http://host/path/to/index.json
#

#
# Defines if the boot features are started in asynchronous mode (in a dedicated thread)
#
featuresBootAsynchronous=false

#
# Service requirements enforcement
#
# By default, the feature resolver checks the service requirements/capabilities of
# bundles for new features (xml schema >= 1.3.0) in order to automatically installs
# the required bundles.
# The following flag can have those values:
#   - disable: service requirements are completely ignored
#   - default: service requirements are ignored for old features
#   - enforce: service requirements are always verified
#
#serviceRequirements=default

#
# Store cfg file for config element in feature
#
#configCfgStore=true

#
# Define if the feature service automatically refresh bundles
#
autoRefresh=true

#
# Configuration of features processing mechanism (overrides, blacklisting, modification of features)
# XML file defines instructions related to features processing
# versions.properties may declare properties to resolve placeholders in XML file
# both files are relative to ${karaf.etc}
#
#featureProcessing=org.apache.karaf.features.xml
#featureProcessingVersions=versions.properties
+ configure_karaf_log karaf4 ''
+ local -r karaf_version=karaf4
+ local -r controllerdebugmap=
+ local logapi=log4j
+ grep log4j2 /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
log4j2.rootLogger.level = INFO
#log4j2.rootLogger.type = asyncRoot
#log4j2.rootLogger.includeLocation = false
log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile
log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi
log4j2.rootLogger.appenderRef.Console.ref = Console
log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter
log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF}
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL
log4j2.logger.spifly.name = org.apache.aries.spifly
log4j2.logger.spifly.level = WARN
log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit
log4j2.logger.audit.level = INFO
log4j2.logger.audit.additivity = false
log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile
# Console appender not used by default (see log4j2.rootLogger.appenderRefs)
log4j2.appender.console.type = Console
log4j2.appender.console.name = Console
log4j2.appender.console.layout.type = PatternLayout
log4j2.appender.console.layout.pattern = ${log4j2.pattern}
log4j2.appender.rolling.type = RollingRandomAccessFile
log4j2.appender.rolling.name = RollingFile
log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i
#log4j2.appender.rolling.immediateFlush = false
log4j2.appender.rolling.append = true
log4j2.appender.rolling.layout.type = PatternLayout
log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
log4j2.appender.rolling.policies.type = Policies
log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.rolling.policies.size.size = 64MB
log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
log4j2.appender.rolling.strategy.max = 7
log4j2.appender.audit.type = RollingRandomAccessFile
log4j2.appender.audit.name = AuditRollingFile
log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log
log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i
log4j2.appender.audit.append = true
log4j2.appender.audit.layout.type = PatternLayout
log4j2.appender.audit.layout.pattern = ${log4j2.pattern}
log4j2.appender.audit.policies.type = Policies
log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.audit.policies.size.size = 8MB
log4j2.appender.audit.strategy.type = DefaultRolloverStrategy
log4j2.appender.audit.strategy.max = 7
log4j2.appender.osgi.type = PaxOsgi
log4j2.appender.osgi.name = PaxOsgi
log4j2.appender.osgi.filter = *
#log4j2.logger.aether.name = shaded.org.eclipse.aether
#log4j2.logger.aether.level = TRACE
#log4j2.logger.http-headers.name = shaded.org.apache.http.headers
#log4j2.logger.http-headers.level = DEBUG
#log4j2.logger.maven.name = org.ops4j.pax.url.mvn
#log4j2.logger.maven.level = TRACE
Configuring the karaf log... karaf_version: karaf4, logapi: log4j2
+ logapi=log4j2
+ echo 'Configuring the karaf log... karaf_version: karaf4, logapi: log4j2'
+ '[' log4j2 == log4j2 ']'
+ sed -ie 's/log4j2.appender.rolling.policies.size.size = 64MB/log4j2.appender.rolling.policies.size.size = 1GB/g' /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
controllerdebugmap: 
cat /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
+ orgmodule=org.opendaylight.yangtools.yang.parser.repo.YangTextSchemaContextResolver
+ orgmodule_=org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver
+ echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN'
+ echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN'
+ unset IFS
+ echo 'controllerdebugmap: '
+ '[' -n '' ']'
+ echo 'cat /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg'
+ cat /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
################################################################################
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#
################################################################################

# Common pattern layout for appenders
log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n

# Root logger
log4j2.rootLogger.level = INFO
# uncomment to use asynchronous loggers, which require mvn:com.lmax/disruptor/3.3.2 library
#log4j2.rootLogger.type = asyncRoot
#log4j2.rootLogger.includeLocation = false
log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile
log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi
log4j2.rootLogger.appenderRef.Console.ref = Console
log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter
log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF}

# Filters for logs marked by org.opendaylight.odlparent.Markers
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL

# Loggers configuration

# Spifly logger
log4j2.logger.spifly.name = org.apache.aries.spifly
log4j2.logger.spifly.level = WARN

# Security audit logger
log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit
log4j2.logger.audit.level = INFO
log4j2.logger.audit.additivity = false
log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile

# Appenders configuration

# Console appender not used by default (see log4j2.rootLogger.appenderRefs)
log4j2.appender.console.type = Console
log4j2.appender.console.name = Console
log4j2.appender.console.layout.type = PatternLayout
log4j2.appender.console.layout.pattern = ${log4j2.pattern}

# Rolling file appender
log4j2.appender.rolling.type = RollingRandomAccessFile
log4j2.appender.rolling.name = RollingFile
log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i
# uncomment to not force a disk flush
#log4j2.appender.rolling.immediateFlush = false
log4j2.appender.rolling.append = true
log4j2.appender.rolling.layout.type = PatternLayout
log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
log4j2.appender.rolling.policies.type = Policies
log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.rolling.policies.size.size = 1GB
log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
log4j2.appender.rolling.strategy.max = 7

# Audit file appender
log4j2.appender.audit.type = RollingRandomAccessFile
log4j2.appender.audit.name = AuditRollingFile
log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log
log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i
log4j2.appender.audit.append = true
log4j2.appender.audit.layout.type = PatternLayout
log4j2.appender.audit.layout.pattern = ${log4j2.pattern}
log4j2.appender.audit.policies.type = Policies
log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.audit.policies.size.size = 8MB
log4j2.appender.audit.strategy.type = DefaultRolloverStrategy
log4j2.appender.audit.strategy.max = 7

# OSGi appender
log4j2.appender.osgi.type = PaxOsgi
log4j2.appender.osgi.name = PaxOsgi
log4j2.appender.osgi.filter = *

# help with identification of maven-related problems with pax-url-aether
#log4j2.logger.aether.name = shaded.org.eclipse.aether
#log4j2.logger.aether.level = TRACE
#log4j2.logger.http-headers.name = shaded.org.apache.http.headers
#log4j2.logger.http-headers.level = DEBUG
#log4j2.logger.maven.name = org.ops4j.pax.url.mvn
#log4j2.logger.maven.level = TRACE
log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN
log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN
Configure
    java home: /usr/lib/jvm/java-17-openjdk-amd64
    max memory: 3072m
    memconf: /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
+ set_java_vars /usr/lib/jvm/java-17-openjdk-amd64 3072m /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
+ local -r java_home=/usr/lib/jvm/java-17-openjdk-amd64
+ local -r controllermem=3072m
+ local -r memconf=/tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
+ echo Configure
+ echo '    java home: /usr/lib/jvm/java-17-openjdk-amd64'
+ echo '    max memory: 3072m'
+ echo '    memconf: /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv'
+ sed -ie 's%^# export JAVA_HOME%export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-17-openjdk-amd64}%g' /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
+ sed -ie 's/JAVA_MAX_MEM="2048m"/JAVA_MAX_MEM=3072m/g' /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
cat /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
+ echo 'cat /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv'
+ cat /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
#!/bin/sh
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#

#
# handle specific scripts; the SCRIPT_NAME is exactly the name of the Karaf
# script: client, instance, shell, start, status, stop, karaf
#
# if [ "${KARAF_SCRIPT}" == "SCRIPT_NAME" ]; then
#   Actions go here...
# fi

#
# general settings which should be applied for all scripts go here; please keep
# in mind that it is possible that scripts might be executed more than once, e.g.
# in example of the start script where the start script is executed first and the
# karaf script afterwards.
#

#
# The following section shows the possible configuration options for the default 
# karaf scripts
#
export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-17-openjdk-amd64} # Location of Java installation
# export JAVA_OPTS # Generic JVM options, for instance, where you can pass the memory configuration
# export JAVA_NON_DEBUG_OPTS # Additional non-debug JVM options
# export EXTRA_JAVA_OPTS # Additional JVM options
# export KARAF_HOME # Karaf home folder
# export KARAF_DATA # Karaf data folder
# export KARAF_BASE # Karaf base folder
# export KARAF_ETC  # Karaf etc  folder
# export KARAF_LOG  # Karaf log  folder
# export KARAF_SYSTEM_OPTS # First citizen Karaf options
# export KARAF_OPTS # Additional available Karaf options
# export KARAF_DEBUG # Enable debug mode
# export KARAF_REDIRECT # Enable/set the std/err redirection when using bin/start
# export KARAF_NOROOT # Prevent execution as root if set to true
Set Java version
+ echo 'Set Java version'
+ sudo /usr/sbin/alternatives --install /usr/bin/java java /usr/lib/jvm/java-17-openjdk-amd64/bin/java 1
sudo: /usr/sbin/alternatives: command not found
+ sudo /usr/sbin/alternatives --set java /usr/lib/jvm/java-17-openjdk-amd64/bin/java
sudo: /usr/sbin/alternatives: command not found
JDK default version ...
+ echo 'JDK default version ...'
+ java -version
openjdk version "11.0.25" 2024-10-15
OpenJDK Runtime Environment (build 11.0.25+9-post-Ubuntu-1ubuntu120.04)
OpenJDK 64-Bit Server VM (build 11.0.25+9-post-Ubuntu-1ubuntu120.04, mixed mode, sharing)
Set JAVA_HOME
+ echo 'Set JAVA_HOME'
+ export JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64
++ readlink -e /usr/lib/jvm/java-17-openjdk-amd64/bin/java
Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-17-openjdk-amd64/bin/java
Listing all open ports on controller system...
+ JAVA_RESOLVED=/usr/lib/jvm/java-17-openjdk-amd64/bin/java
+ echo 'Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-17-openjdk-amd64/bin/java'
+ echo 'Listing all open ports on controller system...'
+ netstat -pnatu
/tmp/configuration-script.sh: line 40: netstat: command not found
Custom shard config exists!!!
Copying the shard config...
+ '[' -f /tmp/custom_shard_config.txt ']'
+ echo 'Custom shard config exists!!!'
+ echo 'Copying the shard config...'
+ cp /tmp/custom_shard_config.txt /tmp/karaf-0.20.4-SNAPSHOT/bin/
Configuring cluster
+ echo 'Configuring cluster'
+ /tmp/karaf-0.20.4-SNAPSHOT/bin/configure_cluster.sh 3 10.30.171.225 10.30.170.179 10.30.171.166
################################################
##             Configure Cluster              ##
################################################
 NOTE: Cluster configuration files not found. Copying from
 /tmp/karaf-0.20.4-SNAPSHOT/system/org/opendaylight/controller/sal-clustering-config/9.0.6
Configuring unique name in akka.conf
Configuring hostname in akka.conf
Configuring data and rpc seed nodes in akka.conf
modules = [
	
	{
		name = "inventory"
		namespace = "urn:opendaylight:inventory"
		shard-strategy = "module"
	},
	{
		name = "topology"
		namespace = "urn:TBD:params:xml:ns:yang:network-topology"
		shard-strategy = "module"
	},
	{
		name = "toaster"
		namespace = "http://netconfcentral.org/ns/toaster"
		shard-strategy = "module"
	},
	{
		name = "car"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car"
		shard-strategy = "module"
	},
	{
		name = "people"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people"
		shard-strategy = "module"
	},
	{
		name = "car-people"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people"
		shard-strategy = "module"
	}
]
Configuring replication type in module-shards.conf
################################################
##   NOTE: Manually restart controller to     ##
##         apply configuration.               ##
################################################
Dump akka.conf
+ echo 'Dump akka.conf'
+ cat /tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/akka.conf

odl-cluster-data {
  akka {
    remote {
      artery {
        enabled = on
        transport = tcp
        canonical.hostname = "10.30.171.166"
        canonical.port = 2550
      }
    }

    cluster {
      # Using artery.
      seed-nodes = ["akka://opendaylight-cluster-data@10.30.171.225:2550",
				"akka://opendaylight-cluster-data@10.30.170.179:2550",
				"akka://opendaylight-cluster-data@10.30.171.166:2550"]

      roles = ["member-3"]

      # when under load we might trip a false positive on the failure detector
      # failure-detector {
        # heartbeat-interval = 4 s
        # acceptable-heartbeat-pause = 16s
      # }
    }

    persistence {
      # By default the snapshots/journal directories live in KARAF_HOME. You can choose to put it somewhere else by
      # modifying the following two properties. The directory location specified may be a relative or absolute path. 
      # The relative path is always relative to KARAF_HOME.

      # snapshot-store.local.dir = "target/snapshots"

      # Use lz4 compression for LocalSnapshotStore snapshots
      snapshot-store.local.use-lz4-compression = false
      # Size of blocks for lz4 compression: 64KB, 256KB, 1MB or 4MB
      snapshot-store.local.lz4-blocksize = 256KB
    }
    disable-default-actor-system-quarantined-event-handling = "false"
  }
}
Dump modules.conf
+ echo 'Dump modules.conf'
+ cat /tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/modules.conf
modules = [
	
	{
		name = "inventory"
		namespace = "urn:opendaylight:inventory"
		shard-strategy = "module"
	},
	{
		name = "topology"
		namespace = "urn:TBD:params:xml:ns:yang:network-topology"
		shard-strategy = "module"
	},
	{
		name = "toaster"
		namespace = "http://netconfcentral.org/ns/toaster"
		shard-strategy = "module"
	},
	{
		name = "car"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car"
		shard-strategy = "module"
	},
	{
		name = "people"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people"
		shard-strategy = "module"
	},
	{
		name = "car-people"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people"
		shard-strategy = "module"
	}
]
Dump module-shards.conf
module-shards = [
	{
		name = "default"
		shards = [
			{
				name = "default"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "inventory"
		shards = [
			{
				name="inventory"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "topology"
		shards = [
			{
				name="topology"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "toaster"
		shards = [
			{
				name="toaster"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "car"
		shards = [
			{
				name="car"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "people"
		shards = [
			{
				name="people"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "car-people"
		shards = [
			{
				name="car-people"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	}
]
+ echo 'Dump module-shards.conf'
+ cat /tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/module-shards.conf
Locating config plan to use...
Finished running config plans
Starting member-1 with IP address 10.30.171.225
Warning: Permanently added '10.30.171.225' (ECDSA) to the list of known hosts.
Warning: Permanently added '10.30.171.225' (ECDSA) to the list of known hosts.
Redirecting karaf console output to karaf_console.log
Starting controller...
start: Redirecting Karaf output to /tmp/karaf-0.20.4-SNAPSHOT/data/log/karaf_console.log
Starting member-2 with IP address 10.30.170.179
Warning: Permanently added '10.30.170.179' (ECDSA) to the list of known hosts.
Warning: Permanently added '10.30.170.179' (ECDSA) to the list of known hosts.
Redirecting karaf console output to karaf_console.log
Starting controller...
start: Redirecting Karaf output to /tmp/karaf-0.20.4-SNAPSHOT/data/log/karaf_console.log
Starting member-3 with IP address 10.30.171.166
Warning: Permanently added '10.30.171.166' (ECDSA) to the list of known hosts.
Warning: Permanently added '10.30.171.166' (ECDSA) to the list of known hosts.
Redirecting karaf console output to karaf_console.log
Starting controller...
start: Redirecting Karaf output to /tmp/karaf-0.20.4-SNAPSHOT/data/log/karaf_console.log
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins11787556324465288597.sh
common-functions.sh is being sourced
common-functions environment:
MAVENCONF: /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
ACTUALFEATURES: 
FEATURESCONF: /tmp/karaf-0.20.4-SNAPSHOT/etc/org.apache.karaf.features.cfg
CUSTOMPROP: /tmp/karaf-0.20.4-SNAPSHOT/etc/custom.properties
LOGCONF: /tmp/karaf-0.20.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
MEMCONF: /tmp/karaf-0.20.4-SNAPSHOT/bin/setenv
CONTROLLERMEM: 2048m
AKKACONF: /tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/akka.conf
MODULESCONF: /tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/modules.conf
MODULESHARDSCONF: /tmp/karaf-0.20.4-SNAPSHOT/configuration/initial/module-shards.conf
SUITES: 

+ echo '#################################################'
#################################################
+ echo '##         Verify Cluster is UP                ##'
##         Verify Cluster is UP                ##
+ echo '#################################################'
#################################################
+ create_post_startup_script
+ cat
+ copy_and_run_post_startup_script
+ seed_index=1
++ seq 1 3
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_1_IP
+ echo 'Execute the post startup script on controller 10.30.171.225'
Execute the post startup script on controller 10.30.171.225
+ scp /w/workspace/controller-csit-3node-clustering-tell-all-calcium/post-startup-script.sh 10.30.171.225:/tmp/
Warning: Permanently added '10.30.171.225' (ECDSA) to the list of known hosts.
+ ssh 10.30.171.225 'bash /tmp/post-startup-script.sh 1'
Warning: Permanently added '10.30.171.225' (ECDSA) to the list of known hosts.
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
Waiting up to 3 minutes for controller to come up, checking every 5 seconds...
2025-02-22T23:46:06,957 | INFO  | SystemReadyService-0 | SimpleSystemReadyMonitor         | 297 - org.opendaylight.infrautils.ready-api - 6.0.11 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.]
Controller is UP
2025-02-22T23:46:06,957 | INFO  | SystemReadyService-0 | SimpleSystemReadyMonitor         | 297 - org.opendaylight.infrautils.ready-api - 6.0.11 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.]
Listing all open ports on controller system...
/tmp/post-startup-script.sh: line 51: netstat: command not found
looking for "BindException: Address already in use" in log file
looking for "server is unhealthy" in log file
+ '[' 1 == 0 ']'
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_2_IP
+ echo 'Execute the post startup script on controller 10.30.170.179'
Execute the post startup script on controller 10.30.170.179
+ scp /w/workspace/controller-csit-3node-clustering-tell-all-calcium/post-startup-script.sh 10.30.170.179:/tmp/
Warning: Permanently added '10.30.170.179' (ECDSA) to the list of known hosts.
+ ssh 10.30.170.179 'bash /tmp/post-startup-script.sh 2'
Warning: Permanently added '10.30.170.179' (ECDSA) to the list of known hosts.
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
Waiting up to 3 minutes for controller to come up, checking every 5 seconds...
2025-02-22T23:46:06,398 | INFO  | SystemReadyService-0 | SimpleSystemReadyMonitor         | 297 - org.opendaylight.infrautils.ready-api - 6.0.11 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.]
Controller is UP
2025-02-22T23:46:06,398 | INFO  | SystemReadyService-0 | SimpleSystemReadyMonitor         | 297 - org.opendaylight.infrautils.ready-api - 6.0.11 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.]
Listing all open ports on controller system...
/tmp/post-startup-script.sh: line 51: netstat: command not found
looking for "BindException: Address already in use" in log file
looking for "server is unhealthy" in log file
+ '[' 2 == 0 ']'
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_3_IP
+ echo 'Execute the post startup script on controller 10.30.171.166'
Execute the post startup script on controller 10.30.171.166
+ scp /w/workspace/controller-csit-3node-clustering-tell-all-calcium/post-startup-script.sh 10.30.171.166:/tmp/
Warning: Permanently added '10.30.171.166' (ECDSA) to the list of known hosts.
+ ssh 10.30.171.166 'bash /tmp/post-startup-script.sh 3'
Warning: Permanently added '10.30.171.166' (ECDSA) to the list of known hosts.
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
/tmp/post-startup-script.sh: line 4: netstat: command not found
Waiting up to 3 minutes for controller to come up, checking every 5 seconds...
2025-02-22T23:46:06,256 | INFO  | SystemReadyService-0 | SimpleSystemReadyMonitor         | 297 - org.opendaylight.infrautils.ready-api - 6.0.11 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.]
Controller is UP
2025-02-22T23:46:06,256 | INFO  | SystemReadyService-0 | SimpleSystemReadyMonitor         | 297 - org.opendaylight.infrautils.ready-api - 6.0.11 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.]
Listing all open ports on controller system...
/tmp/post-startup-script.sh: line 51: netstat: command not found
looking for "BindException: Address already in use" in log file
looking for "server is unhealthy" in log file
+ '[' 0 == 0 ']'
+ seed_index=1
+ dump_controller_threads
++ seq 1 3
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_1_IP
+ echo 'Let'\''s take the karaf thread dump'
Let's take the karaf thread dump
+ ssh 10.30.171.225 'sudo ps aux'
Warning: Permanently added '10.30.171.225' (ECDSA) to the list of known hosts.
++ tr -s ' '
++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-tell-all-calcium/ps_before.log
++ grep -v grep
++ cut -f2 '-d '
+ pid=28779
+ echo 'karaf main: org.apache.karaf.main.Main, pid:28779'
karaf main: org.apache.karaf.main.Main, pid:28779
+ ssh 10.30.171.225 '/usr/lib/jvm/java-17-openjdk-amd64/bin/jstack -l 28779'
Warning: Permanently added '10.30.171.225' (ECDSA) to the list of known hosts.
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_2_IP
+ echo 'Let'\''s take the karaf thread dump'
Let's take the karaf thread dump
+ ssh 10.30.170.179 'sudo ps aux'
Warning: Permanently added '10.30.170.179' (ECDSA) to the list of known hosts.
++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-tell-all-calcium/ps_before.log
++ grep -v grep
++ cut -f2 '-d '
++ tr -s ' '
+ pid=28807
+ echo 'karaf main: org.apache.karaf.main.Main, pid:28807'
karaf main: org.apache.karaf.main.Main, pid:28807
+ ssh 10.30.170.179 '/usr/lib/jvm/java-17-openjdk-amd64/bin/jstack -l 28807'
Warning: Permanently added '10.30.170.179' (ECDSA) to the list of known hosts.
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_3_IP
+ echo 'Let'\''s take the karaf thread dump'
Let's take the karaf thread dump
+ ssh 10.30.171.166 'sudo ps aux'
Warning: Permanently added '10.30.171.166' (ECDSA) to the list of known hosts.
++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-tell-all-calcium/ps_before.log
++ grep -v grep
++ cut -f2 '-d '
++ tr -s ' '
+ pid=28747
+ echo 'karaf main: org.apache.karaf.main.Main, pid:28747'
karaf main: org.apache.karaf.main.Main, pid:28747
+ ssh 10.30.171.166 '/usr/lib/jvm/java-17-openjdk-amd64/bin/jstack -l 28747'
Warning: Permanently added '10.30.171.166' (ECDSA) to the list of known hosts.
+ '[' 0 -gt 0 ']'
+ echo 'Generating controller variables...'
Generating controller variables...
++ seq 1 3
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_1_IP
+ odl_variables=' -v ODL_SYSTEM_1_IP:10.30.171.225'
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_2_IP
+ odl_variables=' -v ODL_SYSTEM_1_IP:10.30.171.225 -v ODL_SYSTEM_2_IP:10.30.170.179'
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_3_IP
+ odl_variables=' -v ODL_SYSTEM_1_IP:10.30.171.225 -v ODL_SYSTEM_2_IP:10.30.170.179 -v ODL_SYSTEM_3_IP:10.30.171.166'
+ echo 'Generating mininet variables...'
Generating mininet variables...
++ seq 1 0
+ get_test_suites SUITES
+ local __suite_list=SUITES
+ echo 'Locating test plan to use...'
Locating test plan to use...
+ testplan_filepath=/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/testplans/controller-clustering-tell-calcium.txt
+ '[' '!' -f /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/testplans/controller-clustering-tell-calcium.txt ']'
+ testplan_filepath=/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/testplans/controller-clustering-tell.txt
+ '[' disabled '!=' disabled ']'
+ echo 'Changing the testplan path...'
Changing the testplan path...
+ sed s:integration:/w/workspace/controller-csit-3node-clustering-tell-all-calcium: /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/testplans/controller-clustering-tell.txt
+ cat testplan.txt
# Place the suites in run order:

# disabling all "prefix based" suites. This functionality was newly
# introduced with these tests in 2017. The assumption is that
# development stopped in that area, but may continue at a later
# time. Still, the tests are failing and no resources to solve the
# failures at this time. If/When this changes, we can uncomment
# those suites. Note that every suite is commented out with it's
# accompanying "restart_with_tell_based_true" setup suite.


# /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
# /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/ddb-sanity-prefix-based.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/ddb-sanity-module-based.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/clean_leader_shutdown.robot
# /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
# /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/clean_leader_shutdown_prefbasedshard.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement.robot
# /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
# /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement_prefbasedshard.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/leader_isolation.robot
# /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
# /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/leader_isolation_prefbasedshard.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/client_isolation.robot
# /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
# /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/client_isolation_prefbasedshard.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/listener_stability.robot
# /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
# /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/listener_stability_prefbasedshard.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/master_stability.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot
+ '[' -z '' ']'
++ grep -E -v '(^[[:space:]]*#|^[[:space:]]*$)' testplan.txt
++ tr '\012' ' '
+ suite_list='/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/ddb-sanity-module-based.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/clean_leader_shutdown.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/leader_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/client_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/listener_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot '
+ eval 'SUITES='\''/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/ddb-sanity-module-based.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/clean_leader_shutdown.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/leader_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/client_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/listener_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot '\'''
++ SUITES='/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/ddb-sanity-module-based.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/clean_leader_shutdown.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/leader_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/client_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/listener_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot '
+ echo 'Starting Robot test suites /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/ddb-sanity-module-based.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/clean_leader_shutdown.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/leader_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/client_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/listener_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot  ...'
Starting Robot test suites /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/ddb-sanity-module-based.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/clean_leader_shutdown.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/leader_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/client_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/listener_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot  ...
+ robot -N controller-clustering-tell.txt --removekeywords wuks -e exclude -e skip_if_calcium -v BUNDLEFOLDER:karaf-0.20.4-SNAPSHOT -v BUNDLE_URL:https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.4-SNAPSHOT/karaf-0.20.4-20250222.204906-381.zip -v CONTROLLER:10.30.171.225 -v CONTROLLER1:10.30.170.179 -v CONTROLLER2:10.30.171.166 -v CONTROLLER_USER:jenkins -v JAVA_HOME:/usr/lib/jvm/java-17-openjdk-amd64 -v JDKVERSION:openjdk17 -v JENKINS_WORKSPACE:/w/workspace/controller-csit-3node-clustering-tell-all-calcium -v MININET: -v MININET1: -v MININET2: -v MININET_USER:jenkins -v NEXUSURL_PREFIX:https://nexus.opendaylight.org -v NUM_ODL_SYSTEM:3 -v NUM_TOOLS_SYSTEM:0 -v ODL_STREAM:calcium -v ODL_SYSTEM_IP:10.30.171.225 -v ODL_SYSTEM_1_IP:10.30.171.225 -v ODL_SYSTEM_2_IP:10.30.170.179 -v ODL_SYSTEM_3_IP:10.30.171.166 -v ODL_SYSTEM_USER:jenkins -v TOOLS_SYSTEM_IP: -v TOOLS_SYSTEM_USER:jenkins -v USER_HOME:/home/jenkins -v IS_KARAF_APPL:True -v WORKSPACE:/tmp /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/ddb-sanity-module-based.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/clean_leader_shutdown.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/leader_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/client_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/listener_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot
==============================================================================
controller-clustering-tell.txt                                                
==============================================================================
controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell...
==============================================================================
Stop_All_Members :: Stop every odl node.                              | FAIL |
Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS |
------------------------------------------------------------------------------
Start_All_And_Sync :: Start each member and wait for sync.            | FAIL |
Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: HTTPError: 500 Server Error: Server Error for url: http://10.30.170.179:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore
------------------------------------------------------------------------------
controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL |
3 tests, 1 passed, 2 failed
==============================================================================
controller-clustering-tell.txt.Ddb-Sanity-Module-Based :: DOMDataBroker tes...
==============================================================================
Get_Shard_Role :: Get shard role.                                     | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_ddb_sanity_module_based_get_shard_role"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_ddb_sanity_module_based_get_shard_role&order=bug_status"

HTTPError: 404 Client Error: Not Found for url: http://10.30.171.225:8181/restconf/operations/cluster-admin:get-shard-role
------------------------------------------------------------------------------
Subscribe_Listener_To_Leader :: Subscribe listener to leader.         | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_ddb_sanity_module_based_subscribe_listener_to_leader"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_ddb_sanity_module_based_subscribe_listener_to_leader&order=bug_status"

HTTPError: 500 Server Error: Server Error for url: http://10.30.170.179:8181/jolokia/read/org.opendaylight.controller:Category=Shards,name=member-2-shard-default-config,type=DistributedConfigDatastore
------------------------------------------------------------------------------
Subscribe_Listener_To_Follower :: Subscribe listener to follower.     | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_ddb_sanity_module_based_subscribe_listener_to_follower"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_ddb_sanity_module_based_subscribe_listener_to_follower&order=bug_status"

HTTPError: 500 Server Error: Server Error for url: http://10.30.170.179:8181/jolokia/read/org.opendaylight.controller:Category=Shards,name=member-2-shard-default-config,type=DistributedConfigDatastore
------------------------------------------------------------------------------
Make_Leader_Local :: Make the loeader local and verify.               | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_ddb_sanity_module_based_make_leader_local"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_ddb_sanity_module_based_make_leader_local&order=bug_status"

HTTPError: 500 Server Error: Server Error for url: http://10.30.170.179:8181/jolokia/read/org.opendaylight.controller:Category=Shards,name=member-2-shard-default-config,type=DistributedConfigDatastore
------------------------------------------------------------------------------
Remove_Leader_Shard_Replica_And_Add_It_Back :: Remove and add shar... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_ddb_sanity_module_based_remove_leader_shard_replica_and_add_it_back"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_ddb_sanity_module_based_remove_leader_shard_replica_and_add_it_back&order=bug_status"

HTTPError: 500 Server Error: Server Error for url: http://10.30.170.179:8181/jolokia/read/org.opendaylight.controller:Category=Shards,name=member-2-shard-default-config,type=DistributedConfigDatastore
------------------------------------------------------------------------------
Remove_Follower_Shard_Replica_And_Add_It_Back :: Remove and add sh... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_ddb_sanity_module_based_remove_follower_shard_replica_and_add_it_back"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_ddb_sanity_module_based_remove_follower_shard_replica_and_add_it_back&order=bug_status"

HTTPError: 500 Server Error: Server Error for url: http://10.30.170.179:8181/jolokia/read/org.opendaylight.controller:Category=Shards,name=member-2-shard-default-config,type=DistributedConfigDatastore
------------------------------------------------------------------------------
Write_Transactions_One_Node_Leader :: Write transactions.             | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_ddb_sanity_module_based_write_transactions_one_node_leader"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_ddb_sanity_module_based_write_transactions_one_node_leader&order=bug_status"

HTTPError: 500 Server Error: Server Error for url: http://10.30.170.179:8181/jolokia/read/org.opendaylight.controller:Category=Shards,name=member-2-shard-default-config,type=DistributedConfigDatastore
------------------------------------------------------------------------------
Write_Transactions_One_Node_Follower :: Write transactions.           | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_ddb_sanity_module_based_write_transactions_one_node_follower"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_ddb_sanity_module_based_write_transactions_one_node_follower&order=bug_status"

HTTPError: 500 Server Error: Server Error for url: http://10.30.170.179:8181/jolokia/read/org.opendaylight.controller:Category=Shards,name=member-2-shard-default-config,type=DistributedConfigDatastore
------------------------------------------------------------------------------
Write_Transactions_All_Nodes :: Write transactions.                   | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_ddb_sanity_module_based_write_transactions_all_nodes"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_ddb_sanity_module_based_write_transactions_all_nodes&order=bug_status"

[ 200 | 201 | 204 ] does not contain value '500'.
------------------------------------------------------------------------------
controller-clustering-tell.txt.Ddb-Sanity-Module-Based :: DOMDataB... | FAIL |
9 tests, 0 passed, 9 failed
==============================================================================
controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell...
==============================================================================
Stop_All_Members :: Stop every odl node.                              | FAIL |
Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS |
------------------------------------------------------------------------------
Start_All_And_Sync :: Start each member and wait for sync.            | FAIL |
Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235c5f050>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL |
3 tests, 1 passed, 2 failed
==============================================================================
controller-clustering-tell.txt.Clean Leader Shutdown :: DOMDataBroker testi...
==============================================================================
Local_Leader_Shutdown :: Shutdown the leader on the same node as t... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_clean_leader_shutdown_local_leader_shutdown"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_clean_leader_shutdown_local_leader_shutdown&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235c51a10>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Restart :: Restart odl                                                | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_clean_leader_shutdown_restart"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_clean_leader_shutdown_restart&order=bug_status"

Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Remote_Leader_Shutdown :: Shutdown the leader on different node as... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_clean_leader_shutdown_remote_leader_shutdown"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_clean_leader_shutdown_remote_leader_shutdown&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f623876bbd0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Clean Leader Shutdown :: DOMDataBro... | FAIL |
3 tests, 0 passed, 3 failed
==============================================================================
controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell...
==============================================================================
Stop_All_Members :: Stop every odl node.                              | FAIL |
Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS |
------------------------------------------------------------------------------
Start_All_And_Sync :: Start each member and wait for sync.            | FAIL |
Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62359c04d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL |
3 tests, 1 passed, 2 failed
==============================================================================
controller-clustering-tell.txt.Explicit Leader Movement :: DOMDataBroker te...
==============================================================================
Local_To_Remote_Movement :: Leader moves from local to remote node... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_explicit_leader_movement_local_to_remote_movement"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_explicit_leader_movement_local_to_remote_movement&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62386c3050>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Restart1 :: Restart odl                                               | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_explicit_leader_movement_restart1"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_explicit_leader_movement_restart1&order=bug_status"

Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Remote_To_Remote_Movement :: Leader moves from one remote to other... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_explicit_leader_movement_remote_to_remote_movement"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_explicit_leader_movement_remote_to_remote_movement&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235aaa590>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Restart2 :: Restart odl                                               | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_explicit_leader_movement_restart2"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_explicit_leader_movement_restart2&order=bug_status"

Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Remote_To_Local_Movement :: Leader moves from remote to local node... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_explicit_leader_movement_remote_to_local_movement"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_explicit_leader_movement_remote_to_local_movement&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6236336610>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Explicit Leader Movement :: DOMData... | FAIL |
5 tests, 0 passed, 5 failed
==============================================================================
controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell...
==============================================================================
Stop_All_Members :: Stop every odl node.                              | FAIL |
Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS |
------------------------------------------------------------------------------
Start_All_And_Sync :: Start each member and wait for sync.            | FAIL |
Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62365b5710>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL |
3 tests, 1 passed, 2 failed
==============================================================================
controller-clustering-tell.txt.Leader Isolation :: DOMDataBroker testing: L...
==============================================================================
Healing_Within_Request_Timeout :: The isolated node (leader) is re... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_leader_isolation_healing_within_request_timeout"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_leader_isolation_healing_within_request_timeout&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235d9ba10>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Restart :: Restart odl                                                | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_leader_isolation_restart"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_leader_isolation_restart&order=bug_status"

Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Healing_After_Request_Timeout :: The isolated node (leader) is rej... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_leader_isolation_healing_after_request_timeout"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_leader_isolation_healing_after_request_timeout&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f623585f7d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Leader Isolation :: DOMDataBroker t... | FAIL |
3 tests, 0 passed, 3 failed
==============================================================================
controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell...
==============================================================================
Stop_All_Members :: Stop every odl node.                              | FAIL |
Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS |
------------------------------------------------------------------------------
Start_All_And_Sync :: Start each member and wait for sync.            | FAIL |
Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62386e5490>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL |
3 tests, 1 passed, 2 failed
==============================================================================
controller-clustering-tell.txt.Client Isolation :: DOMDataBroker testing: C...
==============================================================================
Producer_On_Shard_Leader_Node_ChainedTx :: Client isolation with p... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_client_isolation_producer_on_shard_leader_node_chainedtx"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_client_isolation_producer_on_shard_leader_node_chainedtx&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235bb2a90>: Failed to establish a new connection: [Errno 111] Connection refused'))

Also keyword teardown failed:
Several failures occurred:

1) Variable '${client_node_dst}' not found.

2) Variable '${client_node_dst}' not found.

3) Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 1 minute 10 seconds. The last error was: Several failures occurred:

1) ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f...
    [ Message content over the limit has been removed. ]

173) Resolving variable '${response.text}' failed: AttributeError: 'NoneType' object has no attribute 'text'

174) Evaluating expression "json.loads('''None''')" failed: JSONDecodeError: Expecting value: line 1 column 1 (char 0)

175) TypeError: Expected argument 1 to be a dictionary, got None instead.

176) TypeError: Expected argument 1 to be a dictionary, got None instead.

177) Unrecognized Raft state: None

178) No leader found.

179) Too many Leaders.

180) IndexError: Given index 0 is out of the range 0--1.

4) KeyError: 'threads'
------------------------------------------------------------------------------
Restart1 :: Restart odl                                               | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_client_isolation_restart1"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_client_isolation_restart1&order=bug_status"

Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Producer_On_Shard_Leader_Node_SimpleTx :: Client isolation with pr... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_client_isolation_producer_on_shard_leader_node_simpletx"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_client_isolation_producer_on_shard_leader_node_simpletx&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6236449cd0>: Failed to establish a new connection: [Errno 111] Connection refused'))

Also keyword teardown failed:
Several failures occurred:

1) Variable '${client_node_dst}' not found.

2) Variable '${client_node_dst}' not found.

3) Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 1 minute 10 seconds. The last error was: Several failures occurred:

1) ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f...
    [ Message content over the limit has been removed. ]

173) Resolving variable '${response.text}' failed: AttributeError: 'NoneType' object has no attribute 'text'

174) Evaluating expression "json.loads('''None''')" failed: JSONDecodeError: Expecting value: line 1 column 1 (char 0)

175) TypeError: Expected argument 1 to be a dictionary, got None instead.

176) TypeError: Expected argument 1 to be a dictionary, got None instead.

177) Unrecognized Raft state: None

178) No leader found.

179) Too many Leaders.

180) IndexError: Given index 0 is out of the range 0--1.

4) KeyError: 'threads'
------------------------------------------------------------------------------
Restart2 :: Restart odl                                               | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_client_isolation_restart2"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_client_isolation_restart2&order=bug_status"

Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Producer_On_Shard_Non_Leader_Node_ChainedTx :: Client isolation wi... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_client_isolation_producer_on_shard_non_leader_node_chainedtx"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_client_isolation_producer_on_shard_non_leader_node_chainedtx&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235f48190>: Failed to establish a new connection: [Errno 111] Connection refused'))

Also keyword teardown failed:
Several failures occurred:

1) Variable '${client_node_dst}' not found.

2) Variable '${client_node_dst}' not found.

3) Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 1 minute 10 seconds. The last error was: Several failures occurred:

1) ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f...
    [ Message content over the limit has been removed. ]

173) Resolving variable '${response.text}' failed: AttributeError: 'NoneType' object has no attribute 'text'

174) Evaluating expression "json.loads('''None''')" failed: JSONDecodeError: Expecting value: line 1 column 1 (char 0)

175) TypeError: Expected argument 1 to be a dictionary, got None instead.

176) TypeError: Expected argument 1 to be a dictionary, got None instead.

177) Unrecognized Raft state: None

178) No leader found.

179) Too many Leaders.

180) IndexError: Given index 0 is out of the range 0--1.

4) KeyError: 'threads'
------------------------------------------------------------------------------
Restart3 :: Restart odl                                               | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_client_isolation_restart3"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_client_isolation_restart3&order=bug_status"

Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Producer_On_Shard_Non_Leader_Node_SimpleTx :: Client isolation wit... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_client_isolation_producer_on_shard_non_leader_node_simpletx"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_client_isolation_producer_on_shard_non_leader_node_simpletx&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235fc0ed0>: Failed to establish a new connection: [Errno 111] Connection refused'))

Also keyword teardown failed:
Several failures occurred:

1) Variable '${client_node_dst}' not found.

2) Variable '${client_node_dst}' not found.

3) Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 1 minute 10 seconds. The last error was: Several failures occurred:

1) ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f...
    [ Message content over the limit has been removed. ]

173) Resolving variable '${response.text}' failed: AttributeError: 'NoneType' object has no attribute 'text'

174) Evaluating expression "json.loads('''None''')" failed: JSONDecodeError: Expecting value: line 1 column 1 (char 0)

175) TypeError: Expected argument 1 to be a dictionary, got None instead.

176) TypeError: Expected argument 1 to be a dictionary, got None instead.

177) Unrecognized Raft state: None

178) No leader found.

179) Too many Leaders.

180) IndexError: Given index 0 is out of the range 0--1.

4) KeyError: 'threads'
------------------------------------------------------------------------------
controller-clustering-tell.txt.Client Isolation :: DOMDataBroker t... | FAIL |
7 tests, 0 passed, 7 failed
==============================================================================
controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell...
==============================================================================
Stop_All_Members :: Stop every odl node.                              | FAIL |
Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS |
------------------------------------------------------------------------------
Start_All_And_Sync :: Start each member and wait for sync.            | FAIL |
Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235de3150>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL |
3 tests, 1 passed, 2 failed
==============================================================================
controller-clustering-tell.txt.Listener Stability :: DOMDataBroker testing:...
==============================================================================
Move_Leader_From_Listener_Local_To_Remote :: Listener runs on lead... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_listener_stability_move_leader_from_listener_local_to_remote"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_listener_stability_move_leader_from_listener_local_to_remote&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62363cebd0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Restart_1 :: Restart odl.                                             | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_listener_stability_restart_1"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_listener_stability_restart_1&order=bug_status"

Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Move_Leader_From_Listener_Remote_To_Other_Remote :: Listener runs ... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_listener_stability_move_leader_from_listener_remote_to_other_remote"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_listener_stability_move_leader_from_listener_remote_to_other_remote&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f623638a490>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Restart_2 :: Restart odl.                                             | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_listener_stability_restart_2"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_listener_stability_restart_2&order=bug_status"

Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Move_Leader_From_Listener_Remote_To_Local :: Listener runs on foll... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_listener_stability_move_leader_from_listener_remote_to_local"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_listener_stability_move_leader_from_listener_remote_to_local&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6236204850>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Listener Stability :: DOMDataBroker... | FAIL |
5 tests, 0 passed, 5 failed
==============================================================================
controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell...
==============================================================================
Stop_All_Members :: Stop every odl node.                              | FAIL |
Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS |
------------------------------------------------------------------------------
Start_All_And_Sync :: Start each member and wait for sync.            | FAIL |
Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235aff290>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL |
3 tests, 1 passed, 2 failed
==============================================================================
controller-clustering-tell.txt.Master Stability :: Cluster Singleton testin...
==============================================================================
Register_Singleton_Constant_On_Each_Node_And_Verify :: Register a ... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_master_stability_register_singleton_constant_on_each_node_and_verify"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_master_stability_register_singleton_constant_on_each_node_and_verify&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/operations/odl-mdsal-lowlevel-control:register-singleton-constant (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235e499d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Unregister_Singleton_Constant_On_Non_Master_Node :: Unregister the... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_master_stability_unregister_singleton_constant_on_non_master_node"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_master_stability_unregister_singleton_constant_on_non_master_node&order=bug_status"

Variable '@{cs_candidates}' not found. Did you mean:
    @{cs_exp_candidates}
------------------------------------------------------------------------------
Monitor_Stability_While_Unregistered :: Verify that the owner rema... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_master_stability_monitor_stability_while_unregistered"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_master_stability_monitor_stability_while_unregistered&order=bug_status"

Variable '${cs_owner}' not found.
------------------------------------------------------------------------------
Reregister_Singleton_Constant :: Re-register the unregistered cand... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_master_stability_reregister_singleton_constant"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_master_stability_reregister_singleton_constant&order=bug_status"

Variable '${unregistered_node}' not found.
------------------------------------------------------------------------------
Verify_Stability_After_Reregistration :: Verify that the owner rem... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_master_stability_verify_stability_after_reregistration"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_master_stability_verify_stability_after_reregistration&order=bug_status"

Variable '${cs_owner}' not found.
------------------------------------------------------------------------------
Unregister_Singleton_Constant_On_Each_Node :: Unregister the appli... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_master_stability_unregister_singleton_constant_on_each_node"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_master_stability_unregister_singleton_constant_on_each_node&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/operations/odl-mdsal-lowlevel-control:unregister-singleton-constant (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62358fcd50>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Master Stability :: Cluster Singlet... | FAIL |
6 tests, 0 passed, 6 failed
==============================================================================
controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell...
==============================================================================
Stop_All_Members :: Stop every odl node.                              | FAIL |
Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS |
------------------------------------------------------------------------------
Start_All_And_Sync :: Start each member and wait for sync.            | FAIL |
Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62343a81d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL |
3 tests, 1 passed, 2 failed
==============================================================================
controller-clustering-tell.txt.Partition And Heal :: Cluster Singleton test...
==============================================================================
Register_Singleton_Constant_On_Each_Node :: Register a candidate a... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_partition_and_heal_register_singleton_constant_on_each_node"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_partition_and_heal_register_singleton_constant_on_each_node&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/operations/odl-mdsal-lowlevel-control:register-singleton-constant (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f623594e3d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Verify_Singleton_Constant_On_Each_Node :: Store the owner and cand... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_partition_and_heal_verify_singleton_constant_on_each_node"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_partition_and_heal_verify_singleton_constant_on_each_node&order=bug_status"

Could not parse owner and candidates for device get-singleton-constant-service']
------------------------------------------------------------------------------
Isolate_Owner_Node :: Isolate the cluster node which is the owner.... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_partition_and_heal_isolate_owner_node"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_partition_and_heal_isolate_owner_node&order=bug_status"

Variable '${cs_owner}' not found.
------------------------------------------------------------------------------
Monitor_Stability_While_Isolated :: Monitor the stability of the s... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_partition_and_heal_monitor_stability_while_isolated"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_partition_and_heal_monitor_stability_while_isolated&order=bug_status"

Invalid IF condition: Evaluating expression '"${index}" == "${cs_isolated_index}"' failed: Variable '${cs_isolated_index}' not found.
------------------------------------------------------------------------------
Rejoin_Isolated_node :: Rejoin isolated node.                         | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_partition_and_heal_rejoin_isolated_node"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_partition_and_heal_rejoin_isolated_node&order=bug_status"

Variable '${cs_isolated_index}' not found.
------------------------------------------------------------------------------
Unregister_Singleton_Constant_On_Each_Node :: Unregister the appli... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_partition_and_heal_unregister_singleton_constant_on_each_node"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_partition_and_heal_unregister_singleton_constant_on_each_node&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/operations/odl-mdsal-lowlevel-control:unregister-singleton-constant (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235b04f90>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Partition And Heal :: Cluster Singl... | FAIL |
6 tests, 0 passed, 6 failed
==============================================================================
controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell...
==============================================================================
Stop_All_Members :: Stop every odl node.                              | FAIL |
Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS |
------------------------------------------------------------------------------
Start_All_And_Sync :: Start each member and wait for sync.            | FAIL |
Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62365bf510>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL |
3 tests, 1 passed, 2 failed
==============================================================================
controller-clustering-tell.txt.Chasing The Leader :: Cluster Singleton test...
==============================================================================
Register_Candidates :: Register a candidate application on each no... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_chasing_the_leader_register_candidates"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_chasing_the_leader_register_candidates&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/operations/odl-mdsal-lowlevel-control:register-flapping-singleton (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6236411910>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Do_Nothing :: Do nothing for the time of the test duration, becaus... | PASS |
------------------------------------------------------------------------------
Unregister_Candidates_And_Validate_Criteria :: Unregister the test... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_chasing_the_leader_unregister_candidates_and_validate_criteria"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_chasing_the_leader_unregister_candidates_and_validate_criteria&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/operations/odl-mdsal-lowlevel-control:unregister-flapping-singleton (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235cff590>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Chasing The Leader :: Cluster Singl... | FAIL |
3 tests, 1 passed, 2 failed
==============================================================================
controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell...
==============================================================================
Stop_All_Members :: Stop every odl node.                              | FAIL |
Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS |
------------------------------------------------------------------------------
Start_All_And_Sync :: Start each member and wait for sync.            | FAIL |
Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235b5dd50>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL |
3 tests, 1 passed, 2 failed
==============================================================================
controller-clustering-tell.txt.Carpeople Crud :: Suite for performing basic...
==============================================================================
Add_Cars_To_Leader :: Add 30 cars to car Leader by one big PUT.       | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62386c3750>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
See_Added_Cars_On_Leader :: GET response from Leader should match ... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62386c3750>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
See_Added_Cars_On_Followers :: The same check on other members.       | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62386c3750>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Add_People_To_First_Follower :: Add 30 people to people first Foll... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62386c3750>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
See_Added_People_On_Leader :: GET response from Leader should matc... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62386c3750>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
See_Added_People_On_Followers :: The same check on other members.     | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62386c3750>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Buy_Cars_On_Leader :: Buy some cars on car-people Leader, loop of ... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62386c3750>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Buy_Cars_On_Followers :: On each Follower buy corresponding ID seg... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62386c3750>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
See_Added_CarPeople_On_Leader :: GET car-person mappings from Lead... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62386c3750>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
See_Added_CarPeople_On_Followers :: The same check on other members.  | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62386c3750>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Delete_All_CarPeople_On_Leader :: DELETE car-people container. No ... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62386c3750>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Delete_All_People_On_Leader :: DELETE people container. No verific... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62386c3750>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Delete_All_Cars_On_Leader :: DELETE cars container. No verificatio... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62386c3750>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Carpeople Crud :: Suite for perform... | FAIL |
Suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62386c3750>: Failed to establish a new connection: [Errno 111] Connection refused'))

13 tests, 0 passed, 13 failed
==============================================================================
controller-clustering-tell.txt.Car Failover Crud :: Suite mixing basic oper...
==============================================================================
Add_Original_Cars_On_Old_Leader_And_Verify :: Add initial cars on ... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62358d6150>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Stop_Original_Car_Leader :: Stop the car Leader to cause a new lea... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62358d6150>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Wait_For_New_Leader :: Wait until new car Leader is elected.          | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62358d6150>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
See_Original_Cars_On_New_Leader :: GET cars from new Leader, shoul... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62358d6150>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
See_Original_Cars_On_New_Followers :: The same check on other exis... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62358d6150>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Delete_Original_Cars_On_New_Leader :: Delete cars on the new Leader.  | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62358d6150>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Add_Leader_Cars_On_New_Leader :: Add cars on the new Leader.          | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62358d6150>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
See_Leader_Cars_On_New_Leader :: GET cars from new Leader, should ... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62358d6150>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
See_Leader_Cars_On_New_Followers :: The same check on other existi... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62358d6150>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Delete_Leader_Cars_On_New_First_Follower :: Delete cars in new fir... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62358d6150>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Add_Follower_Cars_On_New_First_Follower :: Add cars on the new fir... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62358d6150>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
See_Folower_Cars_On_New_Leader :: Get cars from the new Leader, sh... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62358d6150>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
See_Follower_Cars_On_New_Followers :: The same check on other exis... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62358d6150>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Start_Old_Car_Leader :: Start the stopped member without deleting ... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62358d6150>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
See_Folower_Cars_On_Old_Leader :: GET cars from the restarted memb... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62358d6150>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Delete_Follower_Cars_On_New_Leader :: Delete cars on the last Leader. | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62358d6150>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Car Failover Crud :: Suite mixing b... | FAIL |
Suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62358d6150>: Failed to establish a new connection: [Errno 111] Connection refused'))

16 tests, 0 passed, 16 failed
==============================================================================
controller-clustering-tell.txt.Car Persistence Recovery :: This test restar...
==============================================================================
Add_Cars_On_Leader_And_Verify :: Single big PUT to datastore to ad... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62366588d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Stop_All_Members :: Stop all controllers.                             | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62366588d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Start_All_Members :: Start all controllers (should restore the per... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62366588d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Memorize_Leader_And_Followers :: Locate current Leader of car Shard.  | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62366588d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
See_Cars_On_Leader :: GET cars from Leader, should match the PUT d... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62366588d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
See_Cars_On_Followers :: The same check on other members.             | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62366588d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Delete_Cars_On_Leader :: Delete cars on the new Leader.               | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62366588d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Car Persistence Recovery :: This te... | FAIL |
Suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f62366588d0>: Failed to establish a new connection: [Errno 111] Connection refused'))

7 tests, 0 passed, 7 failed
==============================================================================
controller-clustering-tell.txt.Buycar Failover :: This test focuses on test...
==============================================================================
Add_Cars_To_Leader_And_Verify :: Add all needed cars to car Leader... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f623615ddd0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Add_People_To_First_Follower_And_Verify :: Add all needed people t... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f623615ddd0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Buy_Cars_On_Leader_And_Verify :: Buy some cars on the leader member.  | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f623615ddd0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Buy_Cars_On_Follower_And_Verify :: Buy some cars on the first foll... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f623615ddd0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Reboot_People_Leader :: Previous people Leader is rebooted. We sho... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f623615ddd0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Buy_Cars_On_Leader_After_Reboot_And_Verify :: Buy some cars on the... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f623615ddd0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Buy_Cars_On_Follower_After_Reboot_And_Verify :: Buy some cars on t... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f623615ddd0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Delete_All_CarPeople :: DELETE car-people container. No verificati... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f623615ddd0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Delete_All_People :: DELETE people container. No verification beyo... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f623615ddd0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Delete_All_Cars :: DELETE cars container. No verification beyond h... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f623615ddd0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Buycar Failover :: This test focuse... | FAIL |
Suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f623615ddd0>: Failed to establish a new connection: [Errno 111] Connection refused'))

10 tests, 0 passed, 10 failed
==============================================================================
controller-clustering-tell.txt.Entity Isolate :: Suite for performing membe...
==============================================================================
Check All Shards Before Isolate :: Check all shards in controller.    | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235cfee50>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Isolate Entity Leader :: Isolate the entity-ownership Leader to ca... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235cfee50>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Check All Shards After Isolate :: Check all shards in controller.     | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235cfee50>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Rejoin Entity Leader :: Rejoin the entity-ownership Leader.           | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235cfee50>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Check All Shards After Rejoin :: Check all shards in controller.      | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235cfee50>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Entity Isolate :: Suite for perform... | FAIL |
Suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235cfee50>: Failed to establish a new connection: [Errno 111] Connection refused'))

5 tests, 0 passed, 5 failed
==============================================================================
controller-clustering-tell.txt.Buycar Failover Isolation :: This test focus...
==============================================================================
Add_Cars_To_Leader_And_Verify :: Add all needed cars to car Leader... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235bb05d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Add_People_To_First_Follower_And_Verify :: Add all needed people t... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235bb05d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Buy_Cars_On_Leader_And_Verify :: Buy some cars on the leader member.  | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235bb05d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Buy_Cars_On_Follower_And_Verify :: Buy some cars on the first foll... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235bb05d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Isolate_and_Rejoin_People_Leader :: Previous people Leader is isol... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235bb05d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Buy_Cars_On_Leader_After_Rejoin_And_Verify :: Buy some cars on the... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235bb05d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Buy_Cars_On_Follower_After_Rejoin_And_Verify :: Buy some cars on t... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235bb05d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Delete_All_CarPeople :: DELETE car-people container. No verificati... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235bb05d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Delete_All_People :: DELETE people container. No verification beyo... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235bb05d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Delete_All_Cars :: DELETE cars container. No verification beyond h... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235bb05d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Buycar Failover Isolation :: This t... | FAIL |
Suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235bb05d0>: Failed to establish a new connection: [Errno 111] Connection refused'))

10 tests, 0 passed, 10 failed
==============================================================================
controller-clustering-tell.txt.Car Failover Crud Isolation :: Suite mixing ...
==============================================================================
Add_Original_Cars_On_Old_Leader_And_Verify :: Add initial cars on ... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235ecc490>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Isolate_Original_Car_Leader :: Isolate the car Leader to cause a n... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235ecc490>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Wait_For_New_Leader :: Wait until new car Leader is elected.          | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235ecc490>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
See_Original_Cars_On_New_Leader :: GET cars from new Leader, shoul... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235ecc490>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
See_Original_Cars_On_New_Followers :: The same check on other exis... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235ecc490>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Delete_Original_Cars_On_New_Leader :: Delete cars on the new Leader.  | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235ecc490>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Add_Leader_Cars_On_New_Leader :: Add cars on the new Leader.          | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235ecc490>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
See_Leader_Cars_On_New_Leader :: GET cars from new Leader, should ... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235ecc490>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
See_Leader_Cars_On_New_Followers :: The same check on other existi... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235ecc490>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Delete_Leader_Cars_On_New_First_Follower :: Delete cars in new fir... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235ecc490>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Add_Follower_Cars_On_New_First_Follower :: Add cars on the new fir... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235ecc490>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
See_Folower_Cars_On_New_Leader :: Get cars from the new Leader, sh... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235ecc490>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
See_Follower_Cars_On_New_Followers :: The same check on other exis... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235ecc490>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Rejoin_Old_Car_Leader :: Rejoin the isolated member without deleti... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235ecc490>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
See_Folower_Cars_On_Old_Leader :: GET cars from the restarted memb... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235ecc490>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Delete_Follower_Cars_On_New_Leader :: Delete cars on the last Leader. | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235ecc490>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Car Failover Crud Isolation :: Suit... | FAIL |
Suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235ecc490>: Failed to establish a new connection: [Errno 111] Connection refused'))

16 tests, 0 passed, 16 failed
==============================================================================
controller-clustering-tell.txt.Car Outage Corners :: Cluster suite for test...
==============================================================================
Stop_Majority_Of_The_Followers :: Stop half plus one car Follower ... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235afd550>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Attempt_To_Add_Cars_To_Leader :: Adding cars should fail, as major... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235afd550>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Start_Tipping_Follower :: Start one Follower member without persis... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235afd550>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Add_Cars_On_Tipping_Follower :: Add cars on the tipping Follower.     | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235afd550>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
See_Cars_On_Existing_Members :: On each up member: GET cars, shoul... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235afd550>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Start_Other_Followers :: Start other followers without persisted d... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235afd550>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
See_Cars_On_New_Follower_Leader :: GET cars from a new follower to... | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235afd550>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Delete_Cars_On_Leader :: Delete cars on Leader.                       | FAIL |
Parent suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235afd550>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Car Outage Corners :: Cluster suite... | FAIL |
Suite setup failed:
ConnectionError: HTTPConnectionPool(host='10.30.171.225', port=8181): Max retries exceeded with url: /rests/data/ietf-yang-library:modules-state?content=nonconfig (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f6235afd550>: Failed to establish a new connection: [Errno 111] Connection refused'))

8 tests, 0 passed, 8 failed
==============================================================================
controller-clustering-tell.txt                                        | FAIL |
162 tests, 11 passed, 151 failed
==============================================================================
Output:  /w/workspace/controller-csit-3node-clustering-tell-all-calcium/output.xml
Log:     /w/workspace/controller-csit-3node-clustering-tell-all-calcium/log.html
Report:  /w/workspace/controller-csit-3node-clustering-tell-all-calcium/report.html
+ true
+ echo 'Examining the files in data/log and checking filesize'
Examining the files in data/log and checking filesize
+ ssh 10.30.171.225 'ls -altr /tmp/karaf-0.20.4-SNAPSHOT/data/log/'
Warning: Permanently added '10.30.171.225' (ECDSA) to the list of known hosts.
total 3916
drwxrwxr-x 2 jenkins jenkins    4096 Feb 22 23:45 .
-rw-rw-r-- 1 jenkins jenkins    1720 Feb 22 23:45 karaf_console.log
drwxrwxr-x 4 jenkins jenkins    4096 Feb 23 03:49 ..
-rw-rw-r-- 1 jenkins jenkins 3996340 Feb 23 04:00 karaf.log
+ ssh 10.30.171.225 'du -hs /tmp/karaf-0.20.4-SNAPSHOT/data/log/*'
Warning: Permanently added '10.30.171.225' (ECDSA) to the list of known hosts.
3.9M	/tmp/karaf-0.20.4-SNAPSHOT/data/log/karaf.log
4.0K	/tmp/karaf-0.20.4-SNAPSHOT/data/log/karaf_console.log
+ ssh 10.30.170.179 'ls -altr /tmp/karaf-0.20.4-SNAPSHOT/data/log/'
Warning: Permanently added '10.30.170.179' (ECDSA) to the list of known hosts.
total 3940
drwxrwxr-x 2 jenkins jenkins    4096 Feb 22 23:45 .
-rw-rw-r-- 1 jenkins jenkins   10125 Feb 22 23:55 karaf_console.log
drwxrwxr-x 4 jenkins jenkins    4096 Feb 23 03:49 ..
-rw-rw-r-- 1 jenkins jenkins 4012415 Feb 23 04:01 karaf.log
+ ssh 10.30.170.179 'du -hs /tmp/karaf-0.20.4-SNAPSHOT/data/log/*'
Warning: Permanently added '10.30.170.179' (ECDSA) to the list of known hosts.
3.9M	/tmp/karaf-0.20.4-SNAPSHOT/data/log/karaf.log
12K	/tmp/karaf-0.20.4-SNAPSHOT/data/log/karaf_console.log
+ ssh 10.30.171.166 'ls -altr /tmp/karaf-0.20.4-SNAPSHOT/data/log/'
Warning: Permanently added '10.30.171.166' (ECDSA) to the list of known hosts.
total 585880
drwxrwxr-x 2 jenkins jenkins      4096 Feb 22 23:45 .
-rw-rw-r-- 1 jenkins jenkins      1720 Feb 22 23:45 karaf_console.log
-rw-rw-r-- 1 jenkins jenkins 599921599 Feb 23 01:29 karaf.log
drwxrwxr-x 4 jenkins jenkins      4096 Feb 23 03:49 ..
+ ssh 10.30.171.166 'du -hs /tmp/karaf-0.20.4-SNAPSHOT/data/log/*'
Warning: Permanently added '10.30.171.166' (ECDSA) to the list of known hosts.
573M	/tmp/karaf-0.20.4-SNAPSHOT/data/log/karaf.log
4.0K	/tmp/karaf-0.20.4-SNAPSHOT/data/log/karaf_console.log
+ set +e
++ seq 1 3
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_1_IP
+ echo 'Let'\''s take the karaf thread dump again'
Let's take the karaf thread dump again
+ ssh 10.30.171.225 'sudo ps aux'
Warning: Permanently added '10.30.171.225' (ECDSA) to the list of known hosts.
++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-tell-all-calcium/ps_after.log
++ tr -s ' '
++ grep -v grep
++ cut -f2 '-d '
+ pid=35048
+ echo 'karaf main: org.apache.karaf.main.Main, pid:35048'
karaf main: org.apache.karaf.main.Main, pid:35048
+ ssh 10.30.171.225 '/usr/lib/jvm/java-17-openjdk-amd64/bin/jstack -l 35048'
Warning: Permanently added '10.30.171.225' (ECDSA) to the list of known hosts.
+ echo 'killing karaf process...'
killing karaf process...
+ ssh 10.30.171.225 bash -c 'ps axf | grep karaf | grep -v grep | awk '\''{print "kill -9 " $1}'\'' | sh'
Warning: Permanently added '10.30.171.225' (ECDSA) to the list of known hosts.
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_2_IP
+ echo 'Let'\''s take the karaf thread dump again'
Let's take the karaf thread dump again
+ ssh 10.30.170.179 'sudo ps aux'
Warning: Permanently added '10.30.170.179' (ECDSA) to the list of known hosts.
++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-tell-all-calcium/ps_after.log
++ grep -v grep
++ cut -f2 '-d '
++ tr -s ' '
+ pid=28807
+ echo 'karaf main: org.apache.karaf.main.Main, pid:28807'
karaf main: org.apache.karaf.main.Main, pid:28807
+ ssh 10.30.170.179 '/usr/lib/jvm/java-17-openjdk-amd64/bin/jstack -l 28807'
Warning: Permanently added '10.30.170.179' (ECDSA) to the list of known hosts.
+ echo 'killing karaf process...'
killing karaf process...
+ ssh 10.30.170.179 bash -c 'ps axf | grep karaf | grep -v grep | awk '\''{print "kill -9 " $1}'\'' | sh'
Warning: Permanently added '10.30.170.179' (ECDSA) to the list of known hosts.
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_3_IP
+ echo 'Let'\''s take the karaf thread dump again'
Let's take the karaf thread dump again
+ ssh 10.30.171.166 'sudo ps aux'
Warning: Permanently added '10.30.171.166' (ECDSA) to the list of known hosts.
++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-tell-all-calcium/ps_after.log
++ grep -v grep
++ tr -s ' '
++ cut -f2 '-d '
+ pid=32116
+ echo 'karaf main: org.apache.karaf.main.Main, pid:32116'
karaf main: org.apache.karaf.main.Main, pid:32116
+ ssh 10.30.171.166 '/usr/lib/jvm/java-17-openjdk-amd64/bin/jstack -l 32116'
Warning: Permanently added '10.30.171.166' (ECDSA) to the list of known hosts.
32116: Unable to open socket file /tmp/.java_pid32116: target process 32116 doesn't respond within 10500ms or HotSpot VM not loaded
+ true
+ echo 'killing karaf process...'
killing karaf process...
+ ssh 10.30.171.166 bash -c 'ps axf | grep karaf | grep -v grep | awk '\''{print "kill -9 " $1}'\'' | sh'
Warning: Permanently added '10.30.171.166' (ECDSA) to the list of known hosts.
+ sleep 5
++ seq 1 3
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_1_IP
+ echo 'Compressing karaf.log 1'
Compressing karaf.log 1
+ ssh 10.30.171.225 gzip --best /tmp/karaf-0.20.4-SNAPSHOT/data/log/karaf.log
Warning: Permanently added '10.30.171.225' (ECDSA) to the list of known hosts.
+ echo 'Fetching compressed karaf.log 1'
Fetching compressed karaf.log 1
+ scp 10.30.171.225:/tmp/karaf-0.20.4-SNAPSHOT/data/log/karaf.log.gz odl1_karaf.log.gz
Warning: Permanently added '10.30.171.225' (ECDSA) to the list of known hosts.
+ ssh 10.30.171.225 rm -f /tmp/karaf-0.20.4-SNAPSHOT/data/log/karaf.log.gz
Warning: Permanently added '10.30.171.225' (ECDSA) to the list of known hosts.
+ scp 10.30.171.225:/tmp/karaf-0.20.4-SNAPSHOT/data/log/karaf_console.log odl1_karaf_console.log
Warning: Permanently added '10.30.171.225' (ECDSA) to the list of known hosts.
+ ssh 10.30.171.225 rm -f /tmp/karaf-0.20.4-SNAPSHOT/data/log/karaf_console.log
Warning: Permanently added '10.30.171.225' (ECDSA) to the list of known hosts.
+ echo 'Fetch GC logs'
Fetch GC logs
+ mkdir -p gclogs-1
+ scp '10.30.171.225:/tmp/karaf-0.20.4-SNAPSHOT/data/log/*.log' gclogs-1/
Warning: Permanently added '10.30.171.225' (ECDSA) to the list of known hosts.
scp: /tmp/karaf-0.20.4-SNAPSHOT/data/log/*.log: No such file or directory
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_2_IP
+ echo 'Compressing karaf.log 2'
Compressing karaf.log 2
+ ssh 10.30.170.179 gzip --best /tmp/karaf-0.20.4-SNAPSHOT/data/log/karaf.log
Warning: Permanently added '10.30.170.179' (ECDSA) to the list of known hosts.
+ echo 'Fetching compressed karaf.log 2'
Fetching compressed karaf.log 2
+ scp 10.30.170.179:/tmp/karaf-0.20.4-SNAPSHOT/data/log/karaf.log.gz odl2_karaf.log.gz
Warning: Permanently added '10.30.170.179' (ECDSA) to the list of known hosts.
+ ssh 10.30.170.179 rm -f /tmp/karaf-0.20.4-SNAPSHOT/data/log/karaf.log.gz
Warning: Permanently added '10.30.170.179' (ECDSA) to the list of known hosts.
+ scp 10.30.170.179:/tmp/karaf-0.20.4-SNAPSHOT/data/log/karaf_console.log odl2_karaf_console.log
Warning: Permanently added '10.30.170.179' (ECDSA) to the list of known hosts.
+ ssh 10.30.170.179 rm -f /tmp/karaf-0.20.4-SNAPSHOT/data/log/karaf_console.log
Warning: Permanently added '10.30.170.179' (ECDSA) to the list of known hosts.
+ echo 'Fetch GC logs'
Fetch GC logs
+ mkdir -p gclogs-2
+ scp '10.30.170.179:/tmp/karaf-0.20.4-SNAPSHOT/data/log/*.log' gclogs-2/
Warning: Permanently added '10.30.170.179' (ECDSA) to the list of known hosts.
scp: /tmp/karaf-0.20.4-SNAPSHOT/data/log/*.log: No such file or directory
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_3_IP
+ echo 'Compressing karaf.log 3'
Compressing karaf.log 3
+ ssh 10.30.171.166 gzip --best /tmp/karaf-0.20.4-SNAPSHOT/data/log/karaf.log
Warning: Permanently added '10.30.171.166' (ECDSA) to the list of known hosts.
+ echo 'Fetching compressed karaf.log 3'
Fetching compressed karaf.log 3
+ scp 10.30.171.166:/tmp/karaf-0.20.4-SNAPSHOT/data/log/karaf.log.gz odl3_karaf.log.gz
Warning: Permanently added '10.30.171.166' (ECDSA) to the list of known hosts.
+ ssh 10.30.171.166 rm -f /tmp/karaf-0.20.4-SNAPSHOT/data/log/karaf.log.gz
Warning: Permanently added '10.30.171.166' (ECDSA) to the list of known hosts.
+ scp 10.30.171.166:/tmp/karaf-0.20.4-SNAPSHOT/data/log/karaf_console.log odl3_karaf_console.log
Warning: Permanently added '10.30.171.166' (ECDSA) to the list of known hosts.
+ ssh 10.30.171.166 rm -f /tmp/karaf-0.20.4-SNAPSHOT/data/log/karaf_console.log
Warning: Permanently added '10.30.171.166' (ECDSA) to the list of known hosts.
+ echo 'Fetch GC logs'
Fetch GC logs
+ mkdir -p gclogs-3
+ scp '10.30.171.166:/tmp/karaf-0.20.4-SNAPSHOT/data/log/*.log' gclogs-3/
Warning: Permanently added '10.30.171.166' (ECDSA) to the list of known hosts.
scp: /tmp/karaf-0.20.4-SNAPSHOT/data/log/*.log: No such file or directory
+ echo 'Examine copied files'
Examine copied files
+ ls -lt
total 77400
drwxrwxr-x. 2 jenkins jenkins        6 Feb 23 04:01 gclogs-3
-rw-rw-r--. 1 jenkins jenkins     1720 Feb 23 04:01 odl3_karaf_console.log
-rw-rw-r--. 1 jenkins jenkins  4547443 Feb 23 04:01 odl3_karaf.log.gz
drwxrwxr-x. 2 jenkins jenkins        6 Feb 23 04:01 gclogs-2
-rw-rw-r--. 1 jenkins jenkins    10125 Feb 23 04:01 odl2_karaf_console.log
-rw-rw-r--. 1 jenkins jenkins   155806 Feb 23 04:01 odl2_karaf.log.gz
drwxrwxr-x. 2 jenkins jenkins        6 Feb 23 04:01 gclogs-1
-rw-rw-r--. 1 jenkins jenkins     1720 Feb 23 04:01 odl1_karaf_console.log
-rw-rw-r--. 1 jenkins jenkins   262568 Feb 23 04:01 odl1_karaf.log.gz
-rw-rw-r--. 1 jenkins jenkins        0 Feb 23 04:01 karaf_3_32116_threads_after.log
-rw-rw-r--. 1 jenkins jenkins    16069 Feb 23 04:01 ps_after.log
-rw-rw-r--. 1 jenkins jenkins   138583 Feb 23 04:01 karaf_2_28807_threads_after.log
-rw-rw-r--. 1 jenkins jenkins   132879 Feb 23 04:01 karaf_1_35048_threads_after.log
-rw-rw-r--. 1 jenkins jenkins   317835 Feb 23 04:01 report.html
-rw-rw-r--. 1 jenkins jenkins  3103844 Feb 23 04:01 log.html
-rw-rw-r--. 1 jenkins jenkins 70077533 Feb 23 04:01 output.xml
-rw-rw-r--. 1 jenkins jenkins     6030 Feb 22 23:48 testplan.txt
-rw-rw-r--. 1 jenkins jenkins   128199 Feb 22 23:48 karaf_3_28747_threads_before.log
-rw-rw-r--. 1 jenkins jenkins    15881 Feb 22 23:48 ps_before.log
-rw-rw-r--. 1 jenkins jenkins   126461 Feb 22 23:48 karaf_2_28807_threads_before.log
-rw-rw-r--. 1 jenkins jenkins   126475 Feb 22 23:48 karaf_1_28779_threads_before.log
-rw-rw-r--. 1 jenkins jenkins     3106 Feb 22 23:45 post-startup-script.sh
-rw-rw-r--. 1 jenkins jenkins      252 Feb 22 23:45 startup-script.sh
-rw-rw-r--. 1 jenkins jenkins     3461 Feb 22 23:45 configuration-script.sh
-rw-rw-r--. 1 jenkins jenkins      695 Feb 22 23:45 custom_shard_config.txt
-rw-rw-r--. 1 jenkins jenkins      135 Feb 22 23:45 scriptplan.txt
-rw-rw-r--. 1 jenkins jenkins      337 Feb 22 23:44 detect_variables.env
-rw-rw-r--. 1 jenkins jenkins     2619 Feb 22 23:44 pom.xml
-rw-rw-r--. 1 jenkins jenkins       92 Feb 22 23:44 set_variables.env
-rw-rw-r--. 1 jenkins jenkins      314 Feb 22 23:44 slave_addresses.txt
-rw-rw-r--. 1 jenkins jenkins      570 Feb 22 23:44 requirements.txt
-rw-rw-r--. 1 jenkins jenkins       26 Feb 22 23:44 env.properties
-rw-rw-r--. 1 jenkins jenkins      333 Feb 22 23:42 stack-parameters.yaml
drwxrwxr-x. 7 jenkins jenkins     4096 Feb 22 23:41 test
drwxrwxr-x. 2 jenkins jenkins        6 Feb 22 23:41 test@tmp
-rw-rw-r--. 1 jenkins jenkins     1410 Feb 22 20:49 maven-metadata.xml
+ true
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/sh /tmp/jenkins7705394155992369588.sh
Cleaning up Robot installation...
$ ssh-agent -k
unset SSH_AUTH_SOCK;
unset SSH_AGENT_PID;
echo Agent pid 5278 killed;
[ssh-agent] Stopped.
Recording plot data
Robot results publisher started...
INFO: Checking test criticality is deprecated and will be dropped in a future release!
-Parsing output xml:
Done!
-Copying log files to build dir:
Done!
-Assigning results to build:
Done!
-Checking thresholds:
Done!
Done publishing Robot results.
Build step 'Publish Robot Framework test results' changed build result to UNSTABLE
[PostBuildScript] - [INFO] Executing post build scripts.
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins7711679531737207837.sh
Archiving csit artifacts
mv: cannot stat '*_1.png': No such file or directory
mv: cannot stat '/tmp/odl1_*': No such file or directory
mv: cannot stat '*_2.png': No such file or directory
mv: cannot stat '/tmp/odl2_*': No such file or directory
mv: cannot stat '*_3.png': No such file or directory
mv: cannot stat '/tmp/odl3_*': No such file or directory
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  392k    0  392k    0     0  1446k      0 --:--:-- --:--:-- --:--:-- 1441k
100 5219k    0 5219k    0     0  4135k      0 --:--:--  0:00:01 --:--:-- 4135k
Archive:  robot-plugin.zip
  inflating: ./archives/robot-plugin/log.html  
  inflating: ./archives/robot-plugin/output.xml  
  inflating: ./archives/robot-plugin/report.html  
mv: cannot stat '*.log.gz': No such file or directory
mv: cannot stat '*.csv': No such file or directory
mv: cannot stat '*.png': No such file or directory
[PostBuildScript] - [INFO] Executing post build scripts.
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins16895534196827069407.sh
[PostBuildScript] - [INFO] Executing post build scripts.
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties content 
OS_CLOUD=vex
OS_STACK_NAME=releng-controller-csit-3node-clustering-tell-all-calcium-76

[EnvInject] - Variables injected successfully.
provisioning config files...
copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins13088362039142159271.sh
---> openstack-stack-delete.sh
Setup pyenv:
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-tell-all-calcium/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-QYyr from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes python-heatclient python-openstackclient 
lf-activate-venv(): INFO: Adding /tmp/venv-QYyr/bin to PATH
INFO: Retrieving stack cost for: releng-controller-csit-3node-clustering-tell-all-calcium-76
DEBUG: Successfully retrieved stack cost: total: 1.6500000000000001
INFO: Deleting stack releng-controller-csit-3node-clustering-tell-all-calcium-76
Successfully deleted stack releng-controller-csit-3node-clustering-tell-all-calcium-76
[PostBuildScript] - [INFO] Executing post build scripts.
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins9674543229913747436.sh
---> sysstat.sh
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins11993496940868390287.sh
---> package-listing.sh
++ tr '[:upper:]' '[:lower:]'
++ facter osfamily
+ OS_FAMILY=redhat
+ workspace=/w/workspace/controller-csit-3node-clustering-tell-all-calcium
+ START_PACKAGES=/tmp/packages_start.txt
+ END_PACKAGES=/tmp/packages_end.txt
+ DIFF_PACKAGES=/tmp/packages_diff.txt
+ PACKAGES=/tmp/packages_start.txt
+ '[' /w/workspace/controller-csit-3node-clustering-tell-all-calcium ']'
+ PACKAGES=/tmp/packages_end.txt
+ case "${OS_FAMILY}" in
+ rpm -qa
+ sort
+ '[' -f /tmp/packages_start.txt ']'
+ '[' -f /tmp/packages_end.txt ']'
+ diff /tmp/packages_start.txt /tmp/packages_end.txt
+ '[' /w/workspace/controller-csit-3node-clustering-tell-all-calcium ']'
+ mkdir -p /w/workspace/controller-csit-3node-clustering-tell-all-calcium/archives/
+ cp -f /tmp/packages_diff.txt /tmp/packages_end.txt /tmp/packages_start.txt /w/workspace/controller-csit-3node-clustering-tell-all-calcium/archives/
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins2407346513430145073.sh
---> capture-instance-metadata.sh
Setup pyenv:
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-tell-all-calcium/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-QYyr from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing: lftools 
lf-activate-venv(): INFO: Adding /tmp/venv-QYyr/bin to PATH
INFO: Running in OpenStack, capturing instance metadata
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins6426296487902319444.sh
provisioning config files...
Could not find credentials [logs] for controller-csit-3node-clustering-tell-all-calcium #76
copy managed file [jenkins-log-archives-settings] to file:/w/workspace/controller-csit-3node-clustering-tell-all-calcium@tmp/config3852505720665928045tmp
Regular expression run condition: Expression=[^.*logs-s3.*], Label=[odl-logs-s3-cloudfront-index]
Run condition [Regular expression match] enabling perform for step [Provide Configuration files]
provisioning config files...
copy managed file [jenkins-s3-log-ship] to file:/home/jenkins/.aws/credentials
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties content 
SERVER_ID=logs

[EnvInject] - Variables injected successfully.
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins7793254407459326079.sh
---> create-netrc.sh
WARN: Log server credential not found.
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins10493002965158923017.sh
---> python-tools-install.sh
Setup pyenv:
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-tell-all-calcium/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-QYyr from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing: lftools 
lf-activate-venv(): INFO: Adding /tmp/venv-QYyr/bin to PATH
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins3348587941591925606.sh
---> sudo-logs.sh
Archiving 'sudo' log..
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins17493252715693185702.sh
---> job-cost.sh
Setup pyenv:
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-tell-all-calcium/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-QYyr from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing: zipp==1.1.0 python-openstackclient urllib3~=1.26.15 
lf-activate-venv(): INFO: Adding /tmp/venv-QYyr/bin to PATH
DEBUG: total: 1.6500000000000001
INFO: Retrieving Stack Cost...
INFO: Retrieving Pricing Info for: v3-standard-2
INFO: Archiving Costs
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash -l /tmp/jenkins3426146756361447925.sh
---> logs-deploy.sh
Setup pyenv:
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-tell-all-calcium/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-QYyr from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing: lftools 
lf-activate-venv(): INFO: Adding /tmp/venv-QYyr/bin to PATH
WARNING: Nexus logging server not set
INFO: S3 path logs/releng/vex-yul-odl-jenkins-1/controller-csit-3node-clustering-tell-all-calcium/76/
INFO: archiving logs to S3
---> uname -a:
Linux prd-centos8-robot-2c-8g-1436.novalocal 4.18.0-553.5.1.el8.x86_64 #1 SMP Tue May 21 05:46:01 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux


---> lscpu:
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              2
On-line CPU(s) list: 0,1
Thread(s) per core:  1
Core(s) per socket:  1
Socket(s):           2
NUMA node(s):        1
Vendor ID:           AuthenticAMD
CPU family:          23
Model:               49
Model name:          AMD EPYC-Rome Processor
Stepping:            0
CPU MHz:             2799.998
BogoMIPS:            5599.99
Virtualization:      AMD-V
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           32K
L1i cache:           32K
L2 cache:            512K
L3 cache:            16384K
NUMA node0 CPU(s):   0,1
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr wbnoinvd arat npt nrip_save umip rdpid arch_capabilities


---> nproc:
2


---> df -h:
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        3.8G     0  3.8G   0% /dev
tmpfs           3.8G     0  3.8G   0% /dev/shm
tmpfs           3.8G   17M  3.8G   1% /run
tmpfs           3.8G     0  3.8G   0% /sys/fs/cgroup
/dev/vda1        40G  8.4G   32G  21% /
tmpfs           770M     0  770M   0% /run/user/1001


---> free -m:
              total        used        free      shared  buff/cache   available
Mem:           7697         665        4717          19        2314        6734
Swap:          1023           0        1023


---> ip addr:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1458 qdisc mq state UP group default qlen 1000
    link/ether fa:16:3e:f5:71:62 brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    altname ens3
    inet 10.30.170.150/23 brd 10.30.171.255 scope global dynamic noprefixroute eth0
       valid_lft 70681sec preferred_lft 70681sec
    inet6 fe80::f816:3eff:fef5:7162/64 scope link 
       valid_lft forever preferred_lft forever


---> sar -b -r -n DEV:
Linux 4.18.0-553.5.1.el8.x86_64 (prd-centos8-robot-2c-8g-1436.novalocal) 	02/23/2025 	_x86_64_	(2 CPU)

12:00:01 AM       tps      rtps      wtps   bread/s   bwrtn/s
12:01:01 AM      0.92      0.02      0.90      0.13    207.88
12:02:01 AM      0.68      0.07      0.62      1.73     45.16
12:03:01 AM      0.18      0.02      0.17      0.13     14.64
12:04:01 AM      0.15      0.00      0.15      0.00      8.66
12:05:01 AM      0.17      0.00      0.17      0.00     15.51
12:06:01 AM      0.18      0.00      0.18      0.00     15.08
12:07:01 AM      0.35      0.00      0.35      0.00     20.76
12:08:01 AM      0.50      0.07      0.43      7.60     18.53
12:09:01 AM      0.15      0.00      0.15      0.00     26.39
12:10:01 AM      0.15      0.00      0.15      0.00      3.23
12:11:01 AM      0.18      0.00      0.18      0.00      5.82
12:12:01 AM      0.42      0.00      0.42      0.00      7.96
12:13:01 AM      0.25      0.00      0.25      0.00      7.90
12:14:01 AM      0.20      0.00      0.20      0.00      4.17
12:15:01 AM      0.18      0.00      0.18      0.00     13.26
12:16:01 AM      0.35      0.00      0.35      0.00      3.82
12:17:01 AM      0.57      0.00      0.57      0.00     11.15
12:18:01 AM      0.32      0.00      0.32      0.00     15.05
12:19:01 AM      0.20      0.00      0.20      0.00     11.88
12:20:01 AM      0.15      0.00      0.15      0.00     14.56
12:21:01 AM      0.43      0.00      0.43      0.00     18.99
12:22:01 AM      0.80      0.00      0.80      0.00     21.85
12:23:01 AM      0.52      0.00      0.52      0.00     18.64
12:24:01 AM      0.33      0.00      0.33      0.00      3.75
12:25:02 AM      0.75      0.32      0.43      6.93      7.98
12:26:01 AM      0.85      0.00      0.85      0.00     29.87
12:27:01 AM      0.37      0.00      0.37      0.00     20.54
12:28:01 AM      0.25      0.00      0.25      0.00     17.88
12:29:01 AM      0.20      0.00      0.20      0.00     12.15
12:30:01 AM      0.13      0.00      0.13      0.00     14.53
12:31:01 AM      0.55      0.00      0.55      0.00     18.16
12:32:01 AM      0.38      0.00      0.38      0.00     22.73
12:33:01 AM      0.25      0.00      0.25      0.00      7.63
12:34:01 AM      0.22      0.00      0.22      0.00      3.63
12:35:01 AM      0.13      0.00      0.13      0.00      5.87
12:36:01 AM      5.93      5.75      0.18    555.55      6.20
12:37:01 AM      3.12      0.00      3.12      0.00     76.82
12:38:01 AM      0.13      0.00      0.13      0.00     14.93
12:39:01 AM      0.15      0.00      0.15      0.00     14.71
12:40:01 AM      0.17      0.00      0.17      0.00     14.81
12:41:01 AM      0.18      0.00      0.18      0.00     15.41
12:42:01 AM      0.87      0.00      0.87      0.00     23.95
12:43:01 AM      0.47      0.00      0.47      0.00     16.63
12:44:01 AM      0.18      0.00      0.18      0.00     14.13
12:45:01 AM      0.22      0.00      0.22      0.00     16.05
12:46:01 AM      0.42      0.00      0.42      0.00     21.36
12:47:01 AM      0.28      0.00      0.28      0.00     17.65
12:48:01 AM      0.25      0.00      0.25      0.00     11.21
12:49:01 AM      0.13      0.00      0.13      0.00     14.93
12:50:01 AM      0.25      0.00      0.25      0.00     23.89
12:51:01 AM      0.28      0.00      0.28      0.00     18.94
12:52:01 AM      0.38      0.00      0.38      0.00     15.94
12:53:01 AM      0.65      0.00      0.65      0.00     19.21
12:54:01 AM      0.50      0.00      0.50      0.00     17.18
12:55:01 AM      0.53      0.00      0.53      0.00     16.74
12:56:01 AM      0.40      0.00      0.40      0.00     26.11
12:57:01 AM      0.57      0.00      0.57      0.00     11.28
12:58:01 AM      0.50      0.00      0.50      0.00      9.48
12:59:01 AM      0.38      0.00      0.38      0.00      6.91
01:00:01 AM      0.43      0.03      0.40      0.27      7.11
01:01:01 AM      0.55      0.00      0.55      0.00     10.18
01:02:01 AM      0.58      0.00      0.58      0.00     20.81
01:03:01 AM      0.25      0.00      0.25      0.00     10.03
01:04:01 AM      0.15      0.00      0.15      0.00     15.60
01:05:01 AM      0.17      0.00      0.17      0.00     15.86
01:06:01 AM      0.25      0.00      0.25      0.00     10.80
01:07:01 AM      0.45      0.00      0.45      0.00     22.76
01:08:01 AM      0.20      0.00      0.20      0.00     17.09
01:09:01 AM      0.18      0.00      0.18      0.00     24.15
01:10:01 AM      0.12      0.00      0.12      0.00      8.00
01:11:01 AM      0.17      0.00      0.17      0.00     14.71
01:12:01 AM      0.42      0.00      0.42      0.00     20.14
01:13:01 AM      0.17      0.00      0.17      0.00      8.68
01:14:01 AM      0.28      0.00      0.28      0.00     20.66
01:15:01 AM      0.23      0.00      0.23      0.00     20.83
01:16:01 AM      0.13      0.00      0.13      0.00      5.20
01:17:01 AM      0.32      0.00      0.32      0.00      8.08
01:18:01 AM      0.20      0.00      0.20      0.00      6.53
01:19:01 AM      0.12      0.00      0.12      0.00      2.93
01:20:01 AM      0.10      0.00      0.10      0.00      2.80
01:21:01 AM      0.30      0.00      0.30      0.00     52.15
01:22:01 AM      0.52      0.00      0.52      0.00    124.93
01:23:01 AM      0.33      0.00      0.33      0.00     17.59
01:24:02 AM      0.28      0.00      0.28      0.00     13.88
01:25:01 AM      0.12      0.00      0.12      0.00      7.32
01:26:01 AM      0.13      0.00      0.13      0.00     15.06
01:27:01 AM      0.57      0.00      0.57      0.00     23.96
01:28:01 AM      0.53      0.00      0.53      0.00     17.46
01:29:01 AM      0.37      0.00      0.37      0.00      7.75
01:30:01 AM      0.33      0.00      0.33      0.00      3.58
01:31:01 AM      0.50      0.03      0.47      1.33      9.43
01:32:01 AM      0.52      0.00      0.52      0.00      6.97
01:33:01 AM      0.30      0.00      0.30      0.00      6.05
01:34:01 AM      0.38      0.00      0.38      0.00      4.28
01:35:01 AM      0.32      0.00      0.32      0.00      6.68
01:36:01 AM      0.33      0.00      0.33      0.00      3.68
01:37:01 AM      1.03      0.00      1.03      0.00     17.83
01:38:01 AM      0.53      0.00      0.53      0.00      8.68
01:39:01 AM      0.33      0.00      0.33      0.00      3.37
01:40:01 AM      0.40      0.00      0.40      0.00      7.42
01:41:01 AM      0.45      0.00      0.45      0.00      6.61
01:42:01 AM      0.37      0.00      0.37      0.00     16.25
01:43:01 AM      0.23      0.00      0.23      0.00     17.11
01:44:01 AM      0.13      0.00      0.13      0.00      7.20
01:45:01 AM      0.22      0.00      0.22      0.00     19.13
01:46:01 AM      0.37      0.00      0.37      0.00     15.73
01:47:01 AM      0.37      0.00      0.37      0.00     19.13
01:48:01 AM      0.22      0.00      0.22      0.00     12.51
01:49:01 AM      0.12      0.00      0.12      0.00      1.32
01:50:01 AM      0.10      0.00      0.10      0.00      0.67
01:51:01 AM      0.30      0.00      0.30      0.00      7.30
01:52:01 AM      0.43      0.00      0.43      0.00      6.37
01:53:01 AM      0.15      0.00      0.15      0.00      4.52
01:54:01 AM      0.10      0.00      0.10      0.00      0.78
01:55:01 AM      0.20      0.00      0.20      0.00      5.73
01:56:01 AM      0.08      0.00      0.08      0.00      1.02
01:57:01 AM      0.25      0.00      0.25      0.00      6.35
01:58:01 AM      0.40      0.00      0.40      0.00      7.31
01:59:01 AM      0.22      0.00      0.22      0.00      5.90
02:00:01 AM      0.12      0.00      0.12      0.00      1.30
02:01:01 AM      0.20      0.00      0.20      0.00     11.46
02:02:01 AM      0.38      0.00      0.38      0.00     18.23
02:03:01 AM      0.35      0.00      0.35      0.00     18.71
02:04:01 AM      0.15      0.00      0.15      0.00     14.53
02:05:01 AM      0.13      0.00      0.13      0.00     15.19
02:06:01 AM      0.15      0.00      0.15      0.00      9.72
02:07:01 AM      0.25      0.00      0.25      0.00     15.24
02:08:01 AM      0.25      0.00      0.25      0.00      3.92
02:09:01 AM      0.15      0.00      0.15      0.00      4.13
02:10:01 AM      0.12      0.00      0.12      0.00      1.18
02:11:01 AM      0.28      0.00      0.28      0.00      4.32
02:12:01 AM      0.28      0.00      0.28      0.00      6.96
02:13:01 AM      0.28      0.00      0.28      0.00      3.98
02:14:01 AM      0.13      0.00      0.13      0.00      4.03
02:15:01 AM      0.15      0.00      0.15      0.00      1.45
02:16:01 AM      0.12      0.00      0.12      0.00      4.13
02:17:01 AM      0.30      0.00      0.30      0.00      5.40
02:18:01 AM      0.13      0.00      0.13      0.00      4.13
02:19:01 AM      0.28      0.00      0.28      0.00      3.48
02:20:01 AM      0.28      0.00      0.28      0.00     10.83
02:21:01 AM      0.57      0.00      0.57      0.00     16.99
02:22:01 AM      0.70      0.00      0.70      0.00     21.00
02:23:01 AM      0.53      0.00      0.53      0.00     17.44
02:24:01 AM      0.57      0.00      0.57      0.00     17.38
02:25:01 AM      0.50      0.00      0.50      0.00     16.28
02:26:01 AM      0.63      0.00      0.63      0.00     20.41
02:27:02 AM      0.32      0.00      0.32      0.00     19.66
02:28:01 AM      0.14      0.00      0.14      0.00      5.83
02:29:01 AM      0.13      0.00      0.13      0.00      3.07
02:30:01 AM      0.13      0.00      0.13      0.00      5.33
02:31:01 AM      0.15      0.00      0.15      0.00      3.37
02:32:01 AM      0.35      0.00      0.35      0.00      8.88
02:33:01 AM      0.23      0.00      0.23      0.00     10.66
02:34:01 AM      0.18      0.00      0.18      0.00      3.08
02:35:01 AM      0.12      0.00      0.12      0.00      3.75
02:36:01 AM      0.23      0.00      0.23      0.00     16.86
02:37:01 AM      0.35      0.00      0.35      0.00     19.30
02:38:01 AM      0.13      0.00      0.13      0.00      8.00
02:39:01 AM      0.13      0.00      0.13      0.00     14.53
02:40:01 AM      0.20      0.00      0.20      0.00     12.78
02:41:01 AM      0.25      0.00      0.25      0.00     16.56
02:42:01 AM      0.40      0.00      0.40      0.00     13.73
02:43:01 AM      0.15      0.00      0.15      0.00     13.86
02:44:01 AM      0.25      0.00      0.25      0.00     18.53
02:45:01 AM      0.20      0.00      0.20      0.00     11.71
02:46:01 AM      0.85      0.00      0.85      0.00     22.83
02:47:01 AM      0.65      0.00      0.65      0.00     27.16
02:48:01 AM      0.53      0.00      0.53      0.00     18.18
02:49:01 AM      0.33      0.00      0.33      0.00      9.18
02:50:01 AM      0.17      0.00      0.17      0.00     10.38
02:51:01 AM      0.18      0.00      0.18      0.00     16.48
02:52:01 AM      0.42      0.00      0.42      0.00     20.63
02:53:01 AM      0.23      0.00      0.23      0.00     10.00
02:54:01 AM      0.20      0.00      0.20      0.00     15.34
02:55:01 AM      0.13      0.00      0.13      0.00     14.53
02:56:01 AM      0.12      0.00      0.12      0.00     19.73
02:57:01 AM      0.30      0.00      0.30      0.00      8.91
02:58:01 AM      0.18      0.00      0.18      0.00      5.75
02:59:01 AM      0.13      0.00      0.13      0.00      3.20
03:00:01 AM      0.15      0.00      0.15      0.00      3.15
03:01:01 AM      0.30      0.00      0.30      0.00      7.38
03:02:01 AM      0.42      0.00      0.42      0.00     14.63
03:03:01 AM      0.37      0.00      0.37      0.00      5.35
03:04:01 AM      0.37      0.00      0.37      0.00      8.81
03:05:01 AM      0.37      0.00      0.37      0.00      5.62
03:06:01 AM      0.37      0.00      0.37      0.00      5.62
03:07:01 AM      0.58      0.00      0.58      0.00     15.66
03:08:01 AM      0.17      0.00      0.17      0.00      3.20
03:09:01 AM      0.15      0.00      0.15      0.00     15.28
03:10:01 AM      0.22      0.00      0.22      0.00     14.66
03:11:01 AM      0.27      0.00      0.27      0.00     17.56
03:12:01 AM      0.27      0.00      0.27      0.00     17.63
03:13:01 AM      0.28      0.00      0.28      0.00     11.35
03:14:01 AM      0.43      0.00      0.43      0.00     23.46
03:15:01 AM      0.27      0.00      0.27      0.00     16.14
03:16:01 AM      0.45      0.00      0.45      0.00      8.17
03:17:01 AM      0.52      0.00      0.52      0.00      8.73
03:18:01 AM      0.45      0.00      0.45      0.00      8.03
03:19:01 AM      0.47      0.00      0.47      0.00      7.63
03:20:01 AM      0.48      0.00      0.48      0.00      8.16
03:21:01 AM      0.37      0.00      0.37      0.00     11.61
03:22:01 AM      0.18      0.00      0.18      0.00      3.77
03:23:01 AM      0.42      0.00      0.42      0.00     10.70
03:24:01 AM      0.37      0.00      0.37      0.00      7.48
03:25:01 AM      0.12      0.00      0.12      0.00      0.83
03:26:01 AM      0.17      0.00      0.17      0.00      9.23
03:27:01 AM      0.22      0.00      0.22      0.00     10.20
03:28:02 AM      0.25      0.00      0.25      0.00     10.31
03:29:01 AM      0.15      0.00      0.15      0.00     16.95
03:30:01 AM      0.17      0.00      0.17      0.00     14.83
03:31:01 AM      0.45      0.00      0.45      0.00     17.63
03:32:01 AM      0.40      0.00      0.40      0.00     17.19
03:33:01 AM      0.23      0.00      0.23      0.00     25.53
03:34:01 AM      0.13      0.00      0.13      0.00      3.07
03:35:01 AM      0.15      0.00      0.15      0.00      5.33
03:36:01 AM      0.15      0.00      0.15      0.00      3.40
03:37:01 AM      0.38      0.00      0.38      0.00     10.76
03:38:01 AM      0.22      0.00      0.22      0.00      4.67
03:39:01 AM      0.17      0.00      0.17      0.00      8.55
03:40:01 AM      0.35      0.00      0.35      0.00      7.98
03:41:01 AM      0.38      0.00      0.38      0.00      7.51
03:42:01 AM      0.53      0.00      0.53      0.00      7.60
03:43:01 AM      0.25      0.00      0.25      0.00      4.65
03:44:01 AM      0.47      0.00      0.47      0.00     19.08
03:45:01 AM      0.62      0.00      0.62      0.00     19.56
03:46:01 AM      0.52      0.00      0.52      0.00     16.21
03:47:01 AM      0.67      0.00      0.67      0.00     20.24
03:48:01 AM      0.57      0.00      0.57      0.00     17.72
03:49:01 AM      0.45      0.00      0.45      0.00     15.48
03:50:01 AM      0.43      0.00      0.43      0.00     26.01
03:51:01 AM      0.47      0.00      0.47      0.00      8.56
03:52:01 AM      0.72      0.00      0.72      0.00     12.60
03:53:01 AM      0.40      0.00      0.40      0.00      7.05
03:54:01 AM      0.40      0.00      0.40      0.00      7.31
03:55:01 AM      0.40      0.00      0.40      0.00      6.93
03:56:01 AM      0.48      0.00      0.48      0.00      9.36
03:57:01 AM      0.45      0.00      0.45      0.00     17.16
03:58:01 AM      0.33      0.00      0.33      0.00      9.26
03:59:01 AM      0.18      0.00      0.18      0.00      8.82
04:00:01 AM      0.12      0.00      0.12      0.00      7.46
04:01:01 AM      0.27      0.00      0.27      0.00     15.49
04:02:01 AM      6.48      0.28      6.20     33.85    357.95
04:03:01 AM     30.14      0.43     29.71     49.18   3889.74
Average:         0.50      0.03      0.47      2.70     31.09

12:00:01 AM kbmemfree   kbavail kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit  kbactive   kbinact   kbdirty
12:01:01 AM   4902616   6974784   2979808     37.80      2688   2241616    752408      8.42    265760   2339868        56
12:02:01 AM   4901428   6974660   2980996     37.82      2688   2242680    771804      8.64    265892   2340804        12
12:03:01 AM   4901144   6974792   2981280     37.82      2688   2243104    772328      8.65    265892   2341228        60
12:04:01 AM   4899184   6973192   2983240     37.85      2688   2243464    772328      8.65    265892   2343176       204
12:05:01 AM   4898460   6972832   2983964     37.86      2688   2243828    772328      8.65    265892   2343776       160
12:06:01 AM   4898444   6973176   2983980     37.86      2688   2244188    772328      8.65    265892   2344080       112
12:07:01 AM   4897508   6972608   2984916     37.87      2688   2244564    763284      8.55    265892   2344744        24
12:08:01 AM   4897528   6973260   2984896     37.87      2688   2245196    770152      8.62    265924   2344940       116
12:09:01 AM   4894724   6971120   2987700     37.90      2688   2245852    770152      8.62    265924   2347536         4
12:10:01 AM   4894524   6971008   2987900     37.91      2688   2245948    772616      8.65    265924   2347628        40
12:11:01 AM   4894680   6971268   2987744     37.90      2688   2246052    772616      8.65    265924   2347616         4
12:12:01 AM   4894492   6971176   2987932     37.91      2688   2246148    772616      8.65    265924   2347788        40
12:13:01 AM   4894392   6971196   2988032     37.91      2688   2246260    772616      8.65    265924   2347860         4
12:14:01 AM   4894504   6971404   2987920     37.91      2688   2246356    772616      8.65    265924   2347956        40
12:15:01 AM   4893848   6971064   2988576     37.91      2688   2246672    751920      8.42    265924   2348276         8
12:16:01 AM   4894036   6971264   2988388     37.91      2688   2246684    751920      8.42    265924   2348288         4
12:17:01 AM   4893636   6971004   2988788     37.92      2688   2246824    773848      8.66    265924   2348548        40
12:18:01 AM   4893388   6971112   2989036     37.92      2688   2247188    773848      8.66    265924   2348896        52
12:19:01 AM   4892968   6971056   2989456     37.93      2688   2247552    773848      8.66    265924   2349256       128
12:20:01 AM   4892444   6970896   2989980     37.93      2688   2247916    772888      8.65    265924   2349744        88
12:21:01 AM   4892132   6970960   2990292     37.94      2688   2248292    772888      8.65    265924   2350052        20
12:22:01 AM   4891480   6970680   2990944     37.94      2688   2248656    772888      8.65    265924   2350692         4
12:23:01 AM   4891240   6970856   2991184     37.95      2688   2249072    772888      8.65    265924   2350960        80
12:24:01 AM   4890912   6970540   2991512     37.95      2688   2249084    772888      8.65    265924   2351468        16
12:25:02 AM   4890488   6970496   2991936     37.96      2688   2249464    769572      8.62    266036   2351548       120
12:26:01 AM   4890216   6970588   2992208     37.96      2688   2249836    769572      8.62    266036   2351860       184
12:27:01 AM   4888564   6969300   2993860     37.98      2688   2250200    769572      8.62    266036   2353620       120
12:28:01 AM   4887860   6968964   2994564     37.99      2688   2250560    769572      8.62    266036   2354408         4
12:29:01 AM   4887668   6969140   2994756     37.99      2688   2250936    769572      8.62    266036   2354580        84
12:30:01 AM   4886900   6968740   2995524     38.00      2688   2251296    766784      8.59    266036   2355064        36
12:31:01 AM   4886716   6968920   2995708     38.00      2688   2251664    766784      8.59    266036   2355308        24
12:32:01 AM   4886064   6968804   2996360     38.01      2688   2252200    766784      8.59    266036   2355912        48
12:33:01 AM   4885864   6968704   2996560     38.02      2688   2252300    766784      8.59    266036   2356360        72
12:34:01 AM   4886064   6969000   2996360     38.01      2688   2252396    766784      8.59    266036   2356184        56
12:35:01 AM   4885632   6968680   2996792     38.02      2688   2252500    766784      8.59    266036   2356464        32
12:36:01 AM   4868868   6968744   3013556     38.23      2688   2269256    775308      8.68    268216   2370792       720
12:37:01 AM   4868788   6968756   3013636     38.23      2688   2269348    775308      8.68    268216   2371068        24
12:38:01 AM   4868160   6968708   3014264     38.24      2688   2269936    775308      8.68    268216   2371580       192
12:39:01 AM   4867808   6968720   3014616     38.24      2688   2270300    775308      8.68    268216   2371828       144
12:40:01 AM   4867404   6968676   3015020     38.25      2688   2270660    791408      8.86    268216   2372304        96
12:41:01 AM   4866756   6968400   3015668     38.26      2688   2271028    791408      8.86    268216   2372896        52
12:42:01 AM   4866212   6968224   3016212     38.27      2688   2271392    791408      8.86    268216   2373364        24
12:43:01 AM   4866272   6968660   3016152     38.26      2688   2271768    791408      8.86    268216   2373424        28
12:44:01 AM   4865848   6968756   3016576     38.27      2688   2272280    791408      8.86    268216   2373944       164
12:45:01 AM   4865344   6968620   3017080     38.28      2688   2272656    791408      8.86    268216   2374316       132
12:46:01 AM   4865008   6968648   3017416     38.28      2688   2273020    791408      8.86    268244   2374656        60
12:47:01 AM   4864652   6968664   3017772     38.28      2688   2273384    791408      8.86    268244   2375108        12
12:48:01 AM   4864000   6968364   3018424     38.29      2688   2273744    791408      8.86    268244   2375744       172
12:49:01 AM   4863836   6968560   3018588     38.30      2688   2274104    791408      8.86    268244   2375764       116
12:50:01 AM   4862532   6967864   3019892     38.31      2688   2274712    775412      8.68    268244   2376904        96
12:51:01 AM   4862932   6968644   3019492     38.31      2688   2275076    775412      8.68    268244   2376760         8
12:52:01 AM   4862164   6968240   3020260     38.32      2688   2275440    775412      8.68    268248   2377492        20
12:53:01 AM   4861912   6968340   3020512     38.32      2688   2275804    775412      8.68    268244   2377500        24
12:54:01 AM   4861760   6968556   3020664     38.32      2688   2276164    775412      8.68    268244   2377880         4
12:55:01 AM   4860936   6968108   3021488     38.33      2688   2276544    775412      8.68    268244   2378580        40
12:56:01 AM   4860608   6968452   3021816     38.34      2688   2277216    775412      8.68    268244   2379048        48
12:57:01 AM   4860492   6968436   3021932     38.34      2688   2277308    775412      8.68    268244   2379080         4
12:58:01 AM   4860528   6968572   3021896     38.34      2688   2277408    775412      8.68    268244   2379116         4
12:59:01 AM   4860004   6968144   3022420     38.34      2688   2277504    775412      8.68    268244   2379456        24
01:00:01 AM   4858712   6966940   3023712     38.36      2688   2277608    776892      8.70    268244   2380376        24
01:01:01 AM   4859188   6967528   3023236     38.35      2688   2277716    776892      8.70    268248   2380048         4
01:02:01 AM   4859660   6968448   3022764     38.35      2688   2278156    776892      8.70    268252   2380108        44
01:03:01 AM   4859224   6968384   3023200     38.35      2688   2278524    776892      8.70    268252   2380332       204
01:04:01 AM   4858968   6968504   3023456     38.36      2688   2278900    776892      8.70    268252   2380772       144
01:05:01 AM   4858464   6968352   3023960     38.36      2688   2279260    776892      8.70    268252   2381196        68
01:06:01 AM   4858876   6969132   3023548     38.36      2688   2279620    776892      8.70    268248   2381248       220
01:07:01 AM   4858340   6968968   3024084     38.36      2688   2280000    776892      8.70    268248   2381580       172
01:08:01 AM   4857288   6968492   3025136     38.38      2688   2280568    776892      8.70    268248   2382468       292
01:09:01 AM   4857396   6969004   3025028     38.38      2688   2280968    776892      8.70    268248   2382500         4
01:10:01 AM   4856460   6968424   3025964     38.39      2688   2281332    772556      8.65    268248   2383088       156
01:11:01 AM   4856660   6968992   3025764     38.39      2688   2281696    772556      8.65    268248   2383220       116
01:12:01 AM   4855656   6968352   3026768     38.40      2688   2282060    772556      8.65    268248   2383928        76
01:13:01 AM   4855472   6968536   3026952     38.40      2688   2282420    772556      8.65    268248   2384180       224
01:14:01 AM   4855516   6968948   3026908     38.40      2688   2282796    772556      8.65    268248   2384448        68
01:15:01 AM   4854948   6968908   3027476     38.41      2688   2283324    772556      8.65    268248   2384860        56
01:16:01 AM   4854464   6968528   3027960     38.41      2688   2283420    772556      8.65    268248   2385344        36
01:17:01 AM   4854700   6968852   3027724     38.41      2688   2283516    772556      8.65    268248   2385124        60
01:18:01 AM   4854828   6969092   3027596     38.41      2688   2283620    772556      8.65    268248   2385168         4
01:19:01 AM   4854832   6969192   3027592     38.41      2688   2283716    772556      8.65    268248   2385268        40
01:20:01 AM   4854288   6968744   3028136     38.42      2688   2283812    766668      8.58    268248   2385480        72
01:21:01 AM   4846416   6963432   3036008     38.52      2688   2286372    766668      8.58    268248   2393260      1164
01:22:01 AM   4840712   6960192   3041712     38.59      2688   2288840    785308      8.79    268260   2399116       100
01:23:01 AM   4839296   6959140   3043128     38.61      2688   2289204    785308      8.79    268256   2400548         4
01:24:02 AM   4838852   6959056   3043572     38.61      2688   2289564    785308      8.79    268256   2400908        36
01:25:01 AM   4838276   6958844   3044148     38.62      2688   2289924    785308      8.79    268256   2401540       216
01:26:01 AM   4838880   6959820   3043544     38.61      2688   2290288    785308      8.79    268256   2400676       160
01:27:01 AM   4838620   6959940   3043804     38.62      2688   2290668    785308      8.79    268256   2401124        36
01:28:01 AM   4838492   6960160   3043932     38.62      2688   2291016    785308      8.79    268256   2401404         4
01:29:01 AM   4837960   6959752   3044464     38.62      2688   2291140    785308      8.79    268600   2401324         4
01:30:01 AM   4837940   6959732   3044484     38.62      2688   2291140    786180      8.80    268624   2401532         4
01:31:01 AM   4838212   6960144   3044212     38.62      2688   2291276    786180      8.80    268628   2401316         4
01:32:01 AM   4837916   6959848   3044508     38.62      2688   2291276    786180      8.80    268628   2401384         4
01:33:01 AM   4837752   6959776   3044672     38.63      2688   2291368    786180      8.80    268628   2401680         4
01:34:01 AM   4837984   6960012   3044440     38.62      2688   2291372    786180      8.80    268628   2401412         4
01:35:01 AM   4837452   6959576   3044972     38.63      2688   2291468    786180      8.80    268628   2401924         4
01:36:01 AM   4837912   6960036   3044512     38.62      2688   2291468    786180      8.80    268628   2401516         4
01:37:01 AM   4837640   6959868   3044784     38.63      2688   2291572    786592      8.81    277916   2392408         4
01:38:01 AM   4837784   6960012   3044640     38.63      2688   2291572    786592      8.81    277916   2392340         4
01:39:01 AM   4837764   6960084   3044660     38.63      2688   2291664    786592      8.81    277916   2392424       100
01:40:01 AM   4837496   6959820   3044928     38.63      2688   2291668    784648      8.79    277916   2392552         4
01:41:01 AM   4837144   6959476   3045280     38.63      2688   2291668    784648      8.79    277916   2392864         4
01:42:01 AM   4836804   6959540   3045620     38.64      2688   2292080    784648      8.79    277916   2393308        96
01:43:01 AM   4836816   6959916   3045608     38.64      2688   2292444    784648      8.79    277916   2393400        20
01:44:01 AM   4836436   6959896   3045988     38.64      2688   2292804    784648      8.79    277916   2393656       188
01:45:01 AM   4835964   6959800   3046460     38.65      2688   2293180    784648      8.79    277916   2394084        36
01:46:01 AM   4835576   6959780   3046848     38.65      2688   2293548    784648      8.79    277916   2394476        84
01:47:01 AM   4835104   6959668   3047320     38.66      2688   2293908    784648      8.79    277916   2394904        40
01:48:01 AM   4835212   6960060   3047212     38.66      2688   2294180    784648      8.79    277916   2395128         4
01:49:01 AM   4834936   6959788   3047488     38.66      2688   2294184    784648      8.79    277916   2395128         4
01:50:01 AM   4834468   6959412   3047956     38.67      2688   2294276    784648      8.79    277916   2395708       112
01:51:01 AM   4834912   6959876   3047512     38.66      2688   2294284    800044      8.96    277916   2395236         8
01:52:01 AM   4834732   6959788   3047692     38.66      2688   2294376    800044      8.96    277916   2395396       100
01:53:01 AM   4834708   6959764   3047716     38.66      2688   2294376    800044      8.96    277916   2395444         4
01:54:01 AM   4834796   6959948   3047628     38.66      2688   2294472    800044      8.96    277916   2395424       100
01:55:01 AM   4834796   6959948   3047628     38.66      2688   2294472    784480      8.78    277916   2395408         4
01:56:01 AM   4834612   6959856   3047812     38.67      2688   2294564    784480      8.78    277916   2395520       100
01:57:01 AM   4834548   6959796   3047876     38.67      2688   2294568    784480      8.78    277916   2395592        12
01:58:01 AM   4834512   6959852   3047912     38.67      2688   2294660    784480      8.78    277916   2395608       104
01:59:01 AM   4834680   6960020   3047744     38.67      2688   2294660    784480      8.78    277916   2395616         4
02:00:01 AM   4834068   6959496   3048356     38.67      2688   2294756    801584      8.98    277916   2395812       100
02:01:01 AM   4833072   6958856   3049352     38.69      2688   2295108    801584      8.98    277916   2396744       188
02:02:01 AM   4833316   6959464   3049108     38.68      2688   2295472    801584      8.98    277936   2396596       156
02:03:01 AM   4833128   6959644   3049296     38.68      2688   2295832    801584      8.98    277936   2396964        96
02:04:01 AM   4832492   6959372   3049932     38.69      2688   2296196    801584      8.98    277936   2397548        68
02:05:01 AM   4832240   6959492   3050184     38.70      2688   2296576    801584      8.98    277936   2397788        28
02:06:01 AM   4831780   6959392   3050644     38.70      2688   2296936    801584      8.98    277936   2398244       168
02:07:01 AM   4831756   6959616   3050668     38.70      2688   2297176    801584      8.98    277936   2398228         8
02:08:01 AM   4831868   6959736   3050556     38.70      2688   2297184    801584      8.98    277936   2398284         4
02:09:01 AM   4831836   6959796   3050588     38.70      2688   2297276    801584      8.98    277936   2398260         4
02:10:01 AM   4831636   6959596   3050788     38.70      2688   2297276    784780      8.79    277936   2398380         4
02:11:01 AM   4831848   6959896   3050576     38.70      2688   2297372    784780      8.79    277936   2398360       100
02:12:01 AM   4831616   6959664   3050808     38.70      2688   2297372    784780      8.79    277936   2398428         4
02:13:01 AM   4831832   6959976   3050592     38.70      2688   2297468    784780      8.79    277936   2398456       104
02:14:01 AM   4831396   6959540   3051028     38.71      2688   2297468    784780      8.79    277936   2398456         4
02:15:01 AM   4831424   6959664   3051000     38.71      2688   2297564    784780      8.79    277936   2398552       104
02:16:01 AM   4831460   6959704   3050964     38.71      2688   2297568    784780      8.79    277936   2398556         4
02:17:01 AM   4831496   6959832   3050928     38.71      2688   2297660    784780      8.79    277936   2398716       108
02:18:01 AM   4831480   6959816   3050944     38.71      2688   2297660    784780      8.79    277936   2398648         8
02:19:01 AM   4831420   6959880   3051004     38.71      2688   2297800    761592      8.53    277936   2398740       144
02:20:01 AM   4830892   6959488   3051532     38.71      2688   2297928    778752      8.72    277936   2399008        24
02:21:01 AM   4830792   6959748   3051632     38.71      2688   2298288    778752      8.72    277936   2399248        40
02:22:01 AM   4829972   6959292   3052452     38.72      2688   2298652    778752      8.72    277936   2400024        24
02:23:01 AM   4830036   6959724   3052388     38.72      2688   2299012    778752      8.72    277936   2400120         4
02:24:01 AM   4829880   6959932   3052544     38.73      2688   2299376    778752      8.72    277936   2400360         0
02:25:01 AM   4829408   6959828   3053016     38.73      2688   2299752    778752      8.72    277936   2400720        36
02:26:01 AM   4828816   6959688   3053608     38.74      2688   2300196    778752      8.72    277936   2401296        80
02:27:02 AM   4828668   6959944   3053756     38.74      2688   2300608    778752      8.72    277936   2401640        68
02:28:01 AM   4828256   6959632   3054168     38.75      2688   2300708    778752      8.72    277936   2401672        28
02:29:01 AM   4828248   6959728   3054176     38.75      2688   2300804    778752      8.72    277936   2401776        56
02:30:01 AM   4827928   6959508   3054496     38.75      2688   2300904    777096      8.70    277936   2401992        24
02:31:01 AM   4828124   6959788   3054300     38.75      2688   2301004    777096      8.70    277936   2401960        56
02:32:01 AM   4827912   6959680   3054512     38.75      2688   2301100    777096      8.70    277936   2402248        48
02:33:01 AM   4827340   6959312   3055084     38.76      2688   2301300    777096      8.70    277936   2402588         4
02:34:01 AM   4827568   6959596   3054856     38.76      2688   2301356    777096      8.70    277936   2402324        24
02:35:01 AM   4827656   6959888   3054768     38.75      2688   2301568    777096      8.70    277936   2402536       148
02:36:01 AM   4827312   6959912   3055112     38.76      2688   2301932    777096      8.70    277936   2402900        80
02:37:01 AM   4826784   6959748   3055640     38.77      2688   2302296    777096      8.70    277936   2403448        40
02:38:01 AM   4826484   6959816   3055940     38.77      2688   2302656    777096      8.70    277936   2403632       188
02:39:01 AM   4826156   6959864   3056268     38.77      2688   2303036    777096      8.70    277936   2404008       160
02:40:01 AM   4825700   6959768   3056724     38.78      2688   2303396    768376      8.60    277936   2404480       204
02:41:01 AM   4825088   6959532   3057336     38.79      2688   2303764    768376      8.60    277936   2404864       160
02:42:01 AM   4824984   6959632   3057440     38.79      2688   2303976    768376      8.60    277936   2405036       156
02:43:01 AM   4824384   6959392   3058040     38.80      2688   2304336    768376      8.60    277936   2405528       140
02:44:01 AM   4824332   6959708   3058092     38.80      2688   2304704    768376      8.60    277936   2405692         4
02:45:01 AM   4821964   6957708   3060460     38.83      2688   2305064    784808      8.79    277936   2408044        84
02:46:01 AM   4823788   6959900   3058636     38.80      2688   2305440    776572      8.70    278716   2405652       164
02:47:01 AM   4822956   6959432   3059468     38.81      2688   2305804    776572      8.70    278716   2406084        24
02:48:01 AM   4822548   6959396   3059876     38.82      2688   2306168    776572      8.70    278716   2406616         4
02:49:01 AM   4822316   6959380   3060108     38.82      2688   2306384    776572      8.70    278716   2406748        48
02:50:01 AM   4821616   6959056   3060808     38.83      2688   2306764    776636      8.70    278716   2407420       188
02:51:01 AM   4821656   6959460   3060768     38.83      2688   2307132    776636      8.70    278716   2407400       104
02:52:01 AM   4821308   6959488   3061116     38.83      2688   2307508    776636      8.70    278716   2407844        48
02:53:01 AM   4821032   6959576   3061392     38.84      2688   2307872    776636      8.70    278716   2408144       192
02:54:01 AM   4820440   6959352   3061984     38.85      2688   2308232    776636      8.70    278716   2408540       144
02:55:01 AM   4820244   6959508   3062180     38.85      2688   2308592    776636      8.70    278716   2408984        96
02:56:01 AM   4819608   6959416   3062816     38.86      2688   2309136    776636      8.70    278716   2409528        72
02:57:01 AM   4819780   6959692   3062644     38.85      2688   2309232    776636      8.70    278716   2409588        48
02:58:01 AM   4819596   6959604   3062828     38.86      2688   2309328    776636      8.70    278716   2409716         8
02:59:01 AM   4819432   6959540   3062992     38.86      2688   2309428    776636      8.70    278716   2409700        40
03:00:01 AM   4818968   6959160   3063456     38.86      2688   2309528    794904      8.90    278716   2409924        72
03:01:01 AM   4818120   6958420   3064304     38.88      2688   2309624    794904      8.90    278716   2410728        40
03:02:01 AM   4819080   6959588   3063344     38.86      2688   2309824    778536      8.72    278716   2410184         4
03:03:01 AM   4818812   6959376   3063612     38.87      2688   2309880    778536      8.72    278716   2410168         4
03:04:01 AM   4818772   6959480   3063652     38.87      2688   2310024    778536      8.72    278716   2410396         4
03:05:01 AM   4818796   6959564   3063628     38.87      2688   2310084    778536      8.72    278716   2410372         4
03:06:01 AM   4818776   6959608   3063648     38.87      2688   2310148    778536      8.72    278716   2410436         4
03:07:01 AM   4818468   6959532   3063956     38.87      2688   2310380    778536      8.72    278716   2410736        40
03:08:01 AM   4818328   6959544   3064096     38.87      2688   2310540    778536      8.72    278716   2410944       164
03:09:01 AM   4817688   6959264   3064736     38.88      2688   2310900    778536      8.72    278716   2411480       116
03:10:01 AM   4816960   6958896   3065464     38.89      2688   2311260    795568      8.91    278716   2411984       104
03:11:01 AM   4816792   6959100   3065632     38.89      2688   2311628    795568      8.91    278716   2412192        60
03:12:01 AM   4816492   6959176   3065932     38.90      2688   2312004    795568      8.91    278716   2412576        32
03:13:01 AM   4816048   6959104   3066376     38.90      2688   2312368    795568      8.91    278716   2412860       180
03:14:01 AM   4815716   6959500   3066708     38.91      2688   2313096    795568      8.91    278716   2413472       324
03:15:01 AM   4815044   6958916   3067380     38.91      2688   2313192    795568      8.91    278716   2413868        24
03:16:01 AM   4815208   6959180   3067216     38.91      2688   2313292    780068      8.73    278716   2413996        24
03:17:01 AM   4815260   6959336   3067164     38.91      2688   2313388    780068      8.73    278716   2413904         4
03:18:01 AM   4814968   6959132   3067456     38.92      2688   2313492    780068      8.73    278716   2414060        24
03:19:01 AM   4815132   6959396   3067292     38.91      2688   2313592    780068      8.73    278716   2414140        24
03:20:01 AM   4814400   6958780   3068024     38.92      2688   2313692    776780      8.70    278716   2414468        32
03:21:01 AM   4814516   6959108   3067908     38.92      2688   2313904    776780      8.70    278724   2414348        56
03:22:01 AM   4814512   6959196   3067912     38.92      2688   2313996    776780      8.70    278732   2414668       112
03:23:01 AM   4814172   6958928   3068252     38.93      2688   2314068    776780      8.70    278748   2414792         4
03:24:01 AM   4814364   6959196   3068060     38.92      2688   2314144    776780      8.70    278748   2414712         4
03:25:01 AM   4814436   6959364   3067988     38.92      2688   2314240    776780      8.70    278748   2414696        56
03:26:01 AM   4814024   6959144   3068400     38.93      2688   2314432    776780      8.70    278752   2414884         4
03:27:01 AM   4813752   6959128   3068672     38.93      2688   2314696    792260      8.87    278752   2415228        20
03:28:02 AM   4813452   6959176   3068972     38.93      2688   2315044    792260      8.87    278752   2415620       164
03:29:01 AM   4813116   6959216   3069308     38.94      2688   2315420    792260      8.87    278752   2415996        68
03:30:01 AM   4812532   6958996   3069892     38.95      2688   2315784    786972      8.81    278752   2416360        24
03:31:01 AM   4812544   6959372   3069880     38.95      2688   2316144    786972      8.81    278752   2416608        36
03:32:01 AM   4811904   6959100   3070520     38.95      2688   2316512    786972      8.81    278752   2417196        68
03:33:01 AM   4811172   6959036   3071252     38.96      2688   2317168    786972      8.81    278752   2417880        24
03:34:01 AM   4811380   6959340   3071044     38.96      2688   2317264    786972      8.81    278752   2417772        56
03:35:01 AM   4811304   6959352   3071120     38.96      2688   2317360    786972      8.81    278752   2417860        24
03:36:01 AM   4811164   6959312   3071260     38.96      2688   2317460    786972      8.81    278752   2417960        56
03:37:01 AM   4810952   6959212   3071472     38.97      2688   2317564    786972      8.81    278752   2418140        24
03:38:01 AM   4810908   6959248   3071516     38.97      2688   2317660    786972      8.81    278752   2418268        60
03:39:01 AM   4810672   6959244   3071752     38.97      2688   2317876    771356      8.64    278752   2418368        64
03:40:01 AM   4810312   6958964   3072112     38.97      2688   2317956    793644      8.89    278752   2418680         4
03:41:01 AM   4810660   6959420   3071764     38.97      2688   2318060    767008      8.59    278752   2418552         4
03:42:01 AM   4810492   6959268   3071932     38.97      2688   2318076    767008      8.59    278752   2418748         4
03:43:01 AM   4810188   6959192   3072236     38.98      2688   2318304    767008      8.59    278752   2418800       184
03:44:01 AM   4809924   6959188   3072500     38.98      2688   2318572    767008      8.59    278752   2419080        24
03:45:01 AM   4809684   6959308   3072740     38.98      2688   2318932    767008      8.59    278752   2419436        40
03:46:01 AM   4809288   6959288   3073136     38.99      2688   2319308    767008      8.59    278752   2419816        40
03:47:01 AM   4808840   6959204   3073584     38.99      2688   2319672    767008      8.59    278752   2420244        24
03:48:01 AM   4808468   6959200   3073956     39.00      2688   2320032    767008      8.59    278752   2420652         4
03:49:01 AM   4808404   6959488   3074020     39.00      2688   2320392    767008      8.59    278752   2420896        20
03:50:01 AM   4807172   6958924   3075252     39.01      2688   2321052    790008      8.85    278752   2421876        32
03:51:01 AM   4807212   6959064   3075212     39.01      2688   2321148    790008      8.85    278752   2421780         4
03:52:01 AM   4807284   6959236   3075140     39.01      2688   2321248    790008      8.85    278752   2421832         4
03:53:01 AM   4807196   6959244   3075228     39.01      2688   2321344    790008      8.85    278752   2421976         4
03:54:01 AM   4807240   6959392   3075184     39.01      2688   2321448    790008      8.85    278752   2421964         4
03:55:01 AM   4807188   6959440   3075236     39.01      2688   2321548    790008      8.85    278752   2422064         4
03:56:01 AM   4806764   6959260   3075660     39.02      2688   2321792    790008      8.85    278752   2422292       144
03:57:01 AM   4806200   6958884   3076224     39.03      2688   2321980    790008      8.85    278752   2422576         4
03:58:01 AM   4803928   6956808   3078496     39.06      2688   2322176    805668      9.02    278756   2424976        48
03:59:01 AM   4803704   6956772   3078720     39.06      2688   2322364    805668      9.02    278756   2425300         4
04:00:01 AM   4800188   6953644   3082236     39.10      2688   2322752    844004      9.45    278756   2428128       200
04:01:01 AM   4798656   6952540   3083768     39.12      2688   2323176    876700      9.82    278756   2429356       240
04:02:01 AM   4867344   6930656   3015080     38.25      2688   2237864    733132      8.21    340940   2316580     84620
04:03:01 AM   4867840   6933420   3014584     38.24      2688   2240512    733388      8.21    539496   2121568     17660
Average:      4841060   6962553   3041364     38.58      2688   2290889    780694      8.74    275372   2391681       491

12:00:01 AM     IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
12:01:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:01:01 AM      eth0      1.53      1.00      0.38      0.16      0.00      0.00      0.00      0.00
12:02:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:02:01 AM      eth0     71.56     70.60     11.21      5.44      0.00      0.00      0.00      0.00
12:03:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:03:01 AM      eth0     21.93     17.80      3.99      2.35      0.00      0.00      0.00      0.00
12:04:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:04:01 AM      eth0     13.46      9.13      2.68      1.76      0.00      0.00      0.00      0.00
12:05:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:05:01 AM      eth0     13.63      9.50      2.95      1.97      0.00      0.00      0.00      0.00
12:06:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:06:01 AM      eth0     13.53      9.42      2.71      1.80      0.00      0.00      0.00      0.00
12:07:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:07:01 AM      eth0     14.01      9.83      2.95      1.89      0.00      0.00      0.00      0.00
12:08:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:08:01 AM      eth0     13.56      9.11      2.71      1.76      0.00      0.00      0.00      0.00
12:09:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:09:01 AM      eth0     23.56     16.11      4.90      3.00      0.00      0.00      0.00      0.00
12:10:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:10:01 AM      eth0      0.58      0.18      0.03      0.01      0.00      0.00      0.00      0.00
12:11:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:11:01 AM      eth0      0.37      0.20      0.02      0.01      0.00      0.00      0.00      0.00
12:12:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:12:01 AM      eth0      0.92      0.32      0.17      0.08      0.00      0.00      0.00      0.00
12:13:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:13:01 AM      eth0      0.73      0.80      0.04      2.46      0.00      0.00      0.00      0.00
12:14:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:14:01 AM      eth0      1.02      0.20      0.13      0.01      0.00      0.00      0.00      0.00
12:15:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:15:01 AM      eth0     19.81     18.35      3.11      1.66      0.00      0.00      0.00      0.00
12:16:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:16:01 AM      eth0      0.47      0.07      0.02      0.00      0.00      0.00      0.00      0.00
12:17:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:17:01 AM      eth0      8.05      6.98      1.54      0.83      0.00      0.00      0.00      0.00
12:18:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:18:01 AM      eth0     13.41      9.25      2.69      1.77      0.00      0.00      0.00      0.00
12:19:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:19:01 AM      eth0     12.78      9.23      2.70      1.81      0.00      0.00      0.00      0.00
12:20:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:20:01 AM      eth0     14.20      9.83      2.83      1.81      0.00      0.00      0.00      0.00
12:21:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:21:01 AM      eth0     13.43      9.43      2.71      1.78      0.00      0.00      0.00      0.00
12:22:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:22:01 AM      eth0     14.08      9.48      2.98      1.86      0.00      0.00      0.00      0.00
12:23:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:23:01 AM      eth0     17.24     13.48      3.37      2.00      0.00      0.00      0.00      0.00
12:24:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:24:01 AM      eth0      1.33      0.28      0.37      0.19      0.00      0.00      0.00      0.00
12:25:02 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:25:02 AM      eth0     14.01     12.43      2.52      1.45      0.00      0.00      0.00      0.00
12:26:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:26:01 AM      eth0     14.00      9.69      2.87      1.82      0.00      0.00      0.00      0.00
12:27:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:27:01 AM      eth0     13.46      9.38      2.81      1.83      0.00      0.00      0.00      0.00
12:28:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:28:01 AM      eth0     14.25      9.72      3.10      2.03      0.00      0.00      0.00      0.00
12:29:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:29:01 AM      eth0     13.10      9.23      2.68      1.77      0.00      0.00      0.00      0.00
12:30:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:30:01 AM      eth0     13.33      9.20      2.75      1.76      0.00      0.00      0.00      0.00
12:31:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:31:01 AM      eth0     13.11      9.28      2.66      1.77      0.00      0.00      0.00      0.00
12:32:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:32:01 AM      eth0     18.16     12.08      3.80      2.26      0.00      0.00      0.00      0.00
12:33:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:33:01 AM      eth0      1.35      0.42      0.63      0.39      0.00      0.00      0.00      0.00
12:34:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:34:01 AM      eth0      0.98      0.15      0.07      0.01      0.00      0.00      0.00      0.00
12:35:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:35:01 AM      eth0      1.35      0.58      0.57      0.40      0.00      0.00      0.00      0.00
12:36:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:36:01 AM      eth0      1.43      1.00      0.10      0.08      0.00      0.00      0.00      0.00
12:37:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:37:01 AM      eth0      0.60      0.27      0.15      0.07      0.00      0.00      0.00      0.00
12:38:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:38:01 AM      eth0     33.96     29.92      5.85      3.31      0.00      0.00      0.00      0.00
12:39:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:39:01 AM      eth0     13.25      9.43      2.72      1.78      0.00      0.00      0.00      0.00
12:40:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:40:01 AM      eth0     13.68      9.26      2.74      1.77      0.00      0.00      0.00      0.00
12:41:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:41:01 AM      eth0     13.65      9.51      3.02      2.02      0.00      0.00      0.00      0.00
12:42:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:42:01 AM      eth0     14.50      9.48      3.20      2.04      0.00      0.00      0.00      0.00
12:43:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:43:01 AM      eth0     13.38      9.30      2.78      1.78      0.00      0.00      0.00      0.00
12:44:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:44:01 AM      eth0     24.33     19.66      4.65      2.63      0.00      0.00      0.00      0.00
12:45:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:45:01 AM      eth0     13.58      9.55      2.78      1.79      0.00      0.00      0.00      0.00
12:46:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:46:01 AM      eth0     13.00      9.06      2.66      1.76      0.00      0.00      0.00      0.00
12:47:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:47:01 AM      eth0     13.43      9.37      2.84      1.83      0.00      0.00      0.00      0.00
12:48:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:48:01 AM      eth0     14.56      9.60      3.08      1.97      0.00      0.00      0.00      0.00
12:49:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:49:01 AM      eth0     12.93      9.23      2.64      1.77      0.00      0.00      0.00      0.00
12:50:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:50:01 AM      eth0     30.99     26.11      5.91      3.47      0.00      0.00      0.00      0.00
12:51:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:51:01 AM      eth0     13.60      9.28      2.78      1.77      0.00      0.00      0.00      0.00
12:52:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:52:01 AM      eth0     13.96      9.73      3.11      2.12      0.00      0.00      0.00      0.00
12:53:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:53:01 AM      eth0     13.25      9.25      2.71      1.79      0.00      0.00      0.00      0.00
12:54:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:54:01 AM      eth0     13.95      9.45      2.82      1.80      0.00      0.00      0.00      0.00
12:55:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:55:01 AM      eth0     15.26      9.46      3.16      1.96      0.00      0.00      0.00      0.00
12:56:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:56:01 AM      eth0     23.61     16.75      5.12      3.30      0.00      0.00      0.00      0.00
12:57:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:57:01 AM      eth0      0.77      0.28      0.16      0.08      0.00      0.00      0.00      0.00
12:58:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:58:01 AM      eth0      5.23      0.22      0.56      0.01      0.00      0.00      0.00      0.00
12:59:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:59:01 AM      eth0      1.62      0.43      0.39      0.24      0.00      0.00      0.00      0.00
01:00:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:00:01 AM      eth0      0.93      0.42      0.28      0.21      0.00      0.00      0.00      0.00
01:01:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:01:01 AM      eth0      3.08      0.17      0.36      0.01      0.00      0.00      0.00      0.00
01:02:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:02:01 AM      eth0     27.39     24.89      4.48      2.40      0.00      0.00      0.00      0.00
01:03:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:03:01 AM      eth0     14.31      9.48      2.83      1.82      0.00      0.00      0.00      0.00
01:04:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:04:01 AM      eth0     14.40      9.27      2.90      1.78      0.00      0.00      0.00      0.00
01:05:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:05:01 AM      eth0     14.13      9.53      3.03      1.97      0.00      0.00      0.00      0.00
01:06:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:06:01 AM      eth0     13.81      9.45      2.78      1.80      0.00      0.00      0.00      0.00
01:07:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:07:01 AM      eth0     14.31      9.63      3.01      1.88      0.00      0.00      0.00      0.00
01:08:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:08:01 AM      eth0     29.59     25.14      5.59      3.31      0.00      0.00      0.00      0.00
01:09:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:09:01 AM      eth0     14.66     10.30      2.99      1.94      0.00      0.00      0.00      0.00
01:10:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:10:01 AM      eth0     13.38      9.15      2.74      1.76      0.00      0.00      0.00      0.00
01:11:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:11:01 AM      eth0     14.83      9.53      2.89      1.79      0.00      0.00      0.00      0.00
01:12:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:12:01 AM      eth0     17.24      9.63      3.89      2.39      0.00      0.00      0.00      0.00
01:13:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:13:01 AM      eth0     13.81      9.50      2.99      2.00      0.00      0.00      0.00      0.00
01:14:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:14:01 AM      eth0     19.78      9.45      3.60      1.79      0.00      0.00      0.00      0.00
01:15:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:15:01 AM      eth0     16.74     11.50      3.39      2.11      0.00      0.00      0.00      0.00
01:16:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:16:01 AM      eth0      1.82      0.38      0.36      0.21      0.00      0.00      0.00      0.00
01:17:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:17:01 AM      eth0      0.70      0.28      0.16      0.07      0.00      0.00      0.00      0.00
01:18:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:18:01 AM      eth0      1.12      0.20      0.05      0.01      0.00      0.00      0.00      0.00
01:19:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:19:01 AM      eth0      0.67      0.15      0.04      0.01      0.00      0.00      0.00      0.00
01:20:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:20:01 AM      eth0      2.13      0.18      0.09      0.01      0.00      0.00      0.00      0.00
01:21:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:21:01 AM      eth0     24.88     21.31      4.14      2.28      0.00      0.00      0.00      0.00
01:22:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:22:01 AM      eth0     12.71     10.74      2.42      1.45      0.00      0.00      0.00      0.00
01:23:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:23:01 AM      eth0     13.53      9.30      2.74      1.77      0.00      0.00      0.00      0.00
01:24:02 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:24:02 AM      eth0     13.68      9.36      2.72      1.77      0.00      0.00      0.00      0.00
01:25:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:25:01 AM      eth0     14.00      9.45      3.01      1.98      0.00      0.00      0.00      0.00
01:26:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:26:01 AM      eth0     13.31      9.53      2.73      1.81      0.00      0.00      0.00      0.00
01:27:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:27:01 AM      eth0     13.80      9.50      2.93      1.87      0.00      0.00      0.00      0.00
01:28:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:28:01 AM      eth0     15.14     11.21      2.99      1.77      0.00      0.00      0.00      0.00
01:29:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:29:01 AM      eth0      0.47      0.27      0.05      0.03      0.00      0.00      0.00      0.00
01:30:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:30:01 AM      eth0      0.55      0.05      0.02      0.00      0.00      0.00      0.00      0.00
01:31:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:31:01 AM      eth0      0.47      0.23      0.04      0.02      0.00      0.00      0.00      0.00
01:32:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:32:01 AM      eth0      1.02      0.10      0.20      0.06      0.00      0.00      0.00      0.00
01:33:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:33:01 AM      eth0      0.83      0.42      0.30      0.22      0.00      0.00      0.00      0.00
01:34:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:34:01 AM      eth0      0.68      0.00      0.06      0.00      0.00      0.00      0.00      0.00
01:35:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:35:01 AM      eth0      0.80      0.48      0.30      0.22      0.00      0.00      0.00      0.00
01:36:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:36:01 AM      eth0      0.40      0.00      0.02      0.00      0.00      0.00      0.00      0.00
01:37:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:37:01 AM      eth0      0.62      0.35      0.17      0.09      0.00      0.00      0.00      0.00
01:38:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:38:01 AM      eth0      0.35      0.00      0.01      0.00      0.00      0.00      0.00      0.00
01:39:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:39:01 AM      eth0      0.37      0.17      0.04      0.02      0.00      0.00      0.00      0.00
01:40:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:40:01 AM      eth0      0.58      0.18      0.03      0.01      0.00      0.00      0.00      0.00
01:41:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:41:01 AM      eth0      1.42      0.18      0.36      0.19      0.00      0.00      0.00      0.00
01:42:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:42:01 AM      eth0     15.06     11.48      3.27      2.08      0.00      0.00      0.00      0.00
01:43:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:43:01 AM      eth0     13.13      9.23      2.70      1.79      0.00      0.00      0.00      0.00
01:44:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:44:01 AM      eth0     13.66      9.41      2.84      1.80      0.00      0.00      0.00      0.00
01:45:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:45:01 AM      eth0     12.93      9.13      2.64      1.76      0.00      0.00      0.00      0.00
01:46:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:46:01 AM      eth0     13.20      9.35      2.70      1.77      0.00      0.00      0.00      0.00
01:47:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:47:01 AM      eth0     14.05      9.82      2.98      1.91      0.00      0.00      0.00      0.00
01:48:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:48:01 AM      eth0      6.78      5.43      1.35      0.73      0.00      0.00      0.00      0.00
01:49:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:49:01 AM      eth0      0.33      0.07      0.02      0.00      0.00      0.00      0.00      0.00
01:50:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:50:01 AM      eth0      1.07      0.43      0.31      0.22      0.00      0.00      0.00      0.00
01:51:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:51:01 AM      eth0      0.20      0.02      0.01      0.00      0.00      0.00      0.00      0.00
01:52:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:52:01 AM      eth0      0.75      0.47      0.19      0.11      0.00      0.00      0.00      0.00
01:53:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:53:01 AM      eth0      0.22      0.00      0.01      0.00      0.00      0.00      0.00      0.00
01:54:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:54:01 AM      eth0      0.60      0.30      0.05      0.03      0.00      0.00      0.00      0.00
01:55:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:55:01 AM      eth0      0.13      0.00      0.01      0.00      0.00      0.00      0.00      0.00
01:56:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:56:01 AM      eth0      0.60      0.27      0.05      0.03      0.00      0.00      0.00      0.00
01:57:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:57:01 AM      eth0      0.42      0.17      0.14      0.07      0.00      0.00      0.00      0.00
01:58:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:58:01 AM      eth0      0.72      0.23      0.05      0.02      0.00      0.00      0.00      0.00
01:59:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:59:01 AM      eth0      0.12      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:00:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:00:01 AM      eth0      3.62      3.30      0.50      0.26      0.00      0.00      0.00      0.00
02:01:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:01:01 AM      eth0     12.81      9.15      2.62      1.71      0.00      0.00      0.00      0.00
02:02:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:02:01 AM      eth0     14.88      9.43      3.29      2.02      0.00      0.00      0.00      0.00
02:03:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:03:01 AM      eth0     13.43      9.35      2.98      1.95      0.00      0.00      0.00      0.00
02:04:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:04:01 AM      eth0     16.63      9.56      3.30      2.01      0.00      0.00      0.00      0.00
02:05:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:05:01 AM      eth0     18.12      9.91      3.98      2.35      0.00      0.00      0.00      0.00
02:06:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:06:01 AM      eth0     16.21      9.45      3.53      2.14      0.00      0.00      0.00      0.00
02:07:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:07:01 AM      eth0     12.49      6.01      2.10      0.77      0.00      0.00      0.00      0.00
02:08:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:08:01 AM      eth0      1.33      0.03      0.13      0.00      0.00      0.00      0.00      0.00
02:09:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:09:01 AM      eth0      1.03      0.25      0.08      0.02      0.00      0.00      0.00      0.00
02:10:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:10:01 AM      eth0      0.57      0.05      0.02      0.00      0.00      0.00      0.00      0.00
02:11:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:11:01 AM      eth0      0.98      0.25      0.07      0.02      0.00      0.00      0.00      0.00
02:12:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:12:01 AM      eth0      0.67      0.15      0.16      0.07      0.00      0.00      0.00      0.00
02:13:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:13:01 AM      eth0      1.53      0.27      0.09      0.03      0.00      0.00      0.00      0.00
02:14:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:14:01 AM      eth0      3.42      0.05      0.14      0.00      0.00      0.00      0.00      0.00
02:15:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:15:01 AM      eth0      2.37      0.28      0.12      0.03      0.00      0.00      0.00      0.00
02:16:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:16:01 AM      eth0      0.67      0.00      0.03      0.00      0.00      0.00      0.00      0.00
02:17:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:17:01 AM      eth0      0.83      0.40      0.22      0.09      0.00      0.00      0.00      0.00
02:18:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:18:01 AM      eth0      0.50      0.00      0.02      0.00      0.00      0.00      0.00      0.00
02:19:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:19:01 AM      eth0      3.05      2.10      0.56      0.42      0.00      0.00      0.00      0.00
02:20:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:20:01 AM      eth0      8.97      8.10      1.47      0.78      0.00      0.00      0.00      0.00
02:21:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:21:01 AM      eth0     14.78      9.20      2.91      1.76      0.00      0.00      0.00      0.00
02:22:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:22:01 AM      eth0     16.06      9.62      3.58      2.20      0.00      0.00      0.00      0.00
02:23:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:23:01 AM      eth0     13.76      9.53      2.99      1.97      0.00      0.00      0.00      0.00
02:24:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:24:01 AM      eth0     19.65      9.17      3.65      1.80      0.00      0.00      0.00      0.00
02:25:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:25:01 AM      eth0     15.35      9.48      2.94      1.79      0.00      0.00      0.00      0.00
02:26:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:26:01 AM      eth0     16.31     10.90      3.24      2.13      0.00      0.00      0.00      0.00
02:27:02 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:27:02 AM      eth0     13.53      9.35      2.86      1.68      0.00      0.00      0.00      0.00
02:28:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:28:01 AM      eth0      0.80      0.14      0.04      0.01      0.00      0.00      0.00      0.00
02:29:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:29:01 AM      eth0      0.95      0.20      0.05      0.01      0.00      0.00      0.00      0.00
02:30:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:30:01 AM      eth0      0.62      0.13      0.03      0.01      0.00      0.00      0.00      0.00
02:31:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:31:01 AM      eth0      0.60      0.22      0.03      0.01      0.00      0.00      0.00      0.00
02:32:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:32:01 AM      eth0      1.75      0.30      0.27      0.08      0.00      0.00      0.00      0.00
02:33:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:33:01 AM      eth0      9.90      8.45      1.89      1.14      0.00      0.00      0.00      0.00
02:34:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:34:01 AM      eth0      5.41      4.45      0.80      0.30      0.00      0.00      0.00      0.00
02:35:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:35:01 AM      eth0      7.51      6.10      1.36      0.81      0.00      0.00      0.00      0.00
02:36:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:36:01 AM      eth0     13.76      9.43      2.73      1.78      0.00      0.00      0.00      0.00
02:37:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:37:01 AM      eth0     13.50      9.22      2.83      1.82      0.00      0.00      0.00      0.00
02:38:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:38:01 AM      eth0     14.30      9.60      2.84      1.84      0.00      0.00      0.00      0.00
02:39:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:39:01 AM      eth0     13.80      9.47      2.84      1.79      0.00      0.00      0.00      0.00
02:40:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:40:01 AM      eth0     13.70      9.28      2.73      1.77      0.00      0.00      0.00      0.00
02:41:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:41:01 AM      eth0     15.31     11.58      2.98      1.80      0.00      0.00      0.00      0.00
02:42:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:42:01 AM      eth0      8.25      6.27      1.57      0.88      0.00      0.00      0.00      0.00
02:43:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:43:01 AM      eth0     14.20      9.51      3.04      1.96      0.00      0.00      0.00      0.00
02:44:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:44:01 AM      eth0     13.70      9.48      2.74      1.82      0.00      0.00      0.00      0.00
02:45:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:45:01 AM      eth0     13.88      9.50      2.82      1.79      0.00      0.00      0.00      0.00
02:46:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:46:01 AM      eth0     14.15      9.41      2.83      1.78      0.00      0.00      0.00      0.00
02:47:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:47:01 AM      eth0     13.95      9.23      2.91      1.82      0.00      0.00      0.00      0.00
02:48:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:48:01 AM      eth0     16.30     12.38      3.25      2.02      0.00      0.00      0.00      0.00
02:49:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:49:01 AM      eth0     10.41      9.16      1.74      1.00      0.00      0.00      0.00      0.00
02:50:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:50:01 AM      eth0     13.95      9.57      3.03      2.00      0.00      0.00      0.00      0.00
02:51:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:51:01 AM      eth0     13.61      9.53      2.75      1.83      0.00      0.00      0.00      0.00
02:52:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:52:01 AM      eth0     13.61      9.40      2.83      1.84      0.00      0.00      0.00      0.00
02:53:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:53:01 AM      eth0     13.08      9.08      2.64      1.76      0.00      0.00      0.00      0.00
02:54:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:54:01 AM      eth0     13.28      9.15      2.68      1.76      0.00      0.00      0.00      0.00
02:55:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:55:01 AM      eth0     12.79      9.06      2.65      1.75      0.00      0.00      0.00      0.00
02:56:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:56:01 AM      eth0     17.58     12.05      3.60      2.19      0.00      0.00      0.00      0.00
02:57:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:57:01 AM      eth0      0.58      0.27      0.15      0.07      0.00      0.00      0.00      0.00
02:58:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:58:01 AM      eth0      0.50      0.15      0.02      0.01      0.00      0.00      0.00      0.00
02:59:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:59:01 AM      eth0      0.42      0.13      0.02      0.01      0.00      0.00      0.00      0.00
03:00:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:00:01 AM      eth0      0.40      0.15      0.02      0.01      0.00      0.00      0.00      0.00
03:01:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:01:01 AM      eth0      0.45      0.20      0.02      0.01      0.00      0.00      0.00      0.00
03:02:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:02:01 AM      eth0      8.01      6.56      1.69      0.89      0.00      0.00      0.00      0.00
03:03:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:03:01 AM      eth0      5.08      4.87      0.71      0.32      0.00      0.00      0.00      0.00
03:04:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:04:01 AM      eth0      8.08      7.95      1.24      0.56      0.00      0.00      0.00      0.00
03:05:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:05:01 AM      eth0      3.92      3.75      0.59      0.27      0.00      0.00      0.00      0.00
03:06:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:06:01 AM      eth0      3.65      3.42      0.58      0.24      0.00      0.00      0.00      0.00
03:07:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:07:01 AM      eth0     12.70     12.00      2.15      1.13      0.00      0.00      0.00      0.00
03:08:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:08:01 AM      eth0      5.17      3.82      1.01      0.69      0.00      0.00      0.00      0.00
03:09:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:09:01 AM      eth0     16.74      9.50      3.59      2.14      0.00      0.00      0.00      0.00
03:10:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:10:01 AM      eth0     17.31     10.26      4.76      3.10      0.00      0.00      0.00      0.00
03:11:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:11:01 AM      eth0     14.60      9.73      3.53      2.34      0.00      0.00      0.00      0.00
03:12:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:12:01 AM      eth0     15.41     10.06      3.32      2.11      0.00      0.00      0.00      0.00
03:13:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:13:01 AM      eth0     13.48      9.10      2.74      1.76      0.00      0.00      0.00      0.00
03:14:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:14:01 AM      eth0     27.70     18.23      5.56      3.44      0.00      0.00      0.00      0.00
03:15:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:15:01 AM      eth0      6.66      0.87      1.61      0.78      0.00      0.00      0.00      0.00
03:16:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:16:01 AM      eth0      2.02      0.60      0.89      0.57      0.00      0.00      0.00      0.00
03:17:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:17:01 AM      eth0      5.10      0.27      0.74      0.07      0.00      0.00      0.00      0.00
03:18:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:18:01 AM      eth0      4.13      0.65      0.89      0.44      0.00      0.00      0.00      0.00
03:19:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:19:01 AM      eth0      2.13      0.17      0.22      0.01      0.00      0.00      0.00      0.00
03:20:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:20:01 AM      eth0      2.03      0.72      0.65      0.48      0.00      0.00      0.00      0.00
03:21:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:21:01 AM      eth0     12.44      9.96      2.23      1.03      0.00      0.00      0.00      0.00
03:22:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:22:01 AM      eth0      9.88      4.78      2.08      0.97      0.00      0.00      0.00      0.00
03:23:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:23:01 AM      eth0      4.07      3.38      1.07      0.62      0.00      0.00      0.00      0.00
03:24:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:24:01 AM      eth0     12.20      7.38      1.63      0.51      0.00      0.00      0.00      0.00
03:25:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:25:01 AM      eth0      6.18      4.10      0.73      0.28      0.00      0.00      0.00      0.00
03:26:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:26:01 AM      eth0      8.90      7.51      1.44      0.76      0.00      0.00      0.00      0.00
03:27:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:27:01 AM      eth0      9.60      6.47      2.00      1.26      0.00      0.00      0.00      0.00
03:28:02 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:28:02 AM      eth0     13.50      9.11      2.66      1.76      0.00      0.00      0.00      0.00
03:29:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:29:01 AM      eth0     13.22      9.22      2.69      1.79      0.00      0.00      0.00      0.00
03:30:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:30:01 AM      eth0     13.85      9.23      2.74      1.77      0.00      0.00      0.00      0.00
03:31:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:31:01 AM      eth0     15.36      9.25      2.90      1.77      0.00      0.00      0.00      0.00
03:32:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:32:01 AM      eth0     15.93      9.50      3.29      2.02      0.00      0.00      0.00      0.00
03:33:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:33:01 AM      eth0     24.51     16.60      5.45      3.42      0.00      0.00      0.00      0.00
03:34:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:34:01 AM      eth0      2.83      0.18      0.32      0.01      0.00      0.00      0.00      0.00
03:35:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:35:01 AM      eth0      0.63      0.15      0.03      0.01      0.00      0.00      0.00      0.00
03:36:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:36:01 AM      eth0      0.78      0.22      0.04      0.01      0.00      0.00      0.00      0.00
03:37:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:37:01 AM      eth0      0.77      0.23      0.16      0.07      0.00      0.00      0.00      0.00
03:38:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:38:01 AM      eth0      0.90      0.18      0.05      0.01      0.00      0.00      0.00      0.00
03:39:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:39:01 AM      eth0      4.75      3.23      0.87      0.61      0.00      0.00      0.00      0.00
03:40:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:40:01 AM      eth0      6.23      5.82      1.18      0.39      0.00      0.00      0.00      0.00
03:41:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:41:01 AM      eth0      3.85      3.47      0.53      0.34      0.00      0.00      0.00      0.00
03:42:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:42:01 AM      eth0      0.82      0.32      0.16      0.08      0.00      0.00      0.00      0.00
03:43:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:43:01 AM      eth0     13.26     12.16      2.11      1.07      0.00      0.00      0.00      0.00
03:44:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:44:01 AM      eth0      9.16      6.35      1.84      1.19      0.00      0.00      0.00      0.00
03:45:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:45:01 AM      eth0     16.00      9.22     61.07      1.80      0.00      0.00      0.00      0.00
03:46:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:46:01 AM      eth0     13.90      9.60      2.87      1.80      0.00      0.00      0.00      0.00
03:47:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:47:01 AM      eth0     13.75      9.38      2.90      1.84      0.00      0.00      0.00      0.00
03:48:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:48:01 AM      eth0     13.51      9.08      2.75      1.78      0.00      0.00      0.00      0.00
03:49:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:49:01 AM      eth0     14.43      9.63      2.95      1.82      0.00      0.00      0.00      0.00
03:50:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:50:01 AM      eth0     22.88     16.25      4.92      3.13      0.00      0.00      0.00      0.00
03:51:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:51:01 AM      eth0      0.30      0.15      0.01      0.01      0.00      0.00      0.00      0.00
03:52:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:52:01 AM      eth0      0.62      0.32      0.15      0.08      0.00      0.00      0.00      0.00
03:53:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:53:01 AM      eth0      0.47      0.17      0.03      0.01      0.00      0.00      0.00      0.00
03:54:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:54:01 AM      eth0      0.45      0.15      0.02      0.01      0.00      0.00      0.00      0.00
03:55:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:55:01 AM      eth0      0.42      0.18      0.02      0.01      0.00      0.00      0.00      0.00
03:56:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:56:01 AM      eth0      6.91      5.83      1.50      0.80      0.00      0.00      0.00      0.00
03:57:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:57:01 AM      eth0      5.98      4.68      1.41      0.75      0.00      0.00      0.00      0.00
03:58:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:58:01 AM      eth0      6.00      4.93      1.25      0.73      0.00      0.00      0.00      0.00
03:59:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:59:01 AM      eth0      5.00      4.17      1.14      0.59      0.00      0.00      0.00      0.00
04:00:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
04:00:01 AM      eth0      9.96      7.91      2.12      1.34      0.00      0.00      0.00      0.00
04:01:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
04:01:01 AM      eth0     12.21     10.60      2.99      1.45      0.00      0.00      0.00      0.00
04:02:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
04:02:01 AM      eth0    113.46     76.68    250.80    327.16      0.00      0.00      0.00      0.00
04:03:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
04:03:01 AM      eth0     20.18     19.26     17.47      9.56      0.00      0.00      0.00      0.00
Average:           lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
Average:         eth0     10.03      7.01      3.22      2.53      0.00      0.00      0.00      0.00


---> sar -P ALL:
Linux 4.18.0-553.5.1.el8.x86_64 (prd-centos8-robot-2c-8g-1436.novalocal) 	02/23/2025 	_x86_64_	(2 CPU)

12:00:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
12:01:01 AM     all      0.99      0.03      0.28      0.05      0.08     98.56
12:01:01 AM       0      1.30      0.05      0.38      0.05      0.08     98.13
12:01:01 AM       1      0.69      0.00      0.18      0.05      0.08     99.00
12:02:01 AM     all      3.50      0.00      0.32      0.00      0.08     96.11
12:02:01 AM       0      2.88      0.00      0.42      0.00      0.10     96.60
12:02:01 AM       1      4.11      0.00      0.22      0.00      0.05     95.62
12:03:01 AM     all      4.17      0.00      0.37      0.00      0.08     95.39
12:03:01 AM       0      4.66      0.00      0.37      0.00      0.07     94.91
12:03:01 AM       1      3.68      0.00      0.37      0.00      0.08     95.86
12:04:01 AM     all      4.50      0.00      0.37      0.00      0.09     95.04
12:04:01 AM       0      5.68      0.00      0.42      0.00      0.10     93.80
12:04:01 AM       1      3.33      0.00      0.32      0.00      0.08     96.27
12:05:01 AM     all      4.41      0.00      0.33      0.00      0.10     95.16
12:05:01 AM       0      5.60      0.00      0.38      0.00      0.10     93.91
12:05:01 AM       1      3.23      0.00      0.27      0.00      0.10     96.41
12:06:01 AM     all      4.34      0.00      0.33      0.00      0.10     95.22
12:06:01 AM       0      4.08      0.00      0.32      0.00      0.10     95.50
12:06:01 AM       1      4.60      0.00      0.35      0.00      0.10     94.95
12:07:01 AM     all      4.45      0.00      0.35      0.00      0.09     95.11
12:07:01 AM       0      6.31      0.00      0.45      0.00      0.10     93.14
12:07:01 AM       1      2.59      0.00      0.25      0.00      0.08     97.07
12:08:01 AM     all      4.21      0.00      0.36      0.00      0.10     95.33
12:08:01 AM       0      5.52      0.00      0.45      0.00      0.12     93.91
12:08:01 AM       1      2.89      0.00      0.27      0.00      0.08     96.76
12:09:01 AM     all      6.42      0.00      0.39      0.00      0.09     93.10
12:09:01 AM       0      8.20      0.00      0.44      0.00      0.08     91.28
12:09:01 AM       1      4.64      0.00      0.33      0.00      0.10     94.93
12:10:01 AM     all      0.66      0.00      0.13      0.00      0.08     99.12
12:10:01 AM       0      0.77      0.00      0.10      0.00      0.07     99.06
12:10:01 AM       1      0.55      0.00      0.17      0.00      0.10     99.18
12:11:01 AM     all      0.61      0.00      0.10      0.00      0.09     99.20
12:11:01 AM       0      0.60      0.00      0.07      0.00      0.07     99.27
12:11:01 AM       1      0.62      0.00      0.13      0.00      0.12     99.13

12:11:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
12:12:01 AM     all      0.65      0.00      0.12      0.00      0.08     99.15
12:12:01 AM       0      0.63      0.00      0.08      0.00      0.07     99.22
12:12:01 AM       1      0.67      0.00      0.15      0.00      0.10     99.08
12:13:01 AM     all      0.65      0.00      0.11      0.00      0.10     99.14
12:13:01 AM       0      0.88      0.00      0.08      0.00      0.10     98.93
12:13:01 AM       1      0.42      0.00      0.13      0.00      0.10     99.35
12:14:01 AM     all      0.56      0.00      0.10      0.00      0.08     99.26
12:14:01 AM       0      0.70      0.00      0.08      0.00      0.08     99.13
12:14:01 AM       1      0.42      0.00      0.12      0.00      0.08     99.38
12:15:01 AM     all      1.99      0.00      0.17      0.00      0.09     97.75
12:15:01 AM       0      1.30      0.00      0.12      0.00      0.08     98.50
12:15:01 AM       1      2.68      0.00      0.22      0.00      0.10     97.00
12:16:01 AM     all      0.49      0.00      0.12      0.00      0.08     99.31
12:16:01 AM       0      0.37      0.00      0.15      0.00      0.08     99.40
12:16:01 AM       1      0.62      0.00      0.08      0.00      0.08     99.21
12:17:01 AM     all      1.53      0.00      0.16      0.00      0.09     98.22
12:17:01 AM       0      1.72      0.00      0.18      0.00      0.10     97.99
12:17:01 AM       1      1.34      0.00      0.13      0.00      0.08     98.45
12:18:01 AM     all      4.12      0.00      0.25      0.00      0.10     95.53
12:18:01 AM       0      5.02      0.00      0.25      0.00      0.10     94.63
12:18:01 AM       1      3.22      0.00      0.25      0.00      0.10     96.43
12:19:01 AM     all      4.24      0.00      0.18      0.00      0.11     95.47
12:19:01 AM       0      3.95      0.00      0.17      0.00      0.10     95.79
12:19:01 AM       1      4.54      0.00      0.20      0.00      0.12     95.15
12:20:01 AM     all      4.41      0.00      0.19      0.00      0.10     95.30
12:20:01 AM       0      5.66      0.00      0.18      0.00      0.12     94.04
12:20:01 AM       1      3.16      0.00      0.20      0.00      0.08     96.55
12:21:01 AM     all      4.34      0.00      0.23      0.00      0.09     95.33
12:21:01 AM       0      5.22      0.00      0.25      0.00      0.08     94.44
12:21:01 AM       1      3.46      0.00      0.22      0.00      0.10     96.22
12:22:01 AM     all      4.21      0.00      0.21      0.00      0.10     95.48
12:22:01 AM       0      2.34      0.00      0.18      0.00      0.10     97.37
12:22:01 AM       1      6.08      0.00      0.23      0.00      0.10     93.58

12:22:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
12:23:01 AM     all      4.10      0.00      0.20      0.00      0.09     95.61
12:23:01 AM       0      3.53      0.00      0.22      0.00      0.08     96.17
12:23:01 AM       1      4.66      0.00      0.18      0.00      0.10     95.05
12:24:01 AM     all      0.63      0.00      0.15      0.00      0.08     99.13
12:24:01 AM       0      0.78      0.00      0.12      0.00      0.08     99.01
12:24:01 AM       1      0.48      0.00      0.18      0.00      0.08     99.25
12:25:02 AM     all      1.72      0.01      0.20      0.01      0.08     97.99
12:25:02 AM       0      1.40      0.00      0.20      0.02      0.07     98.31
12:25:02 AM       1      2.03      0.02      0.20      0.00      0.08     97.67
12:26:01 AM     all      4.48      0.00      0.37      0.00      0.10     95.04
12:26:01 AM       0      4.75      0.00      0.34      0.00      0.09     94.83
12:26:01 AM       1      4.22      0.00      0.41      0.00      0.12     95.25
12:27:01 AM     all      4.17      0.00      0.38      0.00      0.08     95.37
12:27:01 AM       0      5.09      0.00      0.44      0.00      0.08     94.39
12:27:01 AM       1      3.25      0.00      0.32      0.00      0.08     96.35
12:28:01 AM     all      4.39      0.00      0.35      0.00      0.10     95.16
12:28:01 AM       0      5.25      0.00      0.43      0.00      0.12     94.20
12:28:01 AM       1      3.53      0.00      0.27      0.00      0.08     96.12
12:29:01 AM     all      4.24      0.00      0.33      0.00      0.10     95.32
12:29:01 AM       0      5.42      0.00      0.37      0.00      0.10     94.11
12:29:01 AM       1      3.06      0.00      0.30      0.00      0.10     96.53
12:30:01 AM     all      4.29      0.00      0.34      0.00      0.09     95.28
12:30:01 AM       0      5.45      0.00      0.40      0.00      0.10     94.05
12:30:01 AM       1      3.13      0.00      0.28      0.00      0.08     96.50
12:31:01 AM     all      4.16      0.00      0.35      0.01      0.08     95.40
12:31:01 AM       0      4.57      0.00      0.35      0.00      0.08     95.00
12:31:01 AM       1      3.75      0.00      0.35      0.02      0.08     95.80
12:32:01 AM     all      4.55      0.00      0.25      0.00      0.10     95.10
12:32:01 AM       0      4.38      0.00      0.27      0.00      0.08     95.27
12:32:01 AM       1      4.72      0.00      0.23      0.00      0.12     94.93
12:33:01 AM     all      0.59      0.00      0.13      0.00      0.08     99.21
12:33:01 AM       0      0.74      0.00      0.15      0.00      0.08     99.03
12:33:01 AM       1      0.45      0.00      0.10      0.00      0.07     99.38

12:33:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
12:34:01 AM     all      0.63      0.00      0.11      0.00      0.11     99.16
12:34:01 AM       0      0.67      0.00      0.08      0.00      0.08     99.17
12:34:01 AM       1      0.59      0.00      0.13      0.00      0.13     99.15
12:35:01 AM     all      0.51      0.00      0.14      0.00      0.08     99.27
12:35:01 AM       0      0.62      0.00      0.15      0.00      0.10     99.13
12:35:01 AM       1      0.40      0.00      0.13      0.00      0.05     99.41
12:36:01 AM     all      0.51      0.23      0.22      0.11      0.10     98.84
12:36:01 AM       0      0.50      0.05      0.10      0.02      0.08     99.25
12:36:01 AM       1      0.52      0.40      0.34      0.20      0.12     98.42
12:37:01 AM     all      0.54      0.00      0.11      0.01      0.08     99.26
12:37:01 AM       0      0.52      0.00      0.12      0.00      0.08     99.28
12:37:01 AM       1      0.57      0.00      0.10      0.02      0.08     99.23
12:38:01 AM     all      4.71      0.00      0.26      0.00      0.09     94.94
12:38:01 AM       0      5.39      0.00      0.30      0.00      0.08     94.23
12:38:01 AM       1      4.03      0.00      0.22      0.00      0.10     95.65
12:39:01 AM     all      4.22      0.00      0.19      0.00      0.11     95.48
12:39:01 AM       0      5.45      0.00      0.17      0.00      0.12     94.26
12:39:01 AM       1      2.99      0.00      0.22      0.00      0.10     96.69
12:40:01 AM     all      4.17      0.00      0.20      0.00      0.10     95.53
12:40:01 AM       0      4.44      0.00      0.22      0.00      0.10     95.25
12:40:01 AM       1      3.90      0.00      0.18      0.00      0.10     95.82
12:41:01 AM     all      4.26      0.00      0.22      0.00      0.09     95.43
12:41:01 AM       0      4.65      0.00      0.23      0.00      0.10     95.01
12:41:01 AM       1      3.86      0.00      0.20      0.00      0.08     95.85
12:42:01 AM     all      4.28      0.00      0.20      0.00      0.12     95.41
12:42:01 AM       0      6.09      0.00      0.23      0.00      0.13     93.54
12:42:01 AM       1      2.46      0.00      0.17      0.00      0.10     97.27
12:43:01 AM     all      4.26      0.00      0.19      0.00      0.11     95.44
12:43:01 AM       0      2.56      0.00      0.18      0.00      0.10     97.16
12:43:01 AM       1      5.96      0.00      0.20      0.00      0.12     93.72
12:44:01 AM     all      4.89      0.00      0.23      0.00      0.09     94.78
12:44:01 AM       0      5.14      0.00      0.22      0.00      0.10     94.54
12:44:01 AM       1      4.64      0.00      0.25      0.00      0.08     95.03

12:44:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
12:45:01 AM     all      4.15      0.00      0.23      0.00      0.09     95.53
12:45:01 AM       0      4.42      0.00      0.23      0.00      0.08     95.26
12:45:01 AM       1      3.88      0.00      0.22      0.00      0.10     95.80
12:46:01 AM     all      4.18      0.00      0.22      0.00      0.10     95.50
12:46:01 AM       0      2.68      0.00      0.22      0.00      0.10     97.01
12:46:01 AM       1      5.69      0.00      0.22      0.00      0.10     93.99
12:47:01 AM     all      4.30      0.00      0.21      0.00      0.10     95.39
12:47:01 AM       0      2.64      0.00      0.18      0.00      0.10     97.07
12:47:01 AM       1      5.96      0.00      0.23      0.00      0.10     93.71
12:48:01 AM     all      4.23      0.00      0.23      0.00      0.10     95.43
12:48:01 AM       0      4.40      0.00      0.20      0.00      0.08     95.32
12:48:01 AM       1      4.06      0.00      0.27      0.00      0.12     95.55
12:49:01 AM     all      4.24      0.00      0.21      0.00      0.11     95.44
12:49:01 AM       0      2.91      0.00      0.20      0.00      0.10     96.79
12:49:01 AM       1      5.57      0.00      0.22      0.00      0.12     94.09
12:50:01 AM     all      5.64      0.00      0.39      0.00      0.10     93.87
12:50:01 AM       0      6.08      0.00      0.39      0.00      0.12     93.41
12:50:01 AM       1      5.19      0.00      0.40      0.00      0.08     94.33
12:51:01 AM     all      4.27      0.00      0.37      0.00      0.09     95.27
12:51:01 AM       0      3.66      0.00      0.38      0.00      0.08     95.87
12:51:01 AM       1      4.87      0.00      0.35      0.00      0.10     94.68
12:52:01 AM     all      4.36      0.00      0.39      0.00      0.10     95.15
12:52:01 AM       0      5.54      0.00      0.47      0.00      0.10     93.89
12:52:01 AM       1      3.18      0.00      0.32      0.00      0.10     96.41
12:53:01 AM     all      4.17      0.00      0.35      0.00      0.10     95.38
12:53:01 AM       0      3.40      0.00      0.32      0.00      0.08     96.20
12:53:01 AM       1      4.94      0.00      0.39      0.00      0.12     94.56
12:54:01 AM     all      4.31      0.00      0.36      0.00      0.11     95.22
12:54:01 AM       0      6.23      0.00      0.47      0.00      0.12     93.18
12:54:01 AM       1      2.39      0.00      0.25      0.00      0.10     97.26
12:55:01 AM     all      4.36      0.00      0.35      0.01      0.10     95.18
12:55:01 AM       0      7.15      0.00      0.50      0.00      0.12     92.23
12:55:01 AM       1      1.59      0.00      0.20      0.02      0.08     98.11

12:55:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
12:56:01 AM     all      6.61      0.00      0.43      0.00      0.09     92.87
12:56:01 AM       0      8.43      0.00      0.49      0.00      0.08     91.01
12:56:01 AM       1      4.80      0.00      0.37      0.00      0.10     94.73
12:57:01 AM     all      0.62      0.00      0.11      0.00      0.10     99.17
12:57:01 AM       0      0.65      0.00      0.12      0.00      0.08     99.15
12:57:01 AM       1      0.59      0.00      0.10      0.00      0.12     99.20
12:58:01 AM     all      0.61      0.00      0.15      0.00      0.09     99.15
12:58:01 AM       0      0.72      0.00      0.13      0.00      0.07     99.08
12:58:01 AM       1      0.50      0.00      0.17      0.00      0.12     99.21
12:59:01 AM     all      0.74      0.00      0.16      0.00      0.08     99.01
12:59:01 AM       0      0.74      0.00      0.12      0.00      0.08     99.06
12:59:01 AM       1      0.75      0.00      0.20      0.00      0.08     98.96
01:00:01 AM     all      0.70      0.00      0.16      0.00      0.07     99.07
01:00:01 AM       0      0.85      0.00      0.15      0.00      0.07     98.93
01:00:01 AM       1      0.55      0.00      0.17      0.00      0.07     99.21
01:01:01 AM     all      0.60      0.00      0.17      0.00      0.08     99.15
01:01:01 AM       0      0.70      0.00      0.15      0.00      0.08     99.06
01:01:01 AM       1      0.50      0.00      0.18      0.00      0.08     99.23
01:02:01 AM     all      2.96      0.00      0.23      0.00      0.08     96.73
01:02:01 AM       0      3.04      0.00      0.22      0.00      0.08     96.65