Started by upstream project "integration-distribution-mri-test-calcium" build number 44
originally caused by:
 Started by timer
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on prd-centos8-robot-2c-8g-20286 (centos8-robot-2c-8g) in workspace /w/workspace/controller-csit-3node-clustering-tell-all-calcium
[ssh-agent] Looking for ssh-agent implementation...
[ssh-agent]   Exec ssh-agent (binary ssh-agent on a remote machine)
$ ssh-agent
SSH_AUTH_SOCK=/tmp/ssh-phC9qyIzpDSz/agent.5273
SSH_AGENT_PID=5274
[ssh-agent] Started.
Running ssh-add (command line suppressed)
Identity added: /w/workspace/controller-csit-3node-clustering-tell-all-calcium@tmp/private_key_10995840299288867311.key (/w/workspace/controller-csit-3node-clustering-tell-all-calcium@tmp/private_key_10995840299288867311.key)
[ssh-agent] Using credentials jenkins (Release Engineering Jenkins Key)
The recommended git tool is: NONE
using credential opendaylight-jenkins-ssh
Wiping out workspace first.
Cloning the remote Git repository
Cloning repository git://devvexx.opendaylight.org/mirror/integration/test
 > git init /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test # timeout=10
Fetching upstream changes from git://devvexx.opendaylight.org/mirror/integration/test
 > git --version # timeout=10
 > git --version # 'git version 2.43.0'
using GIT_SSH to set credentials Release Engineering Jenkins Key
[INFO] Currently running in a labeled security context
[INFO] Currently SELinux is 'enforcing' on the host
 > /usr/bin/chcon --type=ssh_home_t /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test@tmp/jenkins-gitclient-ssh13395542544196139500.key
Verifying host key using known hosts file
You're using 'Known hosts file' strategy to verify ssh host keys, but your known_hosts file does not exist, please go to 'Manage Jenkins' -> 'Security' -> 'Git Host Key Verification Configuration' and configure host key verification.
 > git fetch --tags --force --progress -- git://devvexx.opendaylight.org/mirror/integration/test +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url git://devvexx.opendaylight.org/mirror/integration/test # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url git://devvexx.opendaylight.org/mirror/integration/test # timeout=10
Fetching upstream changes from git://devvexx.opendaylight.org/mirror/integration/test
using GIT_SSH to set credentials Release Engineering Jenkins Key
[INFO] Currently running in a labeled security context
[INFO] Currently SELinux is 'enforcing' on the host
 > /usr/bin/chcon --type=ssh_home_t /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test@tmp/jenkins-gitclient-ssh6633496634117750894.key
Verifying host key using known hosts file
You're using 'Known hosts file' strategy to verify ssh host keys, but your known_hosts file does not exist, please go to 'Manage Jenkins' -> 'Security' -> 'Git Host Key Verification Configuration' and configure host key verification.
 > git fetch --tags --force --progress -- git://devvexx.opendaylight.org/mirror/integration/test master # timeout=10
 > git rev-parse FETCH_HEAD^{commit} # timeout=10
Checking out Revision b9619182128fdbcc633129519168ca8029edb080 (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f b9619182128fdbcc633129519168ca8029edb080 # timeout=10
Commit message: "Define scandium stream"
 > git rev-parse FETCH_HEAD^{commit} # timeout=10
 > git rev-list --no-walk 84e81edbe867428cee8930673872d4d2bdf3942f # timeout=10
No emails were triggered.
provisioning config files...
copy managed file [npmrc] to file:/home/jenkins/.npmrc
copy managed file [pipconf] to file:/home/jenkins/.config/pip/pip.conf
copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins17824482622150059100.sh
---> python-tools-install.sh
Setup pyenv:
  system
* 3.8.13 (set by /opt/pyenv/version)
* 3.9.13 (set by /opt/pyenv/version)
* 3.10.13 (set by /opt/pyenv/version)
* 3.11.7 (set by /opt/pyenv/version)
lf-activate-venv(): INFO: Creating python3 venv at /tmp/venv-2KoL
lf-activate-venv(): INFO: Save venv in file: /tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing: lftools 
lf-activate-venv(): INFO: Adding /tmp/venv-2KoL/bin to PATH
Generating Requirements File
Python 3.11.7
pip 24.1.2 from /tmp/venv-2KoL/lib/python3.11/site-packages/pip (python 3.11)
appdirs==1.4.4
argcomplete==3.4.0
aspy.yaml==1.3.0
attrs==23.2.0
autopage==0.5.2
beautifulsoup4==4.12.3
boto3==1.34.144
botocore==1.34.144
bs4==0.0.2
cachetools==5.3.3
certifi==2024.7.4
cffi==1.16.0
cfgv==3.4.0
chardet==5.2.0
charset-normalizer==3.3.2
click==8.1.7
cliff==4.7.0
cmd2==2.4.3
cryptography==3.3.2
debtcollector==3.0.0
decorator==5.1.1
defusedxml==0.7.1
Deprecated==1.2.14
distlib==0.3.8
dnspython==2.6.1
docker==4.2.2
dogpile.cache==1.3.3
email_validator==2.2.0
filelock==3.15.4
future==1.0.0
gitdb==4.0.11
GitPython==3.1.43
google-auth==2.32.0
httplib2==0.22.0
identify==2.6.0
idna==3.7
importlib-resources==1.5.0
iso8601==2.1.0
Jinja2==3.1.4
jmespath==1.0.1
jsonpatch==1.33
jsonpointer==3.0.0
jsonschema==4.23.0
jsonschema-specifications==2023.12.1
keystoneauth1==5.7.0
kubernetes==30.1.0
lftools==0.37.10
lxml==5.2.2
MarkupSafe==2.1.5
msgpack==1.0.8
multi_key_dict==2.0.3
munch==4.0.0
netaddr==1.3.0
netifaces==0.11.0
niet==1.4.2
nodeenv==1.9.1
oauth2client==4.1.3
oauthlib==3.2.2
openstacksdk==3.2.0
os-client-config==2.1.0
os-service-types==1.7.0
osc-lib==3.1.0
oslo.config==9.5.0
oslo.context==5.5.0
oslo.i18n==6.3.0
oslo.log==6.1.0
oslo.serialization==5.4.0
oslo.utils==7.2.0
packaging==24.1
pbr==6.0.0
platformdirs==4.2.2
prettytable==3.10.2
pyasn1==0.6.0
pyasn1_modules==0.4.0
pycparser==2.22
pygerrit2==2.0.15
PyGithub==2.3.0
PyJWT==2.8.0
PyNaCl==1.5.0
pyparsing==2.4.7
pyperclip==1.9.0
pyrsistent==0.20.0
python-cinderclient==9.5.0
python-dateutil==2.9.0.post0
python-heatclient==3.5.0
python-jenkins==1.8.2
python-keystoneclient==5.4.0
python-magnumclient==4.6.0
python-novaclient==18.6.0
python-openstackclient==6.6.0
python-swiftclient==4.6.0
PyYAML==6.0.1
referencing==0.35.1
requests==2.32.3
requests-oauthlib==2.0.0
requestsexceptions==1.4.0
rfc3986==2.0.0
rpds-py==0.19.0
rsa==4.9
ruamel.yaml==0.18.6
ruamel.yaml.clib==0.2.8
s3transfer==0.10.2
simplejson==3.19.2
six==1.16.0
smmap==5.0.1
soupsieve==2.5
stevedore==5.2.0
tabulate==0.9.0
toml==0.10.2
tomlkit==0.13.0
tqdm==4.66.4
typing_extensions==4.12.2
tzdata==2024.1
urllib3==1.26.19
virtualenv==20.26.3
wcwidth==0.2.13
websocket-client==1.8.0
wrapt==1.16.0
xdg==6.0.0
xmltodict==0.13.0
yq==3.4.3
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties content 
OS_STACK_TEMPLATE=csit-2-instance-type.yaml
OS_CLOUD=vex
OS_STACK_NAME=releng-controller-csit-3node-clustering-tell-all-calcium-44
OS_STACK_TEMPLATE_DIR=openstack-hot

[EnvInject] - Variables injected successfully.
provisioning config files...
copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins4044217430012837611.sh
---> Create parameters file for OpenStack HOT
OpenStack Heat parameters generated
-----------------------------------
parameters:
    vm_0_count: '3'
    vm_0_flavor: 'v3-standard-4'
    vm_0_image: 'ZZCI - CentOS Stream 8 - builder - x86_64 - 20240601-160217.263'
    vm_1_count: '0'
    vm_1_flavor: 'v3-standard-2'
    vm_1_image: 'ZZCI - Ubuntu 18.04 - mininet-ovs-28 - x86_64 - 20230601-180106.003'
    
    job_name: '28865-44'
    silo: 'releng'
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash -l /tmp/jenkins13926918842065421956.sh
---> Create HEAT stack
+ source /home/jenkins/lf-env.sh
+ lf-activate-venv --python python3 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
++ mktemp -d /tmp/venv-XXXX
+ lf_venv=/tmp/venv-rrBP
+ local venv_file=/tmp/.os_lf_venv
+ local python=python3
+ local options
+ local set_path=true
+ local install_args=
++ getopt -o np:v: -l no-path,system-site-packages,python:,venv-file: -n lf-activate-venv -- --python python3 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
+ options=' --python '\''python3'\'' -- '\''lftools[openstack]'\'' '\''kubernetes'\'' '\''niet'\'' '\''python-heatclient'\'' '\''python-openstackclient'\'' '\''python-magnumclient'\'' '\''yq'\'''
+ eval set -- ' --python '\''python3'\'' -- '\''lftools[openstack]'\'' '\''kubernetes'\'' '\''niet'\'' '\''python-heatclient'\'' '\''python-openstackclient'\'' '\''python-magnumclient'\'' '\''yq'\'''
++ set -- --python python3 -- 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
+ true
+ case $1 in
+ python=python3
+ shift 2
+ true
+ case $1 in
+ shift
+ break
+ case $python in
+ local pkg_list=
+ [[ -d /opt/pyenv ]]
+ echo 'Setup pyenv:'
Setup pyenv:
+ export PYENV_ROOT=/opt/pyenv
+ PYENV_ROOT=/opt/pyenv
+ export PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
+ PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
+ pyenv versions
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-tell-all-calcium/.python-version)
+ command -v pyenv
++ pyenv init - --no-rehash
+ eval 'PATH="$(bash --norc -ec '\''IFS=:; paths=($PATH); 
for i in ${!paths[@]}; do 
if [[ ${paths[i]} == "'\'''\''/opt/pyenv/shims'\'''\''" ]]; then unset '\''\'\'''\''paths[i]'\''\'\'''\''; 
fi; done; 
echo "${paths[*]}"'\'')"
export PATH="/opt/pyenv/shims:${PATH}"
export PYENV_SHELL=bash
source '\''/opt/pyenv/libexec/../completions/pyenv.bash'\''
pyenv() {
  local command
  command="${1:-}"
  if [ "$#" -gt 0 ]; then
    shift
  fi

  case "$command" in
  rehash|shell)
    eval "$(pyenv "sh-$command" "$@")"
    ;;
  *)
    command pyenv "$command" "$@"
    ;;
  esac
}'
+++ bash --norc -ec 'IFS=:; paths=($PATH); 
for i in ${!paths[@]}; do 
if [[ ${paths[i]} == "/opt/pyenv/shims" ]]; then unset '\''paths[i]'\''; 
fi; done; 
echo "${paths[*]}"'
++ PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
++ export PATH=/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
++ PATH=/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
++ export PYENV_SHELL=bash
++ PYENV_SHELL=bash
++ source /opt/pyenv/libexec/../completions/pyenv.bash
+++ complete -F _pyenv pyenv
++ lf-pyver python3
++ local py_version_xy=python3
++ local py_version_xyz=
++ pyenv versions
++ local command
++ command=versions
++ '[' 1 -gt 0 ']'
++ shift
++ case "$command" in
++ command pyenv versions
++ pyenv versions
++ sed 's/^[ *]* //'
++ grep -E '^[0-9.]*[0-9]$'
++ awk '{ print $1 }'
++ [[ ! -s /tmp/.pyenv_versions ]]
+++ grep '^3' /tmp/.pyenv_versions
+++ tail -n 1
+++ sort -V
++ py_version_xyz=3.11.7
++ [[ -z 3.11.7 ]]
++ echo 3.11.7
++ return 0
+ pyenv local 3.11.7
+ local command
+ command=local
+ '[' 2 -gt 0 ']'
+ shift
+ case "$command" in
+ command pyenv local 3.11.7
+ pyenv local 3.11.7
+ for arg in "$@"
+ case $arg in
+ pkg_list+='lftools[openstack] '
+ for arg in "$@"
+ case $arg in
+ pkg_list+='kubernetes '
+ for arg in "$@"
+ case $arg in
+ pkg_list+='niet '
+ for arg in "$@"
+ case $arg in
+ pkg_list+='python-heatclient '
+ for arg in "$@"
+ case $arg in
+ pkg_list+='python-openstackclient '
+ for arg in "$@"
+ case $arg in
+ pkg_list+='python-magnumclient '
+ for arg in "$@"
+ case $arg in
+ pkg_list+='yq '
+ [[ -f /tmp/.os_lf_venv ]]
++ cat /tmp/.os_lf_venv
+ lf_venv=/tmp/venv-2KoL
+ echo 'lf-activate-venv(): INFO: Reuse venv:/tmp/venv-2KoL from' file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-2KoL from file:/tmp/.os_lf_venv
+ /tmp/venv-2KoL/bin/python3 -m pip install --upgrade --quiet pip virtualenv
+ [[ -z lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq  ]]
+ echo 'lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq '
lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq 
+ /tmp/venv-2KoL/bin/python3 -m pip install --upgrade --quiet --upgrade-strategy eager 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq
+ type python3
+ true
+ echo 'lf-activate-venv(): INFO: Adding /tmp/venv-2KoL/bin to PATH'
lf-activate-venv(): INFO: Adding /tmp/venv-2KoL/bin to PATH
+ PATH=/tmp/venv-2KoL/bin:/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin
+ return 0
+ openstack --os-cloud vex limits show --absolute
+--------------------------+---------+
| Name                     |   Value |
+--------------------------+---------+
| maxTotalInstances        |      -1 |
| maxTotalCores            |     450 |
| maxTotalRAMSize          | 1000000 |
| maxServerMeta            |     128 |
| maxImageMeta             |     128 |
| maxPersonality           |       5 |
| maxPersonalitySize       |   10240 |
| maxTotalKeypairs         |     100 |
| maxServerGroups          |      10 |
| maxServerGroupMembers    |      10 |
| maxTotalFloatingIps      |      -1 |
| maxSecurityGroups        |      -1 |
| maxSecurityGroupRules    |      -1 |
| totalRAMUsed             |  729088 |
| totalCoresUsed           |     178 |
| totalInstancesUsed       |      59 |
| totalFloatingIpsUsed     |       0 |
| totalSecurityGroupsUsed  |       0 |
| totalServerGroupsUsed    |       0 |
| maxTotalVolumes          |      -1 |
| maxTotalSnapshots        |      10 |
| maxTotalVolumeGigabytes  |    4096 |
| maxTotalBackups          |      10 |
| maxTotalBackupGigabytes  |    1000 |
| totalVolumesUsed         |       1 |
| totalGigabytesUsed       |      20 |
| totalSnapshotsUsed       |       0 |
| totalBackupsUsed         |       0 |
| totalBackupGigabytesUsed |       0 |
+--------------------------+---------+
+ pushd /opt/ciman/openstack-hot
/opt/ciman/openstack-hot /w/workspace/controller-csit-3node-clustering-tell-all-calcium
+ lftools openstack --os-cloud vex stack create releng-controller-csit-3node-clustering-tell-all-calcium-44 csit-2-instance-type.yaml /w/workspace/controller-csit-3node-clustering-tell-all-calcium/stack-parameters.yaml
Creating stack releng-controller-csit-3node-clustering-tell-all-calcium-44
Waiting to initialize infrastructure...
Stack initialization successful.
------------------------------------
Stack Details
------------------------------------
{'added': None,
 'capabilities': [],
 'created_at': '2024-07-13T23:38:29Z',
 'deleted': None,
 'deleted_at': None,
 'description': 'No description',
 'environment': None,
 'environment_files': None,
 'files': None,
 'files_container': None,
 'id': 'fefa9cf2-c02d-45aa-a05c-5faabd875153',
 'is_rollback_disabled': True,
 'links': [{'href': 'https://orchestration.public.mtl1.vexxhost.net/v1/12c36e260d8e4bb2913965203b1b491f/stacks/releng-controller-csit-3node-clustering-tell-all-calcium-44/fefa9cf2-c02d-45aa-a05c-5faabd875153',
            'rel': 'self'}],
 'location': Munch({'cloud': 'vex', 'region_name': 'ca-ymq-1', 'zone': None, 'project': Munch({'id': '12c36e260d8e4bb2913965203b1b491f', 'name': '61975f2c-7c17-4d69-82fa-c3ae420ad6fd', 'domain_id': None, 'domain_name': 'Default'})}),
 'name': 'releng-controller-csit-3node-clustering-tell-all-calcium-44',
 'notification_topics': [],
 'outputs': [{'description': 'IP addresses of the 1st vm types',
              'output_key': 'vm_0_ips',
              'output_value': ['10.30.171.141',
                               '10.30.170.225',
                               '10.30.170.154']},
             {'description': 'IP addresses of the 2nd vm types',
              'output_key': 'vm_1_ips',
              'output_value': []}],
 'owner_id': ****,
 'parameters': {'OS::project_id': '12c36e260d8e4bb2913965203b1b491f',
                'OS::stack_id': 'fefa9cf2-c02d-45aa-a05c-5faabd875153',
                'OS::stack_name': 'releng-controller-csit-3node-clustering-tell-all-calcium-44',
                'job_name': '28865-44',
                'silo': 'releng',
                'vm_0_count': '3',
                'vm_0_flavor': 'v3-standard-4',
                'vm_0_image': 'ZZCI - CentOS Stream 8 - builder - x86_64 - '
                              '20240601-160217.263',
                'vm_1_count': '0',
                'vm_1_flavor': 'v3-standard-2',
                'vm_1_image': 'ZZCI - Ubuntu 18.04 - mininet-ovs-28 - x86_64 - '
                              '20230601-180106.003'},
 'parent_id': None,
 'replaced': None,
 'status': 'CREATE_COMPLETE',
 'status_reason': 'Stack CREATE completed successfully',
 'tags': [],
 'template': None,
 'template_description': 'No description',
 'template_url': None,
 'timeout_mins': 15,
 'unchanged': None,
 'updated': None,
 'updated_at': None,
 'user_project_id': '9a3328f333ba4f13a45e79b514cf6c3a'}
------------------------------------
+ popd
/w/workspace/controller-csit-3node-clustering-tell-all-calcium
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash -l /tmp/jenkins780105924983186699.sh
---> Copy SSH public keys to CSIT lab
Setup pyenv:
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-tell-all-calcium/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-2KoL from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes python-heatclient python-openstackclient 
lf-activate-venv(): INFO: Adding /tmp/venv-2KoL/bin to PATH
SSH not responding on 10.30.170.154. Retrying in 10 seconds...
SSH not responding on 10.30.171.141. Retrying in 10 seconds...
SSH not responding on 10.30.170.225. Retrying in 10 seconds...
Ping to 10.30.171.141 successful.
Ping to 10.30.170.154 successful.
Ping to 10.30.170.225 successful.
SSH not responding on 10.30.170.225. Retrying in 10 seconds...
SSH not responding on 10.30.171.141. Retrying in 10 seconds...
SSH not responding on 10.30.170.154. Retrying in 10 seconds...
Ping to 10.30.170.225 successful.
Ping to 10.30.171.141 successful.
Ping to 10.30.170.154 successful.
SSH not responding on 10.30.170.225. Retrying in 10 seconds...
SSH not responding on 10.30.171.141. Retrying in 10 seconds...
SSH not responding on 10.30.170.154. Retrying in 10 seconds...
Ping to 10.30.170.225 successful.
Ping to 10.30.171.141 successful.
Ping to 10.30.170.154 successful.
SSH not responding on 10.30.171.141. Retrying in 10 seconds...
SSH not responding on 10.30.170.154. Retrying in 10 seconds...
Warning: Permanently added '10.30.170.225' (ECDSA) to the list of known hosts.
10.30.170.225 releng-28865-44-0-builder-1.novalocal
Successfully copied public keys to slave 10.30.170.225
Ping to 10.30.171.141 successful.
Ping to 10.30.170.154 successful.
Warning: Permanently added '10.30.171.141' (ECDSA) to the list of known hosts.
Warning: Permanently added '10.30.170.154' (ECDSA) to the list of known hosts.
10.30.171.141 releng-28865-44-0-builder-0.novalocal
Successfully copied public keys to slave 10.30.171.141
Process 6664 ready.
Process 6665 ready.
10.30.170.154 releng-28865-44-0-builder-2.novalocal
Successfully copied public keys to slave 10.30.170.154
Process 6666 ready.
SSH ready on all stack servers.
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash -l /tmp/jenkins8537530255404053842.sh
Setup pyenv:
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-tell-all-calcium/.python-version)
lf-activate-venv(): INFO: Creating python3 venv at /tmp/venv-X5kZ
lf-activate-venv(): INFO: Save venv in file: /w/workspace/controller-csit-3node-clustering-tell-all-calcium/.robot_venv
lf-activate-venv(): INFO: Installing: setuptools wheel 
lf-activate-venv(): INFO: Adding /tmp/venv-X5kZ/bin to PATH
+ echo 'Installing Python Requirements'
Installing Python Requirements
+ cat
+ python -m pip install -r requirements.txt
Looking in indexes: https://nexus3.opendaylight.org/repository/PyPi/simple
Collecting docker-py (from -r requirements.txt (line 1))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/docker-py/1.10.6/docker_py-1.10.6-py2.py3-none-any.whl (50 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 50.0/50.0 kB 7.5 MB/s eta 0:00:00
Collecting ipaddr (from -r requirements.txt (line 2))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/ipaddr/2.2.0/ipaddr-2.2.0.tar.gz (26 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting netaddr (from -r requirements.txt (line 3))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/netaddr/1.3.0/netaddr-1.3.0-py3-none-any.whl (2.3 MB)
Collecting netifaces (from -r requirements.txt (line 4))
  Using cached netifaces-0.11.0-cp311-cp311-linux_x86_64.whl
Collecting pyhocon (from -r requirements.txt (line 5))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/pyhocon/0.3.61/pyhocon-0.3.61-py3-none-any.whl (25 kB)
Collecting requests (from -r requirements.txt (line 6))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/requests/2.32.3/requests-2.32.3-py3-none-any.whl (64 kB)
Collecting robotframework (from -r requirements.txt (line 7))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework/7.0.1/robotframework-7.0.1-py3-none-any.whl (727 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 727.6/727.6 kB 7.3 MB/s eta 0:00:00
Collecting robotframework-httplibrary (from -r requirements.txt (line 8))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-httplibrary/0.4.2/robotframework-httplibrary-0.4.2.tar.gz (9.1 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting robotframework-requests==0.9.7 (from -r requirements.txt (line 9))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-requests/0.9.7/robotframework_requests-0.9.7-py3-none-any.whl (21 kB)
Collecting robotframework-selenium2library (from -r requirements.txt (line 10))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-selenium2library/3.0.0/robotframework_selenium2library-3.0.0-py2.py3-none-any.whl (6.2 kB)
Collecting robotframework-sshlibrary==3.8.0 (from -r requirements.txt (line 11))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-sshlibrary/3.8.0/robotframework-sshlibrary-3.8.0.tar.gz (51 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 51.4/51.4 kB 7.8 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting scapy (from -r requirements.txt (line 12))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/scapy/2.5.0/scapy-2.5.0.tar.gz (1.3 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 16.9 MB/s eta 0:00:00
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting jsonpath-rw (from -r requirements.txt (line 15))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/jsonpath-rw/1.4.0/jsonpath-rw-1.4.0.tar.gz (13 kB)
  Preparing metadata (setup.py): started
  Preparing metadata (setup.py): finished with status 'done'
Collecting elasticsearch (from -r requirements.txt (line 18))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elasticsearch/8.14.0/elasticsearch-8.14.0-py3-none-any.whl (480 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 480.2/480.2 kB 21.4 MB/s eta 0:00:00
Collecting elasticsearch-dsl (from -r requirements.txt (line 19))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elasticsearch-dsl/8.14.0/elasticsearch_dsl-8.14.0-py3-none-any.whl (92 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 92.3/92.3 kB 1.5 MB/s eta 0:00:00
Collecting pyangbind (from -r requirements.txt (line 22))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/pyangbind/0.8.5/pyangbind-0.8.5-py3-none-any.whl (52 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 52.9/52.9 kB 11.7 MB/s eta 0:00:00
Collecting isodate (from -r requirements.txt (line 25))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/isodate/0.6.1/isodate-0.6.1-py2.py3-none-any.whl (41 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.7/41.7 kB 6.0 MB/s eta 0:00:00
Collecting jmespath (from -r requirements.txt (line 28))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/jmespath/1.0.1/jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting jsonpatch (from -r requirements.txt (line 31))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/jsonpatch/1.33/jsonpatch-1.33-py2.py3-none-any.whl (12 kB)
Collecting paramiko>=1.15.3 (from robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/paramiko/3.4.0/paramiko-3.4.0-py3-none-any.whl (225 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 225.9/225.9 kB 29.0 MB/s eta 0:00:00
Collecting scp>=0.13.0 (from robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/scp/0.15.0/scp-0.15.0-py2.py3-none-any.whl (8.8 kB)
Collecting docker-pycreds>=0.2.1 (from docker-py->-r requirements.txt (line 1))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/docker-pycreds/0.4.0/docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB)
Collecting six>=1.4.0 (from docker-py->-r requirements.txt (line 1))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/six/1.16.0/six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting websocket-client>=0.32.0 (from docker-py->-r requirements.txt (line 1))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/websocket-client/1.8.0/websocket_client-1.8.0-py3-none-any.whl (58 kB)
Collecting pyparsing<4,>=2 (from pyhocon->-r requirements.txt (line 5))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/pyparsing/3.1.2/pyparsing-3.1.2-py3-none-any.whl (103 kB)
Collecting charset-normalizer<4,>=2 (from requests->-r requirements.txt (line 6))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/charset-normalizer/3.3.2/charset_normalizer-3.3.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (140 kB)
Collecting idna<4,>=2.5 (from requests->-r requirements.txt (line 6))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/idna/3.7/idna-3.7-py3-none-any.whl (66 kB)
Collecting urllib3<3,>=1.21.1 (from requests->-r requirements.txt (line 6))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/urllib3/2.2.2/urllib3-2.2.2-py3-none-any.whl (121 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 121.4/121.4 kB 14.0 MB/s eta 0:00:00
Collecting certifi>=2017.4.17 (from requests->-r requirements.txt (line 6))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/certifi/2024.7.4/certifi-2024.7.4-py3-none-any.whl (162 kB)
Collecting webtest>=2.0 (from robotframework-httplibrary->-r requirements.txt (line 8))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/webtest/3.0.0/WebTest-3.0.0-py3-none-any.whl (31 kB)
Collecting jsonpointer (from robotframework-httplibrary->-r requirements.txt (line 8))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/jsonpointer/3.0.0/jsonpointer-3.0.0-py2.py3-none-any.whl (7.6 kB)
Collecting robotframework-seleniumlibrary>=3.0.0 (from robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-seleniumlibrary/6.5.0/robotframework_seleniumlibrary-6.5.0-py2.py3-none-any.whl (104 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 104.6/104.6 kB 15.3 MB/s eta 0:00:00
Collecting ply (from jsonpath-rw->-r requirements.txt (line 15))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/ply/3.11/ply-3.11-py2.py3-none-any.whl (49 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 49.6/49.6 kB 7.5 MB/s eta 0:00:00
Collecting decorator (from jsonpath-rw->-r requirements.txt (line 15))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/decorator/5.1.1/decorator-5.1.1-py3-none-any.whl (9.1 kB)
Collecting elastic-transport<9,>=8.13 (from elasticsearch->-r requirements.txt (line 18))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elastic-transport/8.13.1/elastic_transport-8.13.1-py3-none-any.whl (64 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.5/64.5 kB 6.5 MB/s eta 0:00:00
Collecting python-dateutil (from elasticsearch-dsl->-r requirements.txt (line 19))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/python-dateutil/2.9.0.post0/python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Collecting typing-extensions (from elasticsearch-dsl->-r requirements.txt (line 19))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/typing-extensions/4.12.2/typing_extensions-4.12.2-py3-none-any.whl (37 kB)
Collecting pyang (from pyangbind->-r requirements.txt (line 22))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/pyang/2.6.1/pyang-2.6.1-py2.py3-none-any.whl (594 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 594.7/594.7 kB 15.5 MB/s eta 0:00:00
Collecting lxml (from pyangbind->-r requirements.txt (line 22))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/lxml/5.2.2/lxml-5.2.2-cp311-cp311-manylinux_2_28_x86_64.whl (5.0 MB)
Collecting regex (from pyangbind->-r requirements.txt (line 22))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/regex/2024.5.15/regex-2024.5.15-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (785 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 785.0/785.0 kB 17.2 MB/s eta 0:00:00
Collecting enum34 (from pyangbind->-r requirements.txt (line 22))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/enum34/1.1.10/enum34-1.1.10-py3-none-any.whl (11 kB)
Collecting bcrypt>=3.2 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/bcrypt/4.1.3/bcrypt-4.1.3-cp39-abi3-manylinux_2_28_x86_64.whl (283 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 283.7/283.7 kB 20.4 MB/s eta 0:00:00
Collecting cryptography>=3.3 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/cryptography/42.0.8/cryptography-42.0.8-cp39-abi3-manylinux_2_28_x86_64.whl (3.9 MB)
Collecting pynacl>=1.5 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/pynacl/1.5.0/PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (856 kB)
Collecting selenium>=4.3.0 (from robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/selenium/4.22.0/selenium-4.22.0-py3-none-any.whl (9.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.4/9.4 MB 47.3 MB/s eta 0:00:00
Collecting robotframework-pythonlibcore>=4.4.1 (from robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-pythonlibcore/4.4.1/robotframework_pythonlibcore-4.4.1-py2.py3-none-any.whl (12 kB)
Collecting click>=8.1.7 (from robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/click/8.1.7/click-8.1.7-py3-none-any.whl (97 kB)
Collecting WebOb>=1.2 (from webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/webob/1.8.7/WebOb-1.8.7-py2.py3-none-any.whl (114 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 115.0/115.0 kB 17.8 MB/s eta 0:00:00
Collecting waitress>=0.8.5 (from webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/waitress/3.0.0/waitress-3.0.0-py3-none-any.whl (56 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 56.7/56.7 kB 15.8 MB/s eta 0:00:00
Collecting beautifulsoup4 (from webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/beautifulsoup4/4.12.3/beautifulsoup4-4.12.3-py3-none-any.whl (147 kB)
Collecting cffi>=1.12 (from cryptography>=3.3->paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/cffi/1.16.0/cffi-1.16.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (464 kB)
Collecting trio~=0.17 (from selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/trio/0.26.0/trio-0.26.0-py3-none-any.whl (475 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 475.7/475.7 kB 12.2 MB/s eta 0:00:00
Collecting trio-websocket~=0.9 (from selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/trio-websocket/0.11.1/trio_websocket-0.11.1-py3-none-any.whl (17 kB)
Collecting soupsieve>1.2 (from beautifulsoup4->webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/soupsieve/2.5/soupsieve-2.5-py3-none-any.whl (36 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=3.3->paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/pycparser/2.22/pycparser-2.22-py3-none-any.whl (117 kB)
Collecting attrs>=23.2.0 (from trio~=0.17->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/attrs/23.2.0/attrs-23.2.0-py3-none-any.whl (60 kB)
Collecting sortedcontainers (from trio~=0.17->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/sortedcontainers/2.4.0/sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB)
Collecting outcome (from trio~=0.17->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/outcome/1.3.0.post0/outcome-1.3.0.post0-py2.py3-none-any.whl (10 kB)
Collecting sniffio>=1.3.0 (from trio~=0.17->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/sniffio/1.3.1/sniffio-1.3.1-py3-none-any.whl (10 kB)
Collecting wsproto>=0.14 (from trio-websocket~=0.9->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/wsproto/1.2.0/wsproto-1.2.0-py3-none-any.whl (24 kB)
Collecting pysocks!=1.5.7,<2.0,>=1.5.6 (from urllib3[socks]<3,>=1.26->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/pysocks/1.7.1/PySocks-1.7.1-py3-none-any.whl (16 kB)
Collecting h11<1,>=0.9.0 (from wsproto>=0.14->trio-websocket~=0.9->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10))
  Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/h11/0.14.0/h11-0.14.0-py3-none-any.whl (58 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.3/58.3 kB 13.6 MB/s eta 0:00:00
Building wheels for collected packages: robotframework-sshlibrary, ipaddr, robotframework-httplibrary, scapy, jsonpath-rw
  Building wheel for robotframework-sshlibrary (setup.py): started
  Building wheel for robotframework-sshlibrary (setup.py): finished with status 'done'
  Created wheel for robotframework-sshlibrary: filename=robotframework_sshlibrary-3.8.0-py3-none-any.whl size=55116 sha256=3ab38e40f25bd142278bfa8333bc2ff31977df4b8549583747780cec101c5b89
  Stored in directory: /home/jenkins/.cache/pip/wheels/f7/c9/b3/a977b7bcc410d45ae27d240df3d00a12585509180e373ecccc
  Building wheel for ipaddr (setup.py): started
  Building wheel for ipaddr (setup.py): finished with status 'done'
  Created wheel for ipaddr: filename=ipaddr-2.2.0-py3-none-any.whl size=18280 sha256=cb887475fd0d7c0d869e1320c2a84f1f1cb708bbf0b32ad4059074c2a0f8c2cb
  Stored in directory: /home/jenkins/.cache/pip/wheels/dc/6c/04/da2d847fa8d45c59af3e1d83e2acc29cb8adcbaf04c0898dbf
  Building wheel for robotframework-httplibrary (setup.py): started
  Building wheel for robotframework-httplibrary (setup.py): finished with status 'done'
  Created wheel for robotframework-httplibrary: filename=robotframework_httplibrary-0.4.2-py3-none-any.whl size=9954 sha256=f656b7c6737fb6b94ecb241a80b18177b91f98f68bff5c621dd339ea4dd467cf
  Stored in directory: /home/jenkins/.cache/pip/wheels/aa/bc/0d/9a20dd51effef392aae2733cb4c7b66c6fa29fca33d88b57ed
  Building wheel for scapy (setup.py): started
  Building wheel for scapy (setup.py): finished with status 'done'
  Created wheel for scapy: filename=scapy-2.5.0-py2.py3-none-any.whl size=1444328 sha256=22d3b29968fd93dc12b3fc61015159a0f072fc00a35cb3897567c6cc01b38c7e
  Stored in directory: /home/jenkins/.cache/pip/wheels/3b/a7/d6/87eeec31fbfcb2bd8ace7dad8172b6d6ebbe1ec6827dbb389a
  Building wheel for jsonpath-rw (setup.py): started
  Building wheel for jsonpath-rw (setup.py): finished with status 'done'
  Created wheel for jsonpath-rw: filename=jsonpath_rw-1.4.0-py3-none-any.whl size=15127 sha256=1e9b21bf75bb1146c6f27e6e473b59fea7cd7069e0ace78879995eaf95a79417
  Stored in directory: /home/jenkins/.cache/pip/wheels/f1/54/63/9a8da38cefae13755097b36cc852decc25d8ef69c37d58d4eb
Successfully built robotframework-sshlibrary ipaddr robotframework-httplibrary scapy jsonpath-rw
Installing collected packages: sortedcontainers, ply, netifaces, ipaddr, enum34, websocket-client, WebOb, waitress, urllib3, typing-extensions, soupsieve, sniffio, six, scapy, robotframework-pythonlibcore, robotframework, regex, pysocks, pyparsing, pycparser, netaddr, lxml, jsonpointer, jmespath, idna, h11, decorator, click, charset-normalizer, certifi, bcrypt, attrs, wsproto, requests, python-dateutil, pyhocon, pyang, outcome, jsonpath-rw, jsonpatch, isodate, elastic-transport, docker-pycreds, cffi, beautifulsoup4, webtest, trio, robotframework-requests, pynacl, pyangbind, elasticsearch, docker-py, cryptography, trio-websocket, robotframework-httplibrary, paramiko, elasticsearch-dsl, selenium, scp, robotframework-sshlibrary, robotframework-seleniumlibrary, robotframework-selenium2library
Successfully installed WebOb-1.8.7 attrs-23.2.0 bcrypt-4.1.3 beautifulsoup4-4.12.3 certifi-2024.7.4 cffi-1.16.0 charset-normalizer-3.3.2 click-8.1.7 cryptography-42.0.8 decorator-5.1.1 docker-py-1.10.6 docker-pycreds-0.4.0 elastic-transport-8.13.1 elasticsearch-8.14.0 elasticsearch-dsl-8.14.0 enum34-1.1.10 h11-0.14.0 idna-3.7 ipaddr-2.2.0 isodate-0.6.1 jmespath-1.0.1 jsonpatch-1.33 jsonpath-rw-1.4.0 jsonpointer-3.0.0 lxml-5.2.2 netaddr-1.3.0 netifaces-0.11.0 outcome-1.3.0.post0 paramiko-3.4.0 ply-3.11 pyang-2.6.1 pyangbind-0.8.5 pycparser-2.22 pyhocon-0.3.61 pynacl-1.5.0 pyparsing-3.1.2 pysocks-1.7.1 python-dateutil-2.9.0.post0 regex-2024.5.15 requests-2.32.3 robotframework-7.0.1 robotframework-httplibrary-0.4.2 robotframework-pythonlibcore-4.4.1 robotframework-requests-0.9.7 robotframework-selenium2library-3.0.0 robotframework-seleniumlibrary-6.5.0 robotframework-sshlibrary-3.8.0 scapy-2.5.0 scp-0.15.0 selenium-4.22.0 six-1.16.0 sniffio-1.3.1 sortedcontainers-2.4.0 soupsieve-2.5 trio-0.26.0 trio-websocket-0.11.1 typing-extensions-4.12.2 urllib3-2.2.2 waitress-3.0.0 websocket-client-1.8.0 webtest-3.0.0 wsproto-1.2.0
+ pip freeze
attrs==23.2.0
bcrypt==4.1.3
beautifulsoup4==4.12.3
certifi==2024.7.4
cffi==1.16.0
charset-normalizer==3.3.2
click==8.1.7
cryptography==42.0.8
decorator==5.1.1
distlib==0.3.8
docker-py==1.10.6
docker-pycreds==0.4.0
elastic-transport==8.13.1
elasticsearch==8.14.0
elasticsearch-dsl==8.14.0
enum34==1.1.10
filelock==3.15.4
h11==0.14.0
idna==3.7
ipaddr==2.2.0
isodate==0.6.1
jmespath==1.0.1
jsonpatch==1.33
jsonpath-rw==1.4.0
jsonpointer==3.0.0
lxml==5.2.2
netaddr==1.3.0
netifaces==0.11.0
outcome==1.3.0.post0
paramiko==3.4.0
platformdirs==4.2.2
ply==3.11
pyang==2.6.1
pyangbind==0.8.5
pycparser==2.22
pyhocon==0.3.61
PyNaCl==1.5.0
pyparsing==3.1.2
PySocks==1.7.1
python-dateutil==2.9.0.post0
regex==2024.5.15
requests==2.32.3
robotframework==7.0.1
robotframework-httplibrary==0.4.2
robotframework-pythonlibcore==4.4.1
robotframework-requests==0.9.7
robotframework-selenium2library==3.0.0
robotframework-seleniumlibrary==6.5.0
robotframework-sshlibrary==3.8.0
scapy==2.5.0
scp==0.15.0
selenium==4.22.0
six==1.16.0
sniffio==1.3.1
sortedcontainers==2.4.0
soupsieve==2.5
trio==0.26.0
trio-websocket==0.11.1
typing_extensions==4.12.2
urllib3==2.2.2
virtualenv==20.26.3
waitress==3.0.0
WebOb==1.8.7
websocket-client==1.8.0
WebTest==3.0.0
wsproto==1.2.0
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties file path 'env.properties'
[EnvInject] - Variables injected successfully.
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash -l /tmp/jenkins12742403679996812124.sh
Setup pyenv:
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-tell-all-calcium/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-2KoL from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing: python-heatclient python-openstackclient yq 
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
lftools 0.37.10 requires urllib3<2.1.0, but you have urllib3 2.2.2 which is incompatible.
lf-activate-venv(): INFO: Adding /tmp/venv-2KoL/bin to PATH
+ ODL_SYSTEM=()
+ TOOLS_SYSTEM=()
+ OPENSTACK_SYSTEM=()
+ OPENSTACK_CONTROLLERS=()
+ mapfile -t ADDR
++ jq -r '.outputs[] | select(.output_key | match("^vm_[0-9]+_ips$")) | .output_value | .[]'
++ openstack stack show -f json -c outputs releng-controller-csit-3node-clustering-tell-all-calcium-44
+ for i in "${ADDR[@]}"
++ ssh 10.30.171.141 hostname -s
Warning: Permanently added '10.30.171.141' (ECDSA) to the list of known hosts.
+ REMHOST=releng-28865-44-0-builder-0
+ case ${REMHOST} in
+ ODL_SYSTEM=("${ODL_SYSTEM[@]}" "${i}")
+ for i in "${ADDR[@]}"
++ ssh 10.30.170.225 hostname -s
Warning: Permanently added '10.30.170.225' (ECDSA) to the list of known hosts.
+ REMHOST=releng-28865-44-0-builder-1
+ case ${REMHOST} in
+ ODL_SYSTEM=("${ODL_SYSTEM[@]}" "${i}")
+ for i in "${ADDR[@]}"
++ ssh 10.30.170.154 hostname -s
Warning: Permanently added '10.30.170.154' (ECDSA) to the list of known hosts.
+ REMHOST=releng-28865-44-0-builder-2
+ case ${REMHOST} in
+ ODL_SYSTEM=("${ODL_SYSTEM[@]}" "${i}")
+ echo NUM_ODL_SYSTEM=3
+ echo NUM_TOOLS_SYSTEM=0
+ '[' '' == yes ']'
+ NUM_OPENSTACK_SYSTEM=0
+ echo NUM_OPENSTACK_SYSTEM=0
+ '[' 0 -eq 2 ']'
+ echo ODL_SYSTEM_IP=10.30.171.141
++ seq 0 2
+ for i in $(seq 0 $(( ${#ODL_SYSTEM[@]} - 1 )))
+ echo ODL_SYSTEM_1_IP=10.30.171.141
+ for i in $(seq 0 $(( ${#ODL_SYSTEM[@]} - 1 )))
+ echo ODL_SYSTEM_2_IP=10.30.170.225
+ for i in $(seq 0 $(( ${#ODL_SYSTEM[@]} - 1 )))
+ echo ODL_SYSTEM_3_IP=10.30.170.154
+ echo TOOLS_SYSTEM_IP=
++ seq 0 -1
+ openstack_index=0
+ NUM_OPENSTACK_CONTROL_NODES=1
+ echo NUM_OPENSTACK_CONTROL_NODES=1
++ seq 0 0
+ for i in $(seq 0 $((NUM_OPENSTACK_CONTROL_NODES - 1)))
+ echo OPENSTACK_CONTROL_NODE_1_IP=
+ NUM_OPENSTACK_COMPUTE_NODES=-1
+ echo NUM_OPENSTACK_COMPUTE_NODES=-1
+ '[' -1 -ge 2 ']'
++ seq 0 -2
+ NUM_OPENSTACK_HAPROXY_NODES=0
+ echo NUM_OPENSTACK_HAPROXY_NODES=0
++ seq 0 -1
+ echo 'Contents of slave_addresses.txt:'
Contents of slave_addresses.txt:
+ cat slave_addresses.txt
NUM_ODL_SYSTEM=3
NUM_TOOLS_SYSTEM=0
NUM_OPENSTACK_SYSTEM=0
ODL_SYSTEM_IP=10.30.171.141
ODL_SYSTEM_1_IP=10.30.171.141
ODL_SYSTEM_2_IP=10.30.170.225
ODL_SYSTEM_3_IP=10.30.170.154
TOOLS_SYSTEM_IP=
NUM_OPENSTACK_CONTROL_NODES=1
OPENSTACK_CONTROL_NODE_1_IP=
NUM_OPENSTACK_COMPUTE_NODES=-1
NUM_OPENSTACK_HAPROXY_NODES=0
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties file path 'slave_addresses.txt'
[EnvInject] - Variables injected successfully.
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/sh /tmp/jenkins17369894105366633070.sh
Preparing for JRE Version 17
Karaf artifact is karaf
Karaf project is integration
Java home is /usr/lib/jvm/java-17-openjdk
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties file path 'set_variables.env'
[EnvInject] - Variables injected successfully.
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins6992299736637582681.sh
--2024-07-13 23:40:31--  http://git.opendaylight.org/gerrit/gitweb?p=integration/distribution.git;a=blob_plain;f=pom.xml;hb=refs/heads/stable/calcium
Resolving git.opendaylight.org (git.opendaylight.org)... 52.47.161.242, 35.181.223.58, 13.39.230.180, ...
Connecting to git.opendaylight.org (git.opendaylight.org)|52.47.161.242|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2619 (2.6K) [text/plain]
Saving to: ‘pom.xml’

     0K ..                                                    100%  452M=0s

2024-07-13 23:40:31 (452 MB/s) - ‘pom.xml’ saved [2619/2619]

Bundle version is 0.20.2-SNAPSHOT
--2024-07-13 23:40:31--  https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/maven-metadata.xml
Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6
Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1404 (1.4K) [application/xml]
Saving to: ‘maven-metadata.xml’

     0K .                                                     100% 37.8M=0s

2024-07-13 23:40:31 (37.8 MB/s) - ‘maven-metadata.xml’ saved [1404/1404]

<?xml version="1.0" encoding="UTF-8"?>
<metadata modelVersion="1.1.0">
  <groupId>org.opendaylight.integration</groupId>
  <artifactId>karaf</artifactId>
  <version>0.20.2-SNAPSHOT</version>
  <versioning>
    <snapshot>
      <timestamp>20240713.204807</timestamp>
      <buildNumber>83</buildNumber>
    </snapshot>
    <lastUpdated>20240713204807</lastUpdated>
    <snapshotVersions>
      <snapshotVersion>
        <extension>pom</extension>
        <value>0.20.2-20240713.204807-83</value>
        <updated>20240713204807</updated>
      </snapshotVersion>
      <snapshotVersion>
        <extension>tar.gz</extension>
        <value>0.20.2-20240713.204807-83</value>
        <updated>20240713204807</updated>
      </snapshotVersion>
      <snapshotVersion>
        <extension>zip</extension>
        <value>0.20.2-20240713.204807-83</value>
        <updated>20240713204807</updated>
      </snapshotVersion>
      <snapshotVersion>
        <classifier>cyclonedx</classifier>
        <extension>xml</extension>
        <value>0.20.2-20240713.204807-83</value>
        <updated>20240713204807</updated>
      </snapshotVersion>
      <snapshotVersion>
        <classifier>cyclonedx</classifier>
        <extension>json</extension>
        <value>0.20.2-20240713.204807-83</value>
        <updated>20240713204807</updated>
      </snapshotVersion>
    </snapshotVersions>
  </versioning>
</metadata>
Nexus timestamp is 0.20.2-20240713.204807-83
Distribution bundle URL is https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240713.204807-83.zip
Distribution bundle is karaf-0.20.2-20240713.204807-83.zip
Distribution bundle version is 0.20.2-SNAPSHOT
Distribution folder is karaf-0.20.2-SNAPSHOT
Nexus prefix is https://nexus.opendaylight.org
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties file path 'detect_variables.env'
[EnvInject] - Variables injected successfully.
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash -l /tmp/jenkins5819556877894115483.sh
Setup pyenv:
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-tell-all-calcium/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-2KoL from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing: python-heatclient python-openstackclient 
lf-activate-venv(): INFO: Adding /tmp/venv-2KoL/bin to PATH
Copying common-functions.sh to /tmp
Copying common-functions.sh to 10.30.171.141:/tmp
Warning: Permanently added '10.30.171.141' (ECDSA) to the list of known hosts.
Copying common-functions.sh to 10.30.170.225:/tmp
Warning: Permanently added '10.30.170.225' (ECDSA) to the list of known hosts.
Copying common-functions.sh to 10.30.170.154:/tmp
Warning: Permanently added '10.30.170.154' (ECDSA) to the list of known hosts.
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins7826169255366867715.sh
common-functions.sh is being sourced
common-functions environment:
MAVENCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
ACTUALFEATURES: 
FEATURESCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg
CUSTOMPROP: /tmp/karaf-0.20.2-SNAPSHOT/etc/custom.properties
LOGCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
MEMCONF: /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
CONTROLLERMEM: 2048m
AKKACONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf
MODULESCONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf
MODULESHARDSCONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf
SUITES: 

#################################################
##         Configure Cluster and Start         ##
#################################################
ACTUALFEATURES: odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app
SPACE_SEPARATED_FEATURES: odl-integration-compatible-with-all odl-jolokia odl-restconf odl-clustering-test-app
Locating script plan to use...
script plan exists!!!
Changing the script plan path...
# Place the scripts in run order:
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/scripts/car_people_shard.sh
Executing /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/scripts/car_people_shard.sh...
Add car-people shards file
Copy shard config to member-1 with IP address 10.30.171.141
Warning: Permanently added '10.30.171.141' (ECDSA) to the list of known hosts.
Copy shard config to member-2 with IP address 10.30.170.225
Warning: Permanently added '10.30.170.225' (ECDSA) to the list of known hosts.
Copy shard config to member-3 with IP address 10.30.170.154
Warning: Permanently added '10.30.170.154' (ECDSA) to the list of known hosts.
Finished running script plans
Configuring member-1 with IP address 10.30.171.141
Warning: Permanently added '10.30.171.141' (ECDSA) to the list of known hosts.
Warning: Permanently added '10.30.171.141' (ECDSA) to the list of known hosts.
+ source /tmp/common-functions.sh karaf-0.20.2-SNAPSHOT
++ [[ /tmp/common-functions.sh == \/\t\m\p\/\c\o\n\f\i\g\u\r\a\t\i\o\n\-\s\c\r\i\p\t\.\s\h ]]
common-functions.sh is being sourced
++ echo 'common-functions.sh is being sourced'
++ BUNDLEFOLDER=karaf-0.20.2-SNAPSHOT
++ export MAVENCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
++ MAVENCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
++ export FEATURESCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg
++ FEATURESCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg
++ export CUSTOMPROP=/tmp/karaf-0.20.2-SNAPSHOT/etc/custom.properties
++ CUSTOMPROP=/tmp/karaf-0.20.2-SNAPSHOT/etc/custom.properties
++ export LOGCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
++ LOGCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
++ export MEMCONF=/tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
++ MEMCONF=/tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
++ export CONTROLLERMEM=
++ CONTROLLERMEM=
++ export AKKACONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf
++ AKKACONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf
++ export MODULESCONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf
++ MODULESCONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf
++ export MODULESHARDSCONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf
++ MODULESHARDSCONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf
++ print_common_env
++ cat
common-functions environment:
MAVENCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
ACTUALFEATURES: 
FEATURESCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg
CUSTOMPROP: /tmp/karaf-0.20.2-SNAPSHOT/etc/custom.properties
LOGCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
MEMCONF: /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
CONTROLLERMEM: 
AKKACONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf
MODULESCONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf
MODULESHARDSCONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf
SUITES: 

++ SSH='ssh -t -t'
++ extra_services_cntl='     dnsmasq.service     httpd.service     libvirtd.service     openvswitch.service     ovs-vswitchd.service     ovsdb-server.service     rabbitmq-server.service '
++ extra_services_cmp='     libvirtd.service     openvswitch.service     ovs-vswitchd.service     ovsdb-server.service '
Changing to /tmp
Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240713.204807-83.zip
+ echo 'Changing to /tmp'
+ cd /tmp
+ echo 'Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240713.204807-83.zip'
+ wget --progress=dot:mega https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240713.204807-83.zip
--2024-07-13 23:40:41--  https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240713.204807-83.zip
Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6
Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 268186747 (256M) [application/zip]
Saving to: ‘karaf-0.20.2-20240713.204807-83.zip’

     0K ........ ........ ........ ........ ........ ........  1%  101M 3s
  3072K ........ ........ ........ ........ ........ ........  2%  158M 2s
  6144K ........ ........ ........ ........ ........ ........  3%  155M 2s
  9216K ........ ........ ........ ........ ........ ........  4% 94.0M 2s
 12288K ........ ........ ........ ........ ........ ........  5%  171M 2s
 15360K ........ ........ ........ ........ ........ ........  7%  204M 2s
 18432K ........ ........ ........ ........ ........ ........  8%  227M 2s
 21504K ........ ........ ........ ........ ........ ........  9%  224M 2s
 24576K ........ ........ ........ ........ ........ ........ 10%  244M 1s
 27648K ........ ........ ........ ........ ........ ........ 11%  256M 1s
 30720K ........ ........ ........ ........ ........ ........ 12%  252M 1s
 33792K ........ ........ ........ ........ ........ ........ 14%  264M 1s
 36864K ........ ........ ........ ........ ........ ........ 15%  195M 1s
 39936K ........ ........ ........ ........ ........ ........ 16%  384M 1s
 43008K ........ ........ ........ ........ ........ ........ 17%  312M 1s
 46080K ........ ........ ........ ........ ........ ........ 18%  374M 1s
 49152K ........ ........ ........ ........ ........ ........ 19%  326M 1s
 52224K ........ ........ ........ ........ ........ ........ 21%  341M 1s
 55296K ........ ........ ........ ........ ........ ........ 22%  294M 1s
 58368K ........ ........ ........ ........ ........ ........ 23%  295M 1s
 61440K ........ ........ ........ ........ ........ ........ 24%  341M 1s
 64512K ........ ........ ........ ........ ........ ........ 25%  323M 1s
 67584K ........ ........ ........ ........ ........ ........ 26%  251M 1s
 70656K ........ ........ ........ ........ ........ ........ 28%  305M 1s
 73728K ........ ........ ........ ........ ........ ........ 29%  365M 1s
 76800K ........ ........ ........ ........ ........ ........ 30%  289M 1s
 79872K ........ ........ ........ ........ ........ ........ 31%  307M 1s
 82944K ........ ........ ........ ........ ........ ........ 32%  345M 1s
 86016K ........ ........ ........ ........ ........ ........ 34%  349M 1s
 89088K ........ ........ ........ ........ ........ ........ 35%  330M 1s
 92160K ........ ........ ........ ........ ........ ........ 36%  237M 1s
 95232K ........ ........ ........ ........ ........ ........ 37%  259M 1s
 98304K ........ ........ ........ ........ ........ ........ 38%  266M 1s
101376K ........ ........ ........ ........ ........ ........ 39%  334M 1s
104448K ........ ........ ........ ........ ........ ........ 41%  348M 1s
107520K ........ ........ ........ ........ ........ ........ 42%  347M 1s
110592K ........ ........ ........ ........ ........ ........ 43%  322M 1s
113664K ........ ........ ........ ........ ........ ........ 44%  359M 1s
116736K ........ ........ ........ ........ ........ ........ 45%  302M 1s
119808K ........ ........ ........ ........ ........ ........ 46%  327M 1s
122880K ........ ........ ........ ........ ........ ........ 48%  304M 1s
125952K ........ ........ ........ ........ ........ ........ 49%  328M 1s
129024K ........ ........ ........ ........ ........ ........ 50%  338M 0s
132096K ........ ........ ........ ........ ........ ........ 51%  337M 0s
135168K ........ ........ ........ ........ ........ ........ 52%  335M 0s
138240K ........ ........ ........ ........ ........ ........ 53%  318M 0s
141312K ........ ........ ........ ........ ........ ........ 55%  350M 0s
144384K ........ ........ ........ ........ ........ ........ 56%  345M 0s
147456K ........ ........ ........ ........ ........ ........ 57%  340M 0s
150528K ........ ........ ........ ........ ........ ........ 58%  332M 0s
153600K ........ ........ ........ ........ ........ ........ 59%  333M 0s
156672K ........ ........ ........ ........ ........ ........ 60%  349M 0s
159744K ........ ........ ........ ........ ........ ........ 62%  315M 0s
162816K ........ ........ ........ ........ ........ ........ 63%  191M 0s
165888K ........ ........ ........ ........ ........ ........ 64%  273M 0s
168960K ........ ........ ........ ........ ........ ........ 65%  361M 0s
172032K ........ ........ ........ ........ ........ ........ 66%  381M 0s
175104K ........ ........ ........ ........ ........ ........ 68%  390M 0s
178176K ........ ........ ........ ........ ........ ........ 69%  373M 0s
181248K ........ ........ ........ ........ ........ ........ 70%  383M 0s
184320K ........ ........ ........ ........ ........ ........ 71%  380M 0s
187392K ........ ........ ........ ........ ........ ........ 72%  391M 0s
190464K ........ ........ ........ ........ ........ ........ 73%  379M 0s
193536K ........ ........ ........ ........ ........ ........ 75%  337M 0s
196608K ........ ........ ........ ........ ........ ........ 76%  333M 0s
199680K ........ ........ ........ ........ ........ ........ 77%  334M 0s
202752K ........ ........ ........ ........ ........ ........ 78%  326M 0s
205824K ........ ........ ........ ........ ........ ........ 79%  344M 0s
208896K ........ ........ ........ ........ ........ ........ 80%  335M 0s
211968K ........ ........ ........ ........ ........ ........ 82%  329M 0s
215040K ........ ........ ........ ........ ........ ........ 83%  310M 0s
218112K ........ ........ ........ ........ ........ ........ 84%  320M 0s
221184K ........ ........ ........ ........ ........ ........ 85%  326M 0s
224256K ........ ........ ........ ........ ........ ........ 86%  332M 0s
227328K ........ ........ ........ ........ ........ ........ 87%  326M 0s
230400K ........ ........ ........ ........ ........ ........ 89%  330M 0s
233472K ........ ........ ........ ........ ........ ........ 90%  315M 0s
236544K ........ ........ ........ ........ ........ ........ 91%  308M 0s
239616K ........ ........ ........ ........ ........ ........ 92%  360M 0s
242688K ........ ........ ........ ........ ........ ........ 93%  340M 0s
245760K ........ ........ ........ ........ ........ ........ 95%  319M 0s
248832K ........ ........ ........ ........ ........ ........ 96%  350M 0s
251904K ........ ........ ........ ........ ........ ........ 97%  321M 0s
254976K ........ ........ ........ ........ ........ ........ 98%  358M 0s
258048K ........ ........ ........ ........ ........ ........ 99%  347M 0s
261120K ........ ....                                        100%  371M=0.9s

2024-07-13 23:40:42 (288 MB/s) - ‘karaf-0.20.2-20240713.204807-83.zip’ saved [268186747/268186747]

+ echo 'Extracting the new controller...'
Extracting the new controller...
+ unzip -q karaf-0.20.2-20240713.204807-83.zip
Adding external repositories...
+ echo 'Adding external repositories...'
+ sed -ie 's%org.ops4j.pax.url.mvn.repositories=%org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases%g' /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
+ cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
################################################################################
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#
################################################################################

#
# If set to true, the following property will not allow any certificate to be used
# when accessing Maven repositories through SSL
#
#org.ops4j.pax.url.mvn.certificateCheck=

#
# Path to the local Maven settings file.
# The repositories defined in this file will be automatically added to the list
# of default repositories if the 'org.ops4j.pax.url.mvn.repositories' property
# below is not set.
# The following locations are checked for the existence of the settings.xml file
#   * 1. looks for the specified url
#   * 2. if not found looks for ${user.home}/.m2/settings.xml
#   * 3. if not found looks for ${maven.home}/conf/settings.xml
#   * 4. if not found looks for ${M2_HOME}/conf/settings.xml
#
#org.ops4j.pax.url.mvn.settings=

#
# Path to the local Maven repository which is used to avoid downloading
# artifacts when they already exist locally.
# The value of this property will be extracted from the settings.xml file
# above, or defaulted to:
#     System.getProperty( "user.home" ) + "/.m2/repository"
#
org.ops4j.pax.url.mvn.localRepository=${karaf.home}/${karaf.default.repository}

#
# Default this to false. It's just weird to use undocumented repos
#
org.ops4j.pax.url.mvn.useFallbackRepositories=false

#
# Uncomment if you don't wanna use the proxy settings
# from the Maven conf/settings.xml file
#
# org.ops4j.pax.url.mvn.proxySupport=false

#
# Comma separated list of repositories scanned when resolving an artifact.
# Those repositories will be checked before iterating through the
#    below list of repositories and even before the local repository
# A repository url can be appended with zero or more of the following flags:
#    @snapshots  : the repository contains snaphots
#    @noreleases : the repository does not contain any released artifacts
#
# The following property value will add the system folder as a repo.
#
org.ops4j.pax.url.mvn.defaultRepositories=\
    file:${karaf.home}/${karaf.default.repository}@id=system.repository@snapshots,\
    file:${karaf.data}/kar@id=kar.repository@multi@snapshots,\
    file:${karaf.base}/${karaf.default.repository}@id=child.system.repository@snapshots

# Use the default local repo (e.g.~/.m2/repository) as a "remote" repo
#org.ops4j.pax.url.mvn.defaultLocalRepoAsRemote=false

#
# Comma separated list of repositories scanned when resolving an artifact.
# The default list includes the following repositories:
#    http://repo1.maven.org/maven2@id=central
#    http://repository.springsource.com/maven/bundles/release@id=spring.ebr
#    http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external
#    http://zodiac.springsource.com/maven/bundles/release@id=gemini
#    http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases
#    https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases
#    https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases
# To add repositories to the default ones, prepend '+' to the list of repositories
# to add.
# A repository url can be appended with zero or more of the following flags:
#    @snapshots  : the repository contains snapshots
#    @noreleases : the repository does not contain any released artifacts
#    @id=repository.id : the id for the repository, just like in the settings.xml this is optional but recommended
#
org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases

### ^^^ No remote repositories. This is the only ODL change compared to Karaf defaults.Configuring the startup features...
+ [[ True == \T\r\u\e ]]
+ echo 'Configuring the startup features...'
+ sed -ie 's/\(featuresBoot=\|featuresBoot =\)/featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app,/g' /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg
+ FEATURE_TEST_STRING=features-test
+ FEATURE_TEST_VERSION=0.20.2-SNAPSHOT
+ KARAF_VERSION=karaf4
+ [[ integration == \i\n\t\e\g\r\a\t\i\o\n ]]
+ sed -ie 's%\(featuresRepositories=\|featuresRepositories =\)%featuresRepositories = mvn:org.opendaylight.integration/features-test/0.20.2-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features,%g' /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg
+ [[ ! -z '' ]]
+ cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg
################################################################################
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#
################################################################################

#
# Comma separated list of features repositories to register by default
#
featuresRepositories = mvn:org.opendaylight.integration/features-test/0.20.2-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features, file:${karaf.etc}/36d1a0ea-5dbf-413c-8167-c4d0fdd5e797.xml

#
# Comma separated list of features to install at startup
#
featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app, 04c66cf3-c01b-4f7e-8c33-e3176c80f161

#
# Resource repositories (OBR) that the features resolver can use
# to resolve requirements/capabilities
#
# The format of the resourceRepositories is 
# resourceRepositories=[xml:url|json:url],...
# for Instance:
#
#resourceRepositories=xml:http://host/path/to/index.xml
# or
#resourceRepositories=json:http://host/path/to/index.json
#

#
# Defines if the boot features are started in asynchronous mode (in a dedicated thread)
#
featuresBootAsynchronous=false

#
# Service requirements enforcement
#
# By default, the feature resolver checks the service requirements/capabilities of
# bundles for new features (xml schema >= 1.3.0) in order to automatically installs
# the required bundles.
# The following flag can have those values:
#   - disable: service requirements are completely ignored
#   - default: service requirements are ignored for old features
#   - enforce: service requirements are always verified
#
#serviceRequirements=default

#
# Store cfg file for config element in feature
#
#configCfgStore=true

#
# Define if the feature service automatically refresh bundles
#
autoRefresh=true

#
# Configuration of features processing mechanism (overrides, blacklisting, modification of features)
# XML file defines instructions related to features processing
# versions.properties may declare properties to resolve placeholders in XML file
# both files are relative to ${karaf.etc}
#
#featureProcessing=org.apache.karaf.features.xml
#featureProcessingVersions=versions.properties
+ configure_karaf_log karaf4 ''
+ local -r karaf_version=karaf4
+ local -r controllerdebugmap=
+ local logapi=log4j
+ grep log4j2 /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
log4j2.rootLogger.level = INFO
#log4j2.rootLogger.type = asyncRoot
#log4j2.rootLogger.includeLocation = false
log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile
log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi
log4j2.rootLogger.appenderRef.Console.ref = Console
log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter
log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF}
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL
log4j2.logger.spifly.name = org.apache.aries.spifly
log4j2.logger.spifly.level = WARN
log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit
log4j2.logger.audit.level = INFO
log4j2.logger.audit.additivity = false
log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile
# Console appender not used by default (see log4j2.rootLogger.appenderRefs)
log4j2.appender.console.type = Console
log4j2.appender.console.name = Console
log4j2.appender.console.layout.type = PatternLayout
log4j2.appender.console.layout.pattern = ${log4j2.pattern}
log4j2.appender.rolling.type = RollingRandomAccessFile
log4j2.appender.rolling.name = RollingFile
log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i
#log4j2.appender.rolling.immediateFlush = false
log4j2.appender.rolling.append = true
log4j2.appender.rolling.layout.type = PatternLayout
log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
log4j2.appender.rolling.policies.type = Policies
log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.rolling.policies.size.size = 64MB
log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
log4j2.appender.rolling.strategy.max = 7
log4j2.appender.audit.type = RollingRandomAccessFile
log4j2.appender.audit.name = AuditRollingFile
log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log
log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i
log4j2.appender.audit.append = true
log4j2.appender.audit.layout.type = PatternLayout
log4j2.appender.audit.layout.pattern = ${log4j2.pattern}
log4j2.appender.audit.policies.type = Policies
log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.audit.policies.size.size = 8MB
log4j2.appender.audit.strategy.type = DefaultRolloverStrategy
log4j2.appender.audit.strategy.max = 7
log4j2.appender.osgi.type = PaxOsgi
log4j2.appender.osgi.name = PaxOsgi
log4j2.appender.osgi.filter = *
#log4j2.logger.aether.name = shaded.org.eclipse.aether
#log4j2.logger.aether.level = TRACE
#log4j2.logger.http-headers.name = shaded.org.apache.http.headers
#log4j2.logger.http-headers.level = DEBUG
#log4j2.logger.maven.name = org.ops4j.pax.url.mvn
#log4j2.logger.maven.level = TRACE
Configuring the karaf log... karaf_version: karaf4, logapi: log4j2
+ logapi=log4j2
+ echo 'Configuring the karaf log... karaf_version: karaf4, logapi: log4j2'
+ '[' log4j2 == log4j2 ']'
+ sed -ie 's/log4j2.appender.rolling.policies.size.size = 64MB/log4j2.appender.rolling.policies.size.size = 1GB/g' /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
+ orgmodule=org.opendaylight.yangtools.yang.parser.repo.YangTextSchemaContextResolver
+ orgmodule_=org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver
+ echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN'
controllerdebugmap: 
cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
+ echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN'
+ unset IFS
+ echo 'controllerdebugmap: '
+ '[' -n '' ']'
+ echo 'cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg'
+ cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
################################################################################
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#
################################################################################

# Common pattern layout for appenders
log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n

# Root logger
log4j2.rootLogger.level = INFO
# uncomment to use asynchronous loggers, which require mvn:com.lmax/disruptor/3.3.2 library
#log4j2.rootLogger.type = asyncRoot
#log4j2.rootLogger.includeLocation = false
log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile
log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi
log4j2.rootLogger.appenderRef.Console.ref = Console
log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter
log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF}

# Filters for logs marked by org.opendaylight.odlparent.Markers
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL

# Loggers configuration

# Spifly logger
log4j2.logger.spifly.name = org.apache.aries.spifly
log4j2.logger.spifly.level = WARN

# Security audit logger
log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit
log4j2.logger.audit.level = INFO
log4j2.logger.audit.additivity = false
log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile

# Appenders configuration

# Console appender not used by default (see log4j2.rootLogger.appenderRefs)
log4j2.appender.console.type = Console
log4j2.appender.console.name = Console
log4j2.appender.console.layout.type = PatternLayout
log4j2.appender.console.layout.pattern = ${log4j2.pattern}

# Rolling file appender
log4j2.appender.rolling.type = RollingRandomAccessFile
log4j2.appender.rolling.name = RollingFile
log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i
# uncomment to not force a disk flush
#log4j2.appender.rolling.immediateFlush = false
log4j2.appender.rolling.append = true
log4j2.appender.rolling.layout.type = PatternLayout
log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
log4j2.appender.rolling.policies.type = Policies
log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.rolling.policies.size.size = 1GB
log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
log4j2.appender.rolling.strategy.max = 7

# Audit file appender
log4j2.appender.audit.type = RollingRandomAccessFile
log4j2.appender.audit.name = AuditRollingFile
log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log
log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i
log4j2.appender.audit.append = true
log4j2.appender.audit.layout.type = PatternLayout
log4j2.appender.audit.layout.pattern = ${log4j2.pattern}
log4j2.appender.audit.policies.type = Policies
log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.audit.policies.size.size = 8MB
log4j2.appender.audit.strategy.type = DefaultRolloverStrategy
log4j2.appender.audit.strategy.max = 7

# OSGi appender
log4j2.appender.osgi.type = PaxOsgi
log4j2.appender.osgi.name = PaxOsgi
log4j2.appender.osgi.filter = *

# help with identification of maven-related problems with pax-url-aether
#log4j2.logger.aether.name = shaded.org.eclipse.aether
#log4j2.logger.aether.level = TRACE
#log4j2.logger.http-headers.name = shaded.org.apache.http.headers
#log4j2.logger.http-headers.level = DEBUG
#log4j2.logger.maven.name = org.ops4j.pax.url.mvn
#log4j2.logger.maven.level = TRACE
log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN
log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN
+ set_java_vars /usr/lib/jvm/java-17-openjdk 3072m /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
+ local -r java_home=/usr/lib/jvm/java-17-openjdk
+ local -r controllermem=3072m
+ local -r memconf=/tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
Configure
    java home: /usr/lib/jvm/java-17-openjdk
    max memory: 3072m
    memconf: /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
+ echo Configure
+ echo '    java home: /usr/lib/jvm/java-17-openjdk'
+ echo '    max memory: 3072m'
+ echo '    memconf: /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv'
+ sed -ie 's%^# export JAVA_HOME%export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-17-openjdk}%g' /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
+ sed -ie 's/JAVA_MAX_MEM="2048m"/JAVA_MAX_MEM=3072m/g' /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
cat /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
+ echo 'cat /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv'
+ cat /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
#!/bin/sh
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#

#
# handle specific scripts; the SCRIPT_NAME is exactly the name of the Karaf
# script: client, instance, shell, start, status, stop, karaf
#
# if [ "${KARAF_SCRIPT}" == "SCRIPT_NAME" ]; then
#   Actions go here...
# fi

#
# general settings which should be applied for all scripts go here; please keep
# in mind that it is possible that scripts might be executed more than once, e.g.
# in example of the start script where the start script is executed first and the
# karaf script afterwards.
#

#
# The following section shows the possible configuration options for the default 
# karaf scripts
#
export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-17-openjdk} # Location of Java installation
# export JAVA_OPTS # Generic JVM options, for instance, where you can pass the memory configuration
# export JAVA_NON_DEBUG_OPTS # Additional non-debug JVM options
# export EXTRA_JAVA_OPTS # Additional JVM options
# export KARAF_HOME # Karaf home folder
# export KARAF_DATA # Karaf data folder
# export KARAF_BASE # Karaf base folder
# export KARAF_ETC  # Karaf etc  folder
# export KARAF_LOG  # Karaf log  folder
# export KARAF_SYSTEM_OPTS # First citizen Karaf options
# export KARAF_OPTS # Additional available Karaf options
# export KARAF_DEBUG # Enable debug mode
# export KARAF_REDIRECT # Enable/set the std/err redirection when using bin/start
# export KARAF_NOROOT # Prevent execution as root if set to true
Set Java version
+ echo 'Set Java version'
+ sudo /usr/sbin/alternatives --install /usr/bin/java java /usr/lib/jvm/java-17-openjdk/bin/java 1
+ sudo /usr/sbin/alternatives --set java /usr/lib/jvm/java-17-openjdk/bin/java
JDK default version ...
+ echo 'JDK default version ...'
+ java -version
openjdk version "17.0.6-ea" 2023-01-17 LTS
OpenJDK Runtime Environment (Red_Hat-17.0.6.0.9-0.3.ea.el8) (build 17.0.6-ea+9-LTS)
OpenJDK 64-Bit Server VM (Red_Hat-17.0.6.0.9-0.3.ea.el8) (build 17.0.6-ea+9-LTS, mixed mode, sharing)
Set JAVA_HOME
+ echo 'Set JAVA_HOME'
+ export JAVA_HOME=/usr/lib/jvm/java-17-openjdk
+ JAVA_HOME=/usr/lib/jvm/java-17-openjdk
++ readlink -e /usr/lib/jvm/java-17-openjdk/bin/java
+ JAVA_RESOLVED=/usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin/java
Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin/java
+ echo 'Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin/java'
Listing all open ports on controller system...
+ echo 'Listing all open ports on controller system...'
+ netstat -pnatu
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -                   
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      -                   
tcp        0      0 10.30.171.141:47920     199.204.45.87:443       TIME_WAIT   -                   
tcp        0    164 10.30.171.141:22        10.30.170.213:50820     ESTABLISHED -                   
tcp6       0      0 :::5555                 :::*                    LISTEN      -                   
tcp6       0      0 :::22                   :::*                    LISTEN      -                   
tcp6       0      0 :::111                  :::*                    LISTEN      -                   
udp        0      0 10.30.171.141:68        10.30.170.2:67          ESTABLISHED -                   
udp        0      0 0.0.0.0:111             0.0.0.0:*                           -                   
udp        0      0 127.0.0.1:323           0.0.0.0:*                           -                   
udp6       0      0 :::111                  :::*                                -                   
udp6       0      0 ::1:323                 :::*                                -                   
+ '[' -f /tmp/custom_shard_config.txt ']'
Custom shard config exists!!!
+ echo 'Custom shard config exists!!!'
+ echo 'Copying the shard config...'
Copying the shard config...
+ cp /tmp/custom_shard_config.txt /tmp/karaf-0.20.2-SNAPSHOT/bin/
Configuring cluster
+ echo 'Configuring cluster'
+ /tmp/karaf-0.20.2-SNAPSHOT/bin/configure_cluster.sh 1 10.30.171.141 10.30.170.225 10.30.170.154
################################################
##             Configure Cluster              ##
################################################
 NOTE: Cluster configuration files not found. Copying from
 /tmp/karaf-0.20.2-SNAPSHOT/system/org/opendaylight/controller/sal-clustering-config/9.0.4
Configuring unique name in akka.conf
Configuring hostname in akka.conf
Configuring data and rpc seed nodes in akka.conf
modules = [
	
	{
		name = "inventory"
		namespace = "urn:opendaylight:inventory"
		shard-strategy = "module"
	},
	{
		name = "topology"
		namespace = "urn:TBD:params:xml:ns:yang:network-topology"
		shard-strategy = "module"
	},
	{
		name = "toaster"
		namespace = "http://netconfcentral.org/ns/toaster"
		shard-strategy = "module"
	},
	{
		name = "car"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car"
		shard-strategy = "module"
	},
	{
		name = "people"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people"
		shard-strategy = "module"
	},
	{
		name = "car-people"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people"
		shard-strategy = "module"
	}
]
Configuring replication type in module-shards.conf
################################################
##   NOTE: Manually restart controller to     ##
##         apply configuration.               ##
################################################
Dump akka.conf
+ echo 'Dump akka.conf'
+ cat /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf

odl-cluster-data {
  akka {
    remote {
      artery {
        enabled = on
        transport = tcp
        canonical.hostname = "10.30.171.141"
        canonical.port = 2550
      }
    }

    cluster {
      # Using artery.
      seed-nodes = ["akka://opendaylight-cluster-data@10.30.171.141:2550",
				"akka://opendaylight-cluster-data@10.30.170.225:2550",
				"akka://opendaylight-cluster-data@10.30.170.154:2550"]

      roles = ["member-1"]

      # when under load we might trip a false positive on the failure detector
      # failure-detector {
        # heartbeat-interval = 4 s
        # acceptable-heartbeat-pause = 16s
      # }
    }

    persistence {
      # By default the snapshots/journal directories live in KARAF_HOME. You can choose to put it somewhere else by
      # modifying the following two properties. The directory location specified may be a relative or absolute path. 
      # The relative path is always relative to KARAF_HOME.

      # snapshot-store.local.dir = "target/snapshots"

      # Use lz4 compression for LocalSnapshotStore snapshots
      snapshot-store.local.use-lz4-compression = false
      # Size of blocks for lz4 compression: 64KB, 256KB, 1MB or 4MB
      snapshot-store.local.lz4-blocksize = 256KB
    }
    disable-default-actor-system-quarantined-event-handling = "false"
  }
}
Dump modules.conf
+ echo 'Dump modules.conf'
+ cat /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf
modules = [
	
	{
		name = "inventory"
		namespace = "urn:opendaylight:inventory"
		shard-strategy = "module"
	},
	{
		name = "topology"
		namespace = "urn:TBD:params:xml:ns:yang:network-topology"
		shard-strategy = "module"
	},
	{
		name = "toaster"
		namespace = "http://netconfcentral.org/ns/toaster"
		shard-strategy = "module"
	},
	{
		name = "car"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car"
		shard-strategy = "module"
	},
	{
		name = "people"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people"
		shard-strategy = "module"
	},
	{
		name = "car-people"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people"
		shard-strategy = "module"
	}
]
+ echo 'Dump module-shards.conf'
Dump module-shards.conf
+ cat /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf
module-shards = [
	{
		name = "default"
		shards = [
			{
				name = "default"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "inventory"
		shards = [
			{
				name="inventory"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "topology"
		shards = [
			{
				name="topology"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "toaster"
		shards = [
			{
				name="toaster"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "car"
		shards = [
			{
				name="car"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "people"
		shards = [
			{
				name="people"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "car-people"
		shards = [
			{
				name="car-people"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	}
]
Configuring member-2 with IP address 10.30.170.225
Warning: Permanently added '10.30.170.225' (ECDSA) to the list of known hosts.
Warning: Permanently added '10.30.170.225' (ECDSA) to the list of known hosts.
+ source /tmp/common-functions.sh karaf-0.20.2-SNAPSHOT
common-functions.sh is being sourced
++ [[ /tmp/common-functions.sh == \/\t\m\p\/\c\o\n\f\i\g\u\r\a\t\i\o\n\-\s\c\r\i\p\t\.\s\h ]]
++ echo 'common-functions.sh is being sourced'
++ BUNDLEFOLDER=karaf-0.20.2-SNAPSHOT
++ export MAVENCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
++ MAVENCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
++ export FEATURESCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg
++ FEATURESCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg
++ export CUSTOMPROP=/tmp/karaf-0.20.2-SNAPSHOT/etc/custom.properties
++ CUSTOMPROP=/tmp/karaf-0.20.2-SNAPSHOT/etc/custom.properties
++ export LOGCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
++ LOGCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
++ export MEMCONF=/tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
++ MEMCONF=/tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
++ export CONTROLLERMEM=
++ CONTROLLERMEM=
++ export AKKACONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf
++ AKKACONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf
++ export MODULESCONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf
++ MODULESCONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf
++ export MODULESHARDSCONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf
++ MODULESHARDSCONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf
++ print_common_env
++ cat
common-functions environment:
MAVENCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
ACTUALFEATURES: 
FEATURESCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg
CUSTOMPROP: /tmp/karaf-0.20.2-SNAPSHOT/etc/custom.properties
LOGCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
MEMCONF: /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
CONTROLLERMEM: 
AKKACONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf
MODULESCONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf
MODULESHARDSCONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf
SUITES: 

++ SSH='ssh -t -t'
++ extra_services_cntl='     dnsmasq.service     httpd.service     libvirtd.service     openvswitch.service     ovs-vswitchd.service     ovsdb-server.service     rabbitmq-server.service '
++ extra_services_cmp='     libvirtd.service     openvswitch.service     ovs-vswitchd.service     ovsdb-server.service '
Changing to /tmp
Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240713.204807-83.zip
+ echo 'Changing to /tmp'
+ cd /tmp
+ echo 'Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240713.204807-83.zip'
+ wget --progress=dot:mega https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240713.204807-83.zip
--2024-07-13 23:40:45--  https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240713.204807-83.zip
Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6
Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 268186747 (256M) [application/zip]
Saving to: ‘karaf-0.20.2-20240713.204807-83.zip’

     0K ........ ........ ........ ........ ........ ........  1%  102M 2s
  3072K ........ ........ ........ ........ ........ ........  2%  163M 2s
  6144K ........ ........ ........ ........ ........ ........  3%  181M 2s
  9216K ........ ........ ........ ........ ........ ........  4%  191M 2s
 12288K ........ ........ ........ ........ ........ ........  5%  221M 2s
 15360K ........ ........ ........ ........ ........ ........  7%  244M 1s
 18432K ........ ........ ........ ........ ........ ........  8%  257M 1s
 21504K ........ ........ ........ ........ ........ ........  9%  247M 1s
 24576K ........ ........ ........ ........ ........ ........ 10%  247M 1s
 27648K ........ ........ ........ ........ ........ ........ 11%  300M 1s
 30720K ........ ........ ........ ........ ........ ........ 12%  284M 1s
 33792K ........ ........ ........ ........ ........ ........ 14%  342M 1s
 36864K ........ ........ ........ ........ ........ ........ 15%  376M 1s
 39936K ........ ........ ........ ........ ........ ........ 16%  298M 1s
 43008K ........ ........ ........ ........ ........ ........ 17%  300M 1s
 46080K ........ ........ ........ ........ ........ ........ 18%  329M 1s
 49152K ........ ........ ........ ........ ........ ........ 19%  338M 1s
 52224K ........ ........ ........ ........ ........ ........ 21%  331M 1s
 55296K ........ ........ ........ ........ ........ ........ 22%  351M 1s
 58368K ........ ........ ........ ........ ........ ........ 23%  285M 1s
 61440K ........ ........ ........ ........ ........ ........ 24%  319M 1s
 64512K ........ ........ ........ ........ ........ ........ 25%  343M 1s
 67584K ........ ........ ........ ........ ........ ........ 26%  322M 1s
 70656K ........ ........ ........ ........ ........ ........ 28%  291M 1s
 73728K ........ ........ ........ ........ ........ ........ 29%  344M 1s
 76800K ........ ........ ........ ........ ........ ........ 30%  332M 1s
 79872K ........ ........ ........ ........ ........ ........ 31%  342M 1s
 82944K ........ ........ ........ ........ ........ ........ 32%  334M 1s
 86016K ........ ........ ........ ........ ........ ........ 34%  333M 1s
 89088K ........ ........ ........ ........ ........ ........ 35%  339M 1s
 92160K ........ ........ ........ ........ ........ ........ 36%  317M 1s
 95232K ........ ........ ........ ........ ........ ........ 37%  319M 1s
 98304K ........ ........ ........ ........ ........ ........ 38%  324M 1s
101376K ........ ........ ........ ........ ........ ........ 39%  333M 1s
104448K ........ ........ ........ ........ ........ ........ 41%  333M 1s
107520K ........ ........ ........ ........ ........ ........ 42%  324M 1s
110592K ........ ........ ........ ........ ........ ........ 43%  330M 1s
113664K ........ ........ ........ ........ ........ ........ 44%  328M 1s
116736K ........ ........ ........ ........ ........ ........ 45%  344M 0s
119808K ........ ........ ........ ........ ........ ........ 46%  338M 0s
122880K ........ ........ ........ ........ ........ ........ 48%  312M 0s
125952K ........ ........ ........ ........ ........ ........ 49%  337M 0s
129024K ........ ........ ........ ........ ........ ........ 50%  322M 0s
132096K ........ ........ ........ ........ ........ ........ 51%  328M 0s
135168K ........ ........ ........ ........ ........ ........ 52%  300M 0s
138240K ........ ........ ........ ........ ........ ........ 53%  330M 0s
141312K ........ ........ ........ ........ ........ ........ 55%  309M 0s
144384K ........ ........ ........ ........ ........ ........ 56%  313M 0s
147456K ........ ........ ........ ........ ........ ........ 57%  321M 0s
150528K ........ ........ ........ ........ ........ ........ 58%  297M 0s
153600K ........ ........ ........ ........ ........ ........ 59%  307M 0s
156672K ........ ........ ........ ........ ........ ........ 60%  305M 0s
159744K ........ ........ ........ ........ ........ ........ 62%  293M 0s
162816K ........ ........ ........ ........ ........ ........ 63%  226M 0s
165888K ........ ........ ........ ........ ........ ........ 64%  371M 0s
168960K ........ ........ ........ ........ ........ ........ 65%  346M 0s
172032K ........ ........ ........ ........ ........ ........ 66%  292M 0s
175104K ........ ........ ........ ........ ........ ........ 68%  280M 0s
178176K ........ ........ ........ ........ ........ ........ 69%  291M 0s
181248K ........ ........ ........ ........ ........ ........ 70%  302M 0s
184320K ........ ........ ........ ........ ........ ........ 71%  297M 0s
187392K ........ ........ ........ ........ ........ ........ 72%  292M 0s
190464K ........ ........ ........ ........ ........ ........ 73%  290M 0s
193536K ........ ........ ........ ........ ........ ........ 75%  300M 0s
196608K ........ ........ ........ ........ ........ ........ 76%  292M 0s
199680K ........ ........ ........ ........ ........ ........ 77%  301M 0s
202752K ........ ........ ........ ........ ........ ........ 78%  293M 0s
205824K ........ ........ ........ ........ ........ ........ 79%  289M 0s
208896K ........ ........ ........ ........ ........ ........ 80%  282M 0s
211968K ........ ........ ........ ........ ........ ........ 82%  302M 0s
215040K ........ ........ ........ ........ ........ ........ 83%  295M 0s
218112K ........ ........ ........ ........ ........ ........ 84%  292M 0s
221184K ........ ........ ........ ........ ........ ........ 85%  295M 0s
224256K ........ ........ ........ ........ ........ ........ 86%  286M 0s
227328K ........ ........ ........ ........ ........ ........ 87%  237M 0s
230400K ........ ........ ........ ........ ........ ........ 89%  315M 0s
233472K ........ ........ ........ ........ ........ ........ 90%  158M 0s
236544K ........ ........ ........ ........ ........ ........ 91%  297M 0s
239616K ........ ........ ........ ........ ........ ........ 92%  366M 0s
242688K ........ ........ ........ ........ ........ ........ 93%  361M 0s
245760K ........ ........ ........ ........ ........ ........ 95%  292M 0s
248832K ........ ........ ........ ........ ........ ........ 96%  342M 0s
251904K ........ ........ ........ ........ ........ ........ 97%  302M 0s
254976K ........ ........ ........ ........ ........ ........ 98%  302M 0s
258048K ........ ........ ........ ........ ........ ........ 99%  299M 0s
261120K ........ ....                                        100%  316M=0.9s

2024-07-13 23:40:46 (289 MB/s) - ‘karaf-0.20.2-20240713.204807-83.zip’ saved [268186747/268186747]

Extracting the new controller...
+ echo 'Extracting the new controller...'
+ unzip -q karaf-0.20.2-20240713.204807-83.zip
Adding external repositories...
+ echo 'Adding external repositories...'
+ sed -ie 's%org.ops4j.pax.url.mvn.repositories=%org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases%g' /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
+ cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
################################################################################
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#
################################################################################

#
# If set to true, the following property will not allow any certificate to be used
# when accessing Maven repositories through SSL
#
#org.ops4j.pax.url.mvn.certificateCheck=

#
# Path to the local Maven settings file.
# The repositories defined in this file will be automatically added to the list
# of default repositories if the 'org.ops4j.pax.url.mvn.repositories' property
# below is not set.
# The following locations are checked for the existence of the settings.xml file
#   * 1. looks for the specified url
#   * 2. if not found looks for ${user.home}/.m2/settings.xml
#   * 3. if not found looks for ${maven.home}/conf/settings.xml
#   * 4. if not found looks for ${M2_HOME}/conf/settings.xml
#
#org.ops4j.pax.url.mvn.settings=

#
# Path to the local Maven repository which is used to avoid downloading
# artifacts when they already exist locally.
# The value of this property will be extracted from the settings.xml file
# above, or defaulted to:
#     System.getProperty( "user.home" ) + "/.m2/repository"
#
org.ops4j.pax.url.mvn.localRepository=${karaf.home}/${karaf.default.repository}

#
# Default this to false. It's just weird to use undocumented repos
#
org.ops4j.pax.url.mvn.useFallbackRepositories=false

#
# Uncomment if you don't wanna use the proxy settings
# from the Maven conf/settings.xml file
#
# org.ops4j.pax.url.mvn.proxySupport=false

#
# Comma separated list of repositories scanned when resolving an artifact.
# Those repositories will be checked before iterating through the
#    below list of repositories and even before the local repository
# A repository url can be appended with zero or more of the following flags:
#    @snapshots  : the repository contains snaphots
#    @noreleases : the repository does not contain any released artifacts
#
# The following property value will add the system folder as a repo.
#
org.ops4j.pax.url.mvn.defaultRepositories=\
    file:${karaf.home}/${karaf.default.repository}@id=system.repository@snapshots,\
    file:${karaf.data}/kar@id=kar.repository@multi@snapshots,\
    file:${karaf.base}/${karaf.default.repository}@id=child.system.repository@snapshots

# Use the default local repo (e.g.~/.m2/repository) as a "remote" repo
#org.ops4j.pax.url.mvn.defaultLocalRepoAsRemote=false

#
# Comma separated list of repositories scanned when resolving an artifact.
# The default list includes the following repositories:
#    http://repo1.maven.org/maven2@id=central
#    http://repository.springsource.com/maven/bundles/release@id=spring.ebr
#    http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external
#    http://zodiac.springsource.com/maven/bundles/release@id=gemini
#    http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases
#    https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases
#    https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases
# To add repositories to the default ones, prepend '+' to the list of repositories
# to add.
# A repository url can be appended with zero or more of the following flags:
#    @snapshots  : the repository contains snapshots
#    @noreleases : the repository does not contain any released artifacts
#    @id=repository.id : the id for the repository, just like in the settings.xml this is optional but recommended
#
org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases

### ^^^ No remote repositories. This is the only ODL change compared to Karaf defaults.Configuring the startup features...
+ [[ True == \T\r\u\e ]]
+ echo 'Configuring the startup features...'
+ sed -ie 's/\(featuresBoot=\|featuresBoot =\)/featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app,/g' /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg
+ FEATURE_TEST_STRING=features-test
+ FEATURE_TEST_VERSION=0.20.2-SNAPSHOT
+ KARAF_VERSION=karaf4
+ [[ integration == \i\n\t\e\g\r\a\t\i\o\n ]]
+ sed -ie 's%\(featuresRepositories=\|featuresRepositories =\)%featuresRepositories = mvn:org.opendaylight.integration/features-test/0.20.2-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features,%g' /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg
+ [[ ! -z '' ]]
+ cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg
################################################################################
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#
################################################################################

#
# Comma separated list of features repositories to register by default
#
featuresRepositories = mvn:org.opendaylight.integration/features-test/0.20.2-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features, file:${karaf.etc}/36d1a0ea-5dbf-413c-8167-c4d0fdd5e797.xml

#
# Comma separated list of features to install at startup
#
featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app, 04c66cf3-c01b-4f7e-8c33-e3176c80f161

#
# Resource repositories (OBR) that the features resolver can use
# to resolve requirements/capabilities
#
# The format of the resourceRepositories is 
# resourceRepositories=[xml:url|json:url],...
# for Instance:
#
#resourceRepositories=xml:http://host/path/to/index.xml
# or
#resourceRepositories=json:http://host/path/to/index.json
#

#
# Defines if the boot features are started in asynchronous mode (in a dedicated thread)
#
featuresBootAsynchronous=false

#
# Service requirements enforcement
#
# By default, the feature resolver checks the service requirements/capabilities of
# bundles for new features (xml schema >= 1.3.0) in order to automatically installs
# the required bundles.
# The following flag can have those values:
#   - disable: service requirements are completely ignored
#   - default: service requirements are ignored for old features
#   - enforce: service requirements are always verified
#
#serviceRequirements=default

#
# Store cfg file for config element in feature
#
#configCfgStore=true

#
# Define if the feature service automatically refresh bundles
#
autoRefresh=true

#
# Configuration of features processing mechanism (overrides, blacklisting, modification of features)
# XML file defines instructions related to features processing
# versions.properties may declare properties to resolve placeholders in XML file
# both files are relative to ${karaf.etc}
#
#featureProcessing=org.apache.karaf.features.xml
#featureProcessingVersions=versions.properties
+ configure_karaf_log karaf4 ''
+ local -r karaf_version=karaf4
+ local -r controllerdebugmap=
+ local logapi=log4j
+ grep log4j2 /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
log4j2.rootLogger.level = INFO
#log4j2.rootLogger.type = asyncRoot
#log4j2.rootLogger.includeLocation = false
log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile
log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi
log4j2.rootLogger.appenderRef.Console.ref = Console
log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter
log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF}
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL
log4j2.logger.spifly.name = org.apache.aries.spifly
log4j2.logger.spifly.level = WARN
log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit
log4j2.logger.audit.level = INFO
log4j2.logger.audit.additivity = false
log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile
# Console appender not used by default (see log4j2.rootLogger.appenderRefs)
log4j2.appender.console.type = Console
log4j2.appender.console.name = Console
log4j2.appender.console.layout.type = PatternLayout
log4j2.appender.console.layout.pattern = ${log4j2.pattern}
log4j2.appender.rolling.type = RollingRandomAccessFile
log4j2.appender.rolling.name = RollingFile
log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i
#log4j2.appender.rolling.immediateFlush = false
log4j2.appender.rolling.append = true
log4j2.appender.rolling.layout.type = PatternLayout
log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
log4j2.appender.rolling.policies.type = Policies
log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.rolling.policies.size.size = 64MB
log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
log4j2.appender.rolling.strategy.max = 7
log4j2.appender.audit.type = RollingRandomAccessFile
log4j2.appender.audit.name = AuditRollingFile
log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log
log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i
log4j2.appender.audit.append = true
log4j2.appender.audit.layout.type = PatternLayout
log4j2.appender.audit.layout.pattern = ${log4j2.pattern}
log4j2.appender.audit.policies.type = Policies
log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.audit.policies.size.size = 8MB
log4j2.appender.audit.strategy.type = DefaultRolloverStrategy
log4j2.appender.audit.strategy.max = 7
log4j2.appender.osgi.type = PaxOsgi
log4j2.appender.osgi.name = PaxOsgi
log4j2.appender.osgi.filter = *
#log4j2.logger.aether.name = shaded.org.eclipse.aether
#log4j2.logger.aether.level = TRACE
#log4j2.logger.http-headers.name = shaded.org.apache.http.headers
#log4j2.logger.http-headers.level = DEBUG
#log4j2.logger.maven.name = org.ops4j.pax.url.mvn
#log4j2.logger.maven.level = TRACE
Configuring the karaf log... karaf_version: karaf4, logapi: log4j2
+ logapi=log4j2
+ echo 'Configuring the karaf log... karaf_version: karaf4, logapi: log4j2'
+ '[' log4j2 == log4j2 ']'
+ sed -ie 's/log4j2.appender.rolling.policies.size.size = 64MB/log4j2.appender.rolling.policies.size.size = 1GB/g' /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
+ orgmodule=org.opendaylight.yangtools.yang.parser.repo.YangTextSchemaContextResolver
+ orgmodule_=org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver
+ echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN'
+ echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN'
+ unset IFS
controllerdebugmap: 
cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
+ echo 'controllerdebugmap: '
+ '[' -n '' ']'
+ echo 'cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg'
+ cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
################################################################################
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#
################################################################################

# Common pattern layout for appenders
log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n

# Root logger
log4j2.rootLogger.level = INFO
# uncomment to use asynchronous loggers, which require mvn:com.lmax/disruptor/3.3.2 library
#log4j2.rootLogger.type = asyncRoot
#log4j2.rootLogger.includeLocation = false
log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile
log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi
log4j2.rootLogger.appenderRef.Console.ref = Console
log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter
log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF}

# Filters for logs marked by org.opendaylight.odlparent.Markers
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL

# Loggers configuration

# Spifly logger
log4j2.logger.spifly.name = org.apache.aries.spifly
log4j2.logger.spifly.level = WARN

# Security audit logger
log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit
log4j2.logger.audit.level = INFO
log4j2.logger.audit.additivity = false
log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile

# Appenders configuration

# Console appender not used by default (see log4j2.rootLogger.appenderRefs)
log4j2.appender.console.type = Console
log4j2.appender.console.name = Console
log4j2.appender.console.layout.type = PatternLayout
log4j2.appender.console.layout.pattern = ${log4j2.pattern}

# Rolling file appender
log4j2.appender.rolling.type = RollingRandomAccessFile
log4j2.appender.rolling.name = RollingFile
log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i
# uncomment to not force a disk flush
#log4j2.appender.rolling.immediateFlush = false
log4j2.appender.rolling.append = true
log4j2.appender.rolling.layout.type = PatternLayout
log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
log4j2.appender.rolling.policies.type = Policies
log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.rolling.policies.size.size = 1GB
log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
log4j2.appender.rolling.strategy.max = 7

# Audit file appender
log4j2.appender.audit.type = RollingRandomAccessFile
log4j2.appender.audit.name = AuditRollingFile
log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log
log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i
log4j2.appender.audit.append = true
log4j2.appender.audit.layout.type = PatternLayout
log4j2.appender.audit.layout.pattern = ${log4j2.pattern}
log4j2.appender.audit.policies.type = Policies
log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.audit.policies.size.size = 8MB
log4j2.appender.audit.strategy.type = DefaultRolloverStrategy
log4j2.appender.audit.strategy.max = 7

# OSGi appender
log4j2.appender.osgi.type = PaxOsgi
log4j2.appender.osgi.name = PaxOsgi
log4j2.appender.osgi.filter = *

# help with identification of maven-related problems with pax-url-aether
#log4j2.logger.aether.name = shaded.org.eclipse.aether
#log4j2.logger.aether.level = TRACE
#log4j2.logger.http-headers.name = shaded.org.apache.http.headers
#log4j2.logger.http-headers.level = DEBUG
#log4j2.logger.maven.name = org.ops4j.pax.url.mvn
#log4j2.logger.maven.level = TRACE
log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN
log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN
+ set_java_vars /usr/lib/jvm/java-17-openjdk 3072m /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
+ local -r java_home=/usr/lib/jvm/java-17-openjdk
+ local -r controllermem=3072m
+ local -r memconf=/tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
Configure
    java home: /usr/lib/jvm/java-17-openjdk
    max memory: 3072m
    memconf: /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
+ echo Configure
+ echo '    java home: /usr/lib/jvm/java-17-openjdk'
+ echo '    max memory: 3072m'
+ echo '    memconf: /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv'
+ sed -ie 's%^# export JAVA_HOME%export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-17-openjdk}%g' /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
+ sed -ie 's/JAVA_MAX_MEM="2048m"/JAVA_MAX_MEM=3072m/g' /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
cat /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
+ echo 'cat /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv'
+ cat /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
#!/bin/sh
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#

#
# handle specific scripts; the SCRIPT_NAME is exactly the name of the Karaf
# script: client, instance, shell, start, status, stop, karaf
#
# if [ "${KARAF_SCRIPT}" == "SCRIPT_NAME" ]; then
#   Actions go here...
# fi

#
# general settings which should be applied for all scripts go here; please keep
# in mind that it is possible that scripts might be executed more than once, e.g.
# in example of the start script where the start script is executed first and the
# karaf script afterwards.
#

#
# The following section shows the possible configuration options for the default 
# karaf scripts
#
export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-17-openjdk} # Location of Java installation
# export JAVA_OPTS # Generic JVM options, for instance, where you can pass the memory configuration
# export JAVA_NON_DEBUG_OPTS # Additional non-debug JVM options
# export EXTRA_JAVA_OPTS # Additional JVM options
# export KARAF_HOME # Karaf home folder
# export KARAF_DATA # Karaf data folder
# export KARAF_BASE # Karaf base folder
# export KARAF_ETC  # Karaf etc  folder
# export KARAF_LOG  # Karaf log  folder
# export KARAF_SYSTEM_OPTS # First citizen Karaf options
# export KARAF_OPTS # Additional available Karaf options
# export KARAF_DEBUG # Enable debug mode
# export KARAF_REDIRECT # Enable/set the std/err redirection when using bin/start
# export KARAF_NOROOT # Prevent execution as root if set to true
Set Java version
+ echo 'Set Java version'
+ sudo /usr/sbin/alternatives --install /usr/bin/java java /usr/lib/jvm/java-17-openjdk/bin/java 1
+ sudo /usr/sbin/alternatives --set java /usr/lib/jvm/java-17-openjdk/bin/java
JDK default version ...
+ echo 'JDK default version ...'
+ java -version
openjdk version "17.0.6-ea" 2023-01-17 LTS
OpenJDK Runtime Environment (Red_Hat-17.0.6.0.9-0.3.ea.el8) (build 17.0.6-ea+9-LTS)
OpenJDK 64-Bit Server VM (Red_Hat-17.0.6.0.9-0.3.ea.el8) (build 17.0.6-ea+9-LTS, mixed mode, sharing)
Set JAVA_HOME
+ echo 'Set JAVA_HOME'
+ export JAVA_HOME=/usr/lib/jvm/java-17-openjdk
+ JAVA_HOME=/usr/lib/jvm/java-17-openjdk
++ readlink -e /usr/lib/jvm/java-17-openjdk/bin/java
+ JAVA_RESOLVED=/usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin/java
+ echo 'Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin/java'
Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin/java
Listing all open ports on controller system...
+ echo 'Listing all open ports on controller system...'
+ netstat -pnatu
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      -                   
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -                   
tcp        0      0 10.30.170.225:50832     199.204.45.87:443       TIME_WAIT   -                   
tcp        0      0 10.30.170.225:22        10.30.170.213:37164     ESTABLISHED -                   
tcp6       0      0 :::111                  :::*                    LISTEN      -                   
tcp6       0      0 :::5555                 :::*                    LISTEN      -                   
tcp6       0      0 :::22                   :::*                    LISTEN      -                   
udp        0      0 10.30.170.225:68        10.30.170.3:67          ESTABLISHED -                   
udp        0      0 0.0.0.0:111             0.0.0.0:*                           -                   
udp        0      0 127.0.0.1:323           0.0.0.0:*                           -                   
udp6       0      0 :::111                  :::*                                -                   
udp6       0      0 ::1:323                 :::*                                -                   
+ '[' -f /tmp/custom_shard_config.txt ']'
Custom shard config exists!!!
+ echo 'Custom shard config exists!!!'
+ echo 'Copying the shard config...'
Copying the shard config...
+ cp /tmp/custom_shard_config.txt /tmp/karaf-0.20.2-SNAPSHOT/bin/
Configuring cluster
+ echo 'Configuring cluster'
+ /tmp/karaf-0.20.2-SNAPSHOT/bin/configure_cluster.sh 2 10.30.171.141 10.30.170.225 10.30.170.154
################################################
##             Configure Cluster              ##
################################################
 NOTE: Cluster configuration files not found. Copying from
 /tmp/karaf-0.20.2-SNAPSHOT/system/org/opendaylight/controller/sal-clustering-config/9.0.4
Configuring unique name in akka.conf
Configuring hostname in akka.conf
Configuring data and rpc seed nodes in akka.conf
modules = [
	
	{
		name = "inventory"
		namespace = "urn:opendaylight:inventory"
		shard-strategy = "module"
	},
	{
		name = "topology"
		namespace = "urn:TBD:params:xml:ns:yang:network-topology"
		shard-strategy = "module"
	},
	{
		name = "toaster"
		namespace = "http://netconfcentral.org/ns/toaster"
		shard-strategy = "module"
	},
	{
		name = "car"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car"
		shard-strategy = "module"
	},
	{
		name = "people"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people"
		shard-strategy = "module"
	},
	{
		name = "car-people"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people"
		shard-strategy = "module"
	}
]
Configuring replication type in module-shards.conf
################################################
##   NOTE: Manually restart controller to     ##
##         apply configuration.               ##
################################################
Dump akka.conf
+ echo 'Dump akka.conf'
+ cat /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf

odl-cluster-data {
  akka {
    remote {
      artery {
        enabled = on
        transport = tcp
        canonical.hostname = "10.30.170.225"
        canonical.port = 2550
      }
    }

    cluster {
      # Using artery.
      seed-nodes = ["akka://opendaylight-cluster-data@10.30.171.141:2550",
				"akka://opendaylight-cluster-data@10.30.170.225:2550",
				"akka://opendaylight-cluster-data@10.30.170.154:2550"]

      roles = ["member-2"]

      # when under load we might trip a false positive on the failure detector
      # failure-detector {
        # heartbeat-interval = 4 s
        # acceptable-heartbeat-pause = 16s
      # }
    }

    persistence {
      # By default the snapshots/journal directories live in KARAF_HOME. You can choose to put it somewhere else by
      # modifying the following two properties. The directory location specified may be a relative or absolute path. 
      # The relative path is always relative to KARAF_HOME.

      # snapshot-store.local.dir = "target/snapshots"

      # Use lz4 compression for LocalSnapshotStore snapshots
      snapshot-store.local.use-lz4-compression = false
      # Size of blocks for lz4 compression: 64KB, 256KB, 1MB or 4MB
      snapshot-store.local.lz4-blocksize = 256KB
    }
    disable-default-actor-system-quarantined-event-handling = "false"
  }
}
Dump modules.conf
+ echo 'Dump modules.conf'
+ cat /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf
modules = [
	
	{
		name = "inventory"
		namespace = "urn:opendaylight:inventory"
		shard-strategy = "module"
	},
	{
		name = "topology"
		namespace = "urn:TBD:params:xml:ns:yang:network-topology"
		shard-strategy = "module"
	},
	{
		name = "toaster"
		namespace = "http://netconfcentral.org/ns/toaster"
		shard-strategy = "module"
	},
	{
		name = "car"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car"
		shard-strategy = "module"
	},
	{
		name = "people"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people"
		shard-strategy = "module"
	},
	{
		name = "car-people"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people"
		shard-strategy = "module"
	}
]
Dump module-shards.conf
+ echo 'Dump module-shards.conf'
+ cat /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf
module-shards = [
	{
		name = "default"
		shards = [
			{
				name = "default"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "inventory"
		shards = [
			{
				name="inventory"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "topology"
		shards = [
			{
				name="topology"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "toaster"
		shards = [
			{
				name="toaster"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "car"
		shards = [
			{
				name="car"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "people"
		shards = [
			{
				name="people"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "car-people"
		shards = [
			{
				name="car-people"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	}
]
Configuring member-3 with IP address 10.30.170.154
Warning: Permanently added '10.30.170.154' (ECDSA) to the list of known hosts.
Warning: Permanently added '10.30.170.154' (ECDSA) to the list of known hosts.
common-functions.sh is being sourced
+ source /tmp/common-functions.sh karaf-0.20.2-SNAPSHOT
++ [[ /tmp/common-functions.sh == \/\t\m\p\/\c\o\n\f\i\g\u\r\a\t\i\o\n\-\s\c\r\i\p\t\.\s\h ]]
++ echo 'common-functions.sh is being sourced'
++ BUNDLEFOLDER=karaf-0.20.2-SNAPSHOT
++ export MAVENCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
++ MAVENCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
++ export FEATURESCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg
++ FEATURESCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg
++ export CUSTOMPROP=/tmp/karaf-0.20.2-SNAPSHOT/etc/custom.properties
++ CUSTOMPROP=/tmp/karaf-0.20.2-SNAPSHOT/etc/custom.properties
++ export LOGCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
++ LOGCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
++ export MEMCONF=/tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
++ MEMCONF=/tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
++ export CONTROLLERMEM=
++ CONTROLLERMEM=
++ export AKKACONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf
++ AKKACONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf
++ export MODULESCONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf
++ MODULESCONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf
++ export MODULESHARDSCONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf
++ MODULESHARDSCONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf
++ print_common_env
++ cat
common-functions environment:
MAVENCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
ACTUALFEATURES: 
FEATURESCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg
CUSTOMPROP: /tmp/karaf-0.20.2-SNAPSHOT/etc/custom.properties
LOGCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
MEMCONF: /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
CONTROLLERMEM: 
AKKACONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf
MODULESCONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf
MODULESHARDSCONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf
SUITES: 

++ SSH='ssh -t -t'
++ extra_services_cntl='     dnsmasq.service     httpd.service     libvirtd.service     openvswitch.service     ovs-vswitchd.service     ovsdb-server.service     rabbitmq-server.service '
++ extra_services_cmp='     libvirtd.service     openvswitch.service     ovs-vswitchd.service     ovsdb-server.service '
Changing to /tmp
Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240713.204807-83.zip
+ echo 'Changing to /tmp'
+ cd /tmp
+ echo 'Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240713.204807-83.zip'
+ wget --progress=dot:mega https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240713.204807-83.zip
--2024-07-13 23:40:50--  https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240713.204807-83.zip
Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6
Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 268186747 (256M) [application/zip]
Saving to: ‘karaf-0.20.2-20240713.204807-83.zip’

     0K ........ ........ ........ ........ ........ ........  1% 57.6M 4s
  3072K ........ ........ ........ ........ ........ ........  2%  158M 3s
  6144K ........ ........ ........ ........ ........ ........  3%  171M 2s
  9216K ........ ........ ........ ........ ........ ........  4%  174M 2s
 12288K ........ ........ ........ ........ ........ ........  5%  219M 2s
 15360K ........ ........ ........ ........ ........ ........  7%  173M 2s
 18432K ........ ........ ........ ........ ........ ........  8%  188M 2s
 21504K ........ ........ ........ ........ ........ ........  9%  197M 2s
 24576K ........ ........ ........ ........ ........ ........ 10%  162M 2s
 27648K ........ ........ ........ ........ ........ ........ 11%  195M 2s
 30720K ........ ........ ........ ........ ........ ........ 12%  236M 1s
 33792K ........ ........ ........ ........ ........ ........ 14%  234M 1s
 36864K ........ ........ ........ ........ ........ ........ 15%  316M 1s
 39936K ........ ........ ........ ........ ........ ........ 16%  272M 1s
 43008K ........ ........ ........ ........ ........ ........ 17%  214M 1s
 46080K ........ ........ ........ ........ ........ ........ 18%  257M 1s
 49152K ........ ........ ........ ........ ........ ........ 19%  348M 1s
 52224K ........ ........ ........ ........ ........ ........ 21%  283M 1s
 55296K ........ ........ ........ ........ ........ ........ 22%  307M 1s
 58368K ........ ........ ........ ........ ........ ........ 23%  201M 1s
 61440K ........ ........ ........ ........ ........ ........ 24%  356M 1s
 64512K ........ ........ ........ ........ ........ ........ 25%  221M 1s
 67584K ........ ........ ........ ........ ........ ........ 26%  271M 1s
 70656K ........ ........ ........ ........ ........ ........ 28%  300M 1s
 73728K ........ ........ ........ ........ ........ ........ 29%  276M 1s
 76800K ........ ........ ........ ........ ........ ........ 30%  305M 1s
 79872K ........ ........ ........ ........ ........ ........ 31%  305M 1s
 82944K ........ ........ ........ ........ ........ ........ 32%  302M 1s
 86016K ........ ........ ........ ........ ........ ........ 34%  294M 1s
 89088K ........ ........ ........ ........ ........ ........ 35%  291M 1s
 92160K ........ ........ ........ ........ ........ ........ 36%  289M 1s
 95232K ........ ........ ........ ........ ........ ........ 37%  287M 1s
 98304K ........ ........ ........ ........ ........ ........ 38%  289M 1s
101376K ........ ........ ........ ........ ........ ........ 39%  233M 1s
104448K ........ ........ ........ ........ ........ ........ 41%  272M 1s
107520K ........ ........ ........ ........ ........ ........ 42%  283M 1s
110592K ........ ........ ........ ........ ........ ........ 43%  281M 1s
113664K ........ ........ ........ ........ ........ ........ 44%  275M 1s
116736K ........ ........ ........ ........ ........ ........ 45%  284M 1s
119808K ........ ........ ........ ........ ........ ........ 46%  291M 1s
122880K ........ ........ ........ ........ ........ ........ 48%  273M 1s
125952K ........ ........ ........ ........ ........ ........ 49%  304M 1s
129024K ........ ........ ........ ........ ........ ........ 50%  286M 1s
132096K ........ ........ ........ ........ ........ ........ 51%  281M 1s
135168K ........ ........ ........ ........ ........ ........ 52%  292M 1s
138240K ........ ........ ........ ........ ........ ........ 53%  289M 1s
141312K ........ ........ ........ ........ ........ ........ 55%  316M 0s
144384K ........ ........ ........ ........ ........ ........ 56%  291M 0s
147456K ........ ........ ........ ........ ........ ........ 57%  293M 0s
150528K ........ ........ ........ ........ ........ ........ 58%  249M 0s
153600K ........ ........ ........ ........ ........ ........ 59%  231M 0s
156672K ........ ........ ........ ........ ........ ........ 60%  291M 0s
159744K ........ ........ ........ ........ ........ ........ 62%  295M 0s
162816K ........ ........ ........ ........ ........ ........ 63%  305M 0s
165888K ........ ........ ........ ........ ........ ........ 64%  291M 0s
168960K ........ ........ ........ ........ ........ ........ 65%  300M 0s
172032K ........ ........ ........ ........ ........ ........ 66%  276M 0s
175104K ........ ........ ........ ........ ........ ........ 68%  300M 0s
178176K ........ ........ ........ ........ ........ ........ 69%  287M 0s
181248K ........ ........ ........ ........ ........ ........ 70%  276M 0s
184320K ........ ........ ........ ........ ........ ........ 71%  289M 0s
187392K ........ ........ ........ ........ ........ ........ 72%  289M 0s
190464K ........ ........ ........ ........ ........ ........ 73%  276M 0s
193536K ........ ........ ........ ........ ........ ........ 75%  292M 0s
196608K ........ ........ ........ ........ ........ ........ 76%  299M 0s
199680K ........ ........ ........ ........ ........ ........ 77%  299M 0s
202752K ........ ........ ........ ........ ........ ........ 78%  160M 0s
205824K ........ ........ ........ ........ ........ ........ 79%  292M 0s
208896K ........ ........ ........ ........ ........ ........ 80%  297M 0s
211968K ........ ........ ........ ........ ........ ........ 82%  394M 0s
215040K ........ ........ ........ ........ ........ ........ 83%  377M 0s
218112K ........ ........ ........ ........ ........ ........ 84%  371M 0s
221184K ........ ........ ........ ........ ........ ........ 85%  314M 0s
224256K ........ ........ ........ ........ ........ ........ 86%  301M 0s
227328K ........ ........ ........ ........ ........ ........ 87%  292M 0s
230400K ........ ........ ........ ........ ........ ........ 89%  292M 0s
233472K ........ ........ ........ ........ ........ ........ 90%  297M 0s
236544K ........ ........ ........ ........ ........ ........ 91%  298M 0s
239616K ........ ........ ........ ........ ........ ........ 92%  312M 0s
242688K ........ ........ ........ ........ ........ ........ 93%  303M 0s
245760K ........ ........ ........ ........ ........ ........ 95%  299M 0s
248832K ........ ........ ........ ........ ........ ........ 96%  287M 0s
251904K ........ ........ ........ ........ ........ ........ 97%  301M 0s
254976K ........ ........ ........ ........ ........ ........ 98%  307M 0s
258048K ........ ........ ........ ........ ........ ........ 99%  296M 0s
261120K ........ ....                                        100%  311M=1.0s

2024-07-13 23:40:51 (256 MB/s) - ‘karaf-0.20.2-20240713.204807-83.zip’ saved [268186747/268186747]

Extracting the new controller...
+ echo 'Extracting the new controller...'
+ unzip -q karaf-0.20.2-20240713.204807-83.zip
Adding external repositories...
+ echo 'Adding external repositories...'
+ sed -ie 's%org.ops4j.pax.url.mvn.repositories=%org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases%g' /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
+ cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
################################################################################
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#
################################################################################

#
# If set to true, the following property will not allow any certificate to be used
# when accessing Maven repositories through SSL
#
#org.ops4j.pax.url.mvn.certificateCheck=

#
# Path to the local Maven settings file.
# The repositories defined in this file will be automatically added to the list
# of default repositories if the 'org.ops4j.pax.url.mvn.repositories' property
# below is not set.
# The following locations are checked for the existence of the settings.xml file
#   * 1. looks for the specified url
#   * 2. if not found looks for ${user.home}/.m2/settings.xml
#   * 3. if not found looks for ${maven.home}/conf/settings.xml
#   * 4. if not found looks for ${M2_HOME}/conf/settings.xml
#
#org.ops4j.pax.url.mvn.settings=

#
# Path to the local Maven repository which is used to avoid downloading
# artifacts when they already exist locally.
# The value of this property will be extracted from the settings.xml file
# above, or defaulted to:
#     System.getProperty( "user.home" ) + "/.m2/repository"
#
org.ops4j.pax.url.mvn.localRepository=${karaf.home}/${karaf.default.repository}

#
# Default this to false. It's just weird to use undocumented repos
#
org.ops4j.pax.url.mvn.useFallbackRepositories=false

#
# Uncomment if you don't wanna use the proxy settings
# from the Maven conf/settings.xml file
#
# org.ops4j.pax.url.mvn.proxySupport=false

#
# Comma separated list of repositories scanned when resolving an artifact.
# Those repositories will be checked before iterating through the
#    below list of repositories and even before the local repository
# A repository url can be appended with zero or more of the following flags:
#    @snapshots  : the repository contains snaphots
#    @noreleases : the repository does not contain any released artifacts
#
# The following property value will add the system folder as a repo.
#
org.ops4j.pax.url.mvn.defaultRepositories=\
    file:${karaf.home}/${karaf.default.repository}@id=system.repository@snapshots,\
    file:${karaf.data}/kar@id=kar.repository@multi@snapshots,\
    file:${karaf.base}/${karaf.default.repository}@id=child.system.repository@snapshots

# Use the default local repo (e.g.~/.m2/repository) as a "remote" repo
#org.ops4j.pax.url.mvn.defaultLocalRepoAsRemote=false

#
# Comma separated list of repositories scanned when resolving an artifact.
# The default list includes the following repositories:
#    http://repo1.maven.org/maven2@id=central
#    http://repository.springsource.com/maven/bundles/release@id=spring.ebr
#    http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external
#    http://zodiac.springsource.com/maven/bundles/release@id=gemini
#    http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases
#    https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases
#    https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases
# To add repositories to the default ones, prepend '+' to the list of repositories
# to add.
# A repository url can be appended with zero or more of the following flags:
#    @snapshots  : the repository contains snapshots
#    @noreleases : the repository does not contain any released artifacts
#    @id=repository.id : the id for the repository, just like in the settings.xml this is optional but recommended
#
org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases

### ^^^ No remote repositories. This is the only ODL change compared to Karaf defaults.+ [[ True == \T\r\u\e ]]
+ echo 'Configuring the startup features...'
Configuring the startup features...
+ sed -ie 's/\(featuresBoot=\|featuresBoot =\)/featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app,/g' /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg
+ FEATURE_TEST_STRING=features-test
+ FEATURE_TEST_VERSION=0.20.2-SNAPSHOT
+ KARAF_VERSION=karaf4
+ [[ integration == \i\n\t\e\g\r\a\t\i\o\n ]]
+ sed -ie 's%\(featuresRepositories=\|featuresRepositories =\)%featuresRepositories = mvn:org.opendaylight.integration/features-test/0.20.2-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features,%g' /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg
+ [[ ! -z '' ]]
+ cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg
################################################################################
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#
################################################################################

#
# Comma separated list of features repositories to register by default
#
featuresRepositories = mvn:org.opendaylight.integration/features-test/0.20.2-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features, file:${karaf.etc}/36d1a0ea-5dbf-413c-8167-c4d0fdd5e797.xml

#
# Comma separated list of features to install at startup
#
featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app, 04c66cf3-c01b-4f7e-8c33-e3176c80f161

#
# Resource repositories (OBR) that the features resolver can use
# to resolve requirements/capabilities
#
# The format of the resourceRepositories is 
# resourceRepositories=[xml:url|json:url],...
# for Instance:
#
#resourceRepositories=xml:http://host/path/to/index.xml
# or
#resourceRepositories=json:http://host/path/to/index.json
#

#
# Defines if the boot features are started in asynchronous mode (in a dedicated thread)
#
featuresBootAsynchronous=false

#
# Service requirements enforcement
#
# By default, the feature resolver checks the service requirements/capabilities of
# bundles for new features (xml schema >= 1.3.0) in order to automatically installs
# the required bundles.
# The following flag can have those values:
#   - disable: service requirements are completely ignored
#   - default: service requirements are ignored for old features
#   - enforce: service requirements are always verified
#
#serviceRequirements=default

#
# Store cfg file for config element in feature
#
#configCfgStore=true

#
# Define if the feature service automatically refresh bundles
#
autoRefresh=true

#
# Configuration of features processing mechanism (overrides, blacklisting, modification of features)
# XML file defines instructions related to features processing
# versions.properties may declare properties to resolve placeholders in XML file
# both files are relative to ${karaf.etc}
#
#featureProcessing=org.apache.karaf.features.xml
#featureProcessingVersions=versions.properties
+ configure_karaf_log karaf4 ''
+ local -r karaf_version=karaf4
+ local -r controllerdebugmap=
+ local logapi=log4j
+ grep log4j2 /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n
log4j2.rootLogger.level = INFO
#log4j2.rootLogger.type = asyncRoot
#log4j2.rootLogger.includeLocation = false
log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile
log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi
log4j2.rootLogger.appenderRef.Console.ref = Console
log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter
log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF}
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL
log4j2.logger.spifly.name = org.apache.aries.spifly
log4j2.logger.spifly.level = WARN
log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit
log4j2.logger.audit.level = INFO
log4j2.logger.audit.additivity = false
log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile
# Console appender not used by default (see log4j2.rootLogger.appenderRefs)
log4j2.appender.console.type = Console
log4j2.appender.console.name = Console
log4j2.appender.console.layout.type = PatternLayout
log4j2.appender.console.layout.pattern = ${log4j2.pattern}
log4j2.appender.rolling.type = RollingRandomAccessFile
log4j2.appender.rolling.name = RollingFile
log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i
#log4j2.appender.rolling.immediateFlush = false
log4j2.appender.rolling.append = true
log4j2.appender.rolling.layout.type = PatternLayout
log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
log4j2.appender.rolling.policies.type = Policies
log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.rolling.policies.size.size = 64MB
log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
log4j2.appender.rolling.strategy.max = 7
log4j2.appender.audit.type = RollingRandomAccessFile
log4j2.appender.audit.name = AuditRollingFile
log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log
log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i
log4j2.appender.audit.append = true
log4j2.appender.audit.layout.type = PatternLayout
log4j2.appender.audit.layout.pattern = ${log4j2.pattern}
log4j2.appender.audit.policies.type = Policies
log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.audit.policies.size.size = 8MB
log4j2.appender.audit.strategy.type = DefaultRolloverStrategy
log4j2.appender.audit.strategy.max = 7
log4j2.appender.osgi.type = PaxOsgi
log4j2.appender.osgi.name = PaxOsgi
log4j2.appender.osgi.filter = *
#log4j2.logger.aether.name = shaded.org.eclipse.aether
#log4j2.logger.aether.level = TRACE
#log4j2.logger.http-headers.name = shaded.org.apache.http.headers
#log4j2.logger.http-headers.level = DEBUG
#log4j2.logger.maven.name = org.ops4j.pax.url.mvn
#log4j2.logger.maven.level = TRACE
Configuring the karaf log... karaf_version: karaf4, logapi: log4j2
+ logapi=log4j2
+ echo 'Configuring the karaf log... karaf_version: karaf4, logapi: log4j2'
+ '[' log4j2 == log4j2 ']'
+ sed -ie 's/log4j2.appender.rolling.policies.size.size = 64MB/log4j2.appender.rolling.policies.size.size = 1GB/g' /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
+ orgmodule=org.opendaylight.yangtools.yang.parser.repo.YangTextSchemaContextResolver
+ orgmodule_=org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver
+ echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN'
+ echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN'
controllerdebugmap: 
+ unset IFS
+ echo 'controllerdebugmap: '
cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
+ '[' -n '' ']'
+ echo 'cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg'
+ cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
################################################################################
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#
################################################################################

# Common pattern layout for appenders
log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n

# Root logger
log4j2.rootLogger.level = INFO
# uncomment to use asynchronous loggers, which require mvn:com.lmax/disruptor/3.3.2 library
#log4j2.rootLogger.type = asyncRoot
#log4j2.rootLogger.includeLocation = false
log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile
log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi
log4j2.rootLogger.appenderRef.Console.ref = Console
log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter
log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF}

# Filters for logs marked by org.opendaylight.odlparent.Markers
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY
log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL

# Loggers configuration

# Spifly logger
log4j2.logger.spifly.name = org.apache.aries.spifly
log4j2.logger.spifly.level = WARN

# Security audit logger
log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit
log4j2.logger.audit.level = INFO
log4j2.logger.audit.additivity = false
log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile

# Appenders configuration

# Console appender not used by default (see log4j2.rootLogger.appenderRefs)
log4j2.appender.console.type = Console
log4j2.appender.console.name = Console
log4j2.appender.console.layout.type = PatternLayout
log4j2.appender.console.layout.pattern = ${log4j2.pattern}

# Rolling file appender
log4j2.appender.rolling.type = RollingRandomAccessFile
log4j2.appender.rolling.name = RollingFile
log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log
log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i
# uncomment to not force a disk flush
#log4j2.appender.rolling.immediateFlush = false
log4j2.appender.rolling.append = true
log4j2.appender.rolling.layout.type = PatternLayout
log4j2.appender.rolling.layout.pattern = ${log4j2.pattern}
log4j2.appender.rolling.policies.type = Policies
log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.rolling.policies.size.size = 1GB
log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy
log4j2.appender.rolling.strategy.max = 7

# Audit file appender
log4j2.appender.audit.type = RollingRandomAccessFile
log4j2.appender.audit.name = AuditRollingFile
log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log
log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i
log4j2.appender.audit.append = true
log4j2.appender.audit.layout.type = PatternLayout
log4j2.appender.audit.layout.pattern = ${log4j2.pattern}
log4j2.appender.audit.policies.type = Policies
log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy
log4j2.appender.audit.policies.size.size = 8MB
log4j2.appender.audit.strategy.type = DefaultRolloverStrategy
log4j2.appender.audit.strategy.max = 7

# OSGi appender
log4j2.appender.osgi.type = PaxOsgi
log4j2.appender.osgi.name = PaxOsgi
log4j2.appender.osgi.filter = *

# help with identification of maven-related problems with pax-url-aether
#log4j2.logger.aether.name = shaded.org.eclipse.aether
#log4j2.logger.aether.level = TRACE
#log4j2.logger.http-headers.name = shaded.org.apache.http.headers
#log4j2.logger.http-headers.level = DEBUG
#log4j2.logger.maven.name = org.ops4j.pax.url.mvn
#log4j2.logger.maven.level = TRACE
log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN
log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN
+ set_java_vars /usr/lib/jvm/java-17-openjdk 3072m /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
Configure
    java home: /usr/lib/jvm/java-17-openjdk
+ local -r java_home=/usr/lib/jvm/java-17-openjdk
+ local -r controllermem=3072m
+ local -r memconf=/tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
+ echo Configure
+ echo '    java home: /usr/lib/jvm/java-17-openjdk'
    max memory: 3072m
    memconf: /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
+ echo '    max memory: 3072m'
+ echo '    memconf: /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv'
+ sed -ie 's%^# export JAVA_HOME%export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-17-openjdk}%g' /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
+ sed -ie 's/JAVA_MAX_MEM="2048m"/JAVA_MAX_MEM=3072m/g' /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
cat /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
+ echo 'cat /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv'
+ cat /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
#!/bin/sh
#
#    Licensed to the Apache Software Foundation (ASF) under one or more
#    contributor license agreements.  See the NOTICE file distributed with
#    this work for additional information regarding copyright ownership.
#    The ASF licenses this file to You under the Apache License, Version 2.0
#    (the "License"); you may not use this file except in compliance with
#    the License.  You may obtain a copy of the License at
#
#       http://www.apache.org/licenses/LICENSE-2.0
#
#    Unless required by applicable law or agreed to in writing, software
#    distributed under the License is distributed on an "AS IS" BASIS,
#    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
#    See the License for the specific language governing permissions and
#    limitations under the License.
#

#
# handle specific scripts; the SCRIPT_NAME is exactly the name of the Karaf
# script: client, instance, shell, start, status, stop, karaf
#
# if [ "${KARAF_SCRIPT}" == "SCRIPT_NAME" ]; then
#   Actions go here...
# fi

#
# general settings which should be applied for all scripts go here; please keep
# in mind that it is possible that scripts might be executed more than once, e.g.
# in example of the start script where the start script is executed first and the
# karaf script afterwards.
#

#
# The following section shows the possible configuration options for the default 
# karaf scripts
#
export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-17-openjdk} # Location of Java installation
# export JAVA_OPTS # Generic JVM options, for instance, where you can pass the memory configuration
# export JAVA_NON_DEBUG_OPTS # Additional non-debug JVM options
# export EXTRA_JAVA_OPTS # Additional JVM options
# export KARAF_HOME # Karaf home folder
# export KARAF_DATA # Karaf data folder
# export KARAF_BASE # Karaf base folder
# export KARAF_ETC  # Karaf etc  folder
# export KARAF_LOG  # Karaf log  folder
# export KARAF_SYSTEM_OPTS # First citizen Karaf options
# export KARAF_OPTS # Additional available Karaf options
# export KARAF_DEBUG # Enable debug mode
# export KARAF_REDIRECT # Enable/set the std/err redirection when using bin/start
# export KARAF_NOROOT # Prevent execution as root if set to true
+ echo 'Set Java version'
Set Java version
+ sudo /usr/sbin/alternatives --install /usr/bin/java java /usr/lib/jvm/java-17-openjdk/bin/java 1
+ sudo /usr/sbin/alternatives --set java /usr/lib/jvm/java-17-openjdk/bin/java
JDK default version ...
+ echo 'JDK default version ...'
+ java -version
openjdk version "17.0.6-ea" 2023-01-17 LTS
OpenJDK Runtime Environment (Red_Hat-17.0.6.0.9-0.3.ea.el8) (build 17.0.6-ea+9-LTS)
OpenJDK 64-Bit Server VM (Red_Hat-17.0.6.0.9-0.3.ea.el8) (build 17.0.6-ea+9-LTS, mixed mode, sharing)
Set JAVA_HOME
+ echo 'Set JAVA_HOME'
+ export JAVA_HOME=/usr/lib/jvm/java-17-openjdk
+ JAVA_HOME=/usr/lib/jvm/java-17-openjdk
++ readlink -e /usr/lib/jvm/java-17-openjdk/bin/java
Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin/java
+ JAVA_RESOLVED=/usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin/java
+ echo 'Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin/java'
Listing all open ports on controller system...
+ echo 'Listing all open ports on controller system...'
+ netstat -pnatu
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      -                   
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -                   
tcp        0      0 10.30.170.154:22        10.30.170.213:46804     ESTABLISHED -                   
tcp        0      0 10.30.170.154:35064     199.204.45.87:443       TIME_WAIT   -                   
tcp6       0      0 :::111                  :::*                    LISTEN      -                   
tcp6       0      0 :::5555                 :::*                    LISTEN      -                   
tcp6       0      0 :::22                   :::*                    LISTEN      -                   
udp        0      0 10.30.170.154:68        10.30.170.2:67          ESTABLISHED -                   
udp        0      0 0.0.0.0:111             0.0.0.0:*                           -                   
udp        0      0 127.0.0.1:323           0.0.0.0:*                           -                   
udp6       0      0 :::111                  :::*                                -                   
udp6       0      0 ::1:323                 :::*                                -                   
Custom shard config exists!!!
Copying the shard config...
+ '[' -f /tmp/custom_shard_config.txt ']'
+ echo 'Custom shard config exists!!!'
+ echo 'Copying the shard config...'
+ cp /tmp/custom_shard_config.txt /tmp/karaf-0.20.2-SNAPSHOT/bin/
Configuring cluster
+ echo 'Configuring cluster'
+ /tmp/karaf-0.20.2-SNAPSHOT/bin/configure_cluster.sh 3 10.30.171.141 10.30.170.225 10.30.170.154
################################################
##             Configure Cluster              ##
################################################
 NOTE: Cluster configuration files not found. Copying from
 /tmp/karaf-0.20.2-SNAPSHOT/system/org/opendaylight/controller/sal-clustering-config/9.0.4
Configuring unique name in akka.conf
Configuring hostname in akka.conf
Configuring data and rpc seed nodes in akka.conf
modules = [
	
	{
		name = "inventory"
		namespace = "urn:opendaylight:inventory"
		shard-strategy = "module"
	},
	{
		name = "topology"
		namespace = "urn:TBD:params:xml:ns:yang:network-topology"
		shard-strategy = "module"
	},
	{
		name = "toaster"
		namespace = "http://netconfcentral.org/ns/toaster"
		shard-strategy = "module"
	},
	{
		name = "car"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car"
		shard-strategy = "module"
	},
	{
		name = "people"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people"
		shard-strategy = "module"
	},
	{
		name = "car-people"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people"
		shard-strategy = "module"
	}
]
Configuring replication type in module-shards.conf
################################################
##   NOTE: Manually restart controller to     ##
##         apply configuration.               ##
################################################
+ echo 'Dump akka.conf'
Dump akka.conf
+ cat /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf

odl-cluster-data {
  akka {
    remote {
      artery {
        enabled = on
        transport = tcp
        canonical.hostname = "10.30.170.154"
        canonical.port = 2550
      }
    }

    cluster {
      # Using artery.
      seed-nodes = ["akka://opendaylight-cluster-data@10.30.171.141:2550",
				"akka://opendaylight-cluster-data@10.30.170.225:2550",
				"akka://opendaylight-cluster-data@10.30.170.154:2550"]

      roles = ["member-3"]

      # when under load we might trip a false positive on the failure detector
      # failure-detector {
        # heartbeat-interval = 4 s
        # acceptable-heartbeat-pause = 16s
      # }
    }

    persistence {
      # By default the snapshots/journal directories live in KARAF_HOME. You can choose to put it somewhere else by
      # modifying the following two properties. The directory location specified may be a relative or absolute path. 
      # The relative path is always relative to KARAF_HOME.

      # snapshot-store.local.dir = "target/snapshots"

      # Use lz4 compression for LocalSnapshotStore snapshots
      snapshot-store.local.use-lz4-compression = false
      # Size of blocks for lz4 compression: 64KB, 256KB, 1MB or 4MB
      snapshot-store.local.lz4-blocksize = 256KB
    }
    disable-default-actor-system-quarantined-event-handling = "false"
  }
}
Dump modules.conf
+ echo 'Dump modules.conf'
+ cat /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf
modules = [
	
	{
		name = "inventory"
		namespace = "urn:opendaylight:inventory"
		shard-strategy = "module"
	},
	{
		name = "topology"
		namespace = "urn:TBD:params:xml:ns:yang:network-topology"
		shard-strategy = "module"
	},
	{
		name = "toaster"
		namespace = "http://netconfcentral.org/ns/toaster"
		shard-strategy = "module"
	},
	{
		name = "car"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car"
		shard-strategy = "module"
	},
	{
		name = "people"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people"
		shard-strategy = "module"
	},
	{
		name = "car-people"
		namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people"
		shard-strategy = "module"
	}
]
Dump module-shards.conf
+ echo 'Dump module-shards.conf'
+ cat /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf
module-shards = [
	{
		name = "default"
		shards = [
			{
				name = "default"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "inventory"
		shards = [
			{
				name="inventory"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "topology"
		shards = [
			{
				name="topology"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "toaster"
		shards = [
			{
				name="toaster"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "car"
		shards = [
			{
				name="car"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "people"
		shards = [
			{
				name="people"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	},
	{
		name = "car-people"
		shards = [
			{
				name="car-people"
				replicas = ["member-1",
				"member-2",
				"member-3"]
			}
		]
	}
]
Locating config plan to use...
Finished running config plans
Starting member-1 with IP address 10.30.171.141
Warning: Permanently added '10.30.171.141' (ECDSA) to the list of known hosts.
Warning: Permanently added '10.30.171.141' (ECDSA) to the list of known hosts.
Redirecting karaf console output to karaf_console.log
Starting controller...
start: Redirecting Karaf output to /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf_console.log
Starting member-2 with IP address 10.30.170.225
Warning: Permanently added '10.30.170.225' (ECDSA) to the list of known hosts.
Warning: Permanently added '10.30.170.225' (ECDSA) to the list of known hosts.
Redirecting karaf console output to karaf_console.log
Starting controller...
start: Redirecting Karaf output to /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf_console.log
Starting member-3 with IP address 10.30.170.154
Warning: Permanently added '10.30.170.154' (ECDSA) to the list of known hosts.
Warning: Permanently added '10.30.170.154' (ECDSA) to the list of known hosts.
Redirecting karaf console output to karaf_console.log
Starting controller...
start: Redirecting Karaf output to /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf_console.log
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins5763766447341094221.sh
common-functions.sh is being sourced
common-functions environment:
MAVENCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg
ACTUALFEATURES: 
FEATURESCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg
CUSTOMPROP: /tmp/karaf-0.20.2-SNAPSHOT/etc/custom.properties
LOGCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg
MEMCONF: /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv
CONTROLLERMEM: 2048m
AKKACONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf
MODULESCONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf
MODULESHARDSCONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf
SUITES: 

+ echo '#################################################'
#################################################
+ echo '##         Verify Cluster is UP                ##'
##         Verify Cluster is UP                ##
+ echo '#################################################'
#################################################
+ create_post_startup_script
+ cat
+ copy_and_run_post_startup_script
+ seed_index=1
++ seq 1 3
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_1_IP
+ echo 'Execute the post startup script on controller 10.30.171.141'
Execute the post startup script on controller 10.30.171.141
+ scp /w/workspace/controller-csit-3node-clustering-tell-all-calcium/post-startup-script.sh 10.30.171.141:/tmp/
Warning: Permanently added '10.30.171.141' (ECDSA) to the list of known hosts.
+ ssh 10.30.171.141 'bash /tmp/post-startup-script.sh 1'
Warning: Permanently added '10.30.171.141' (ECDSA) to the list of known hosts.
tcp6       0      0 :::8101                 :::*                    LISTEN     
Waiting up to 3 minutes for controller to come up, checking every 5 seconds...
2024-07-13T23:41:53,858 | INFO  | SystemReadyService-0 | SimpleSystemReadyMonitor         | 299 - org.opendaylight.infrautils.ready-api - 6.0.9 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.]
Controller is UP
2024-07-13T23:41:53,858 | INFO  | SystemReadyService-0 | SimpleSystemReadyMonitor         | 299 - org.opendaylight.infrautils.ready-api - 6.0.9 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.]
Listing all open ports on controller system...
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -                   
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      -                   
tcp        0    164 10.30.171.141:22        10.30.170.213:35190     ESTABLISHED -                   
tcp6       0      0 :::6640                 :::*                    LISTEN      5757/java           
tcp6       0      0 :::5555                 :::*                    LISTEN      -                   
tcp6       0      0 :::8181                 :::*                    LISTEN      5757/java           
tcp6       0      0 10.30.171.141:2550      :::*                    LISTEN      5757/java           
tcp6       0      0 :::22                   :::*                    LISTEN      -                   
tcp6       0      0 127.0.0.1:44444         :::*                    LISTEN      5757/java           
tcp6       0      0 :::6653                 :::*                    LISTEN      5757/java           
tcp6       0      0 :::4189                 :::*                    LISTEN      5757/java           
tcp6       0      0 :::1790                 :::*                    LISTEN      5757/java           
tcp6       0      0 :::8101                 :::*                    LISTEN      5757/java           
tcp6       0      0 :::6633                 :::*                    LISTEN      5757/java           
tcp6       0      0 127.0.0.1:1099          :::*                    LISTEN      5757/java           
tcp6       0      0 127.0.0.1:43919         :::*                    LISTEN      5757/java           
tcp6       0      0 :::111                  :::*                    LISTEN      -                   
tcp6       0      0 10.30.171.141:2550      10.30.170.225:56134     ESTABLISHED 5757/java           
tcp6      32      0 10.30.171.141:59268     199.204.45.87:443       CLOSE_WAIT  5757/java           
tcp6       0      0 10.30.171.141:52830     10.30.170.225:2550      ESTABLISHED 5757/java           
tcp6       0      0 10.30.171.141:36846     10.30.170.154:2550      ESTABLISHED 5757/java           
tcp6       0      0 10.30.171.141:2550      10.30.170.225:56124     ESTABLISHED 5757/java           
tcp6       0      0 127.0.0.1:55940         127.0.0.1:44444         TIME_WAIT   -                   
tcp6       0      0 10.30.171.141:2550      10.30.170.154:42640     ESTABLISHED 5757/java           
tcp6       0      0 10.30.171.141:36832     10.30.170.154:2550      ESTABLISHED 5757/java           
tcp6       0      0 10.30.171.141:52822     10.30.170.225:2550      ESTABLISHED 5757/java           
tcp6       0      0 127.0.0.1:54886         127.0.0.1:1099          TIME_WAIT   -                   
tcp6       0      0 127.0.0.1:54894         127.0.0.1:1099          TIME_WAIT   -                   
tcp6       0      0 10.30.171.141:2550      10.30.170.154:42646     ESTABLISHED 5757/java           
tcp6       0      0 127.0.0.1:55928         127.0.0.1:44444         TIME_WAIT   -                   
udp        0      0 10.30.171.141:68        10.30.170.2:67          ESTABLISHED -                   
udp        0      0 0.0.0.0:111             0.0.0.0:*                           -                   
udp        0      0 127.0.0.1:323           0.0.0.0:*                           -                   
udp6       0      0 :::4342                 :::*                                5757/java           
udp6       0      0 :::111                  :::*                                -                   
udp6       0      0 ::1:323                 :::*                                -                   
looking for "BindException: Address already in use" in log file
looking for "server is unhealthy" in log file
+ '[' 1 == 0 ']'
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_2_IP
+ echo 'Execute the post startup script on controller 10.30.170.225'
Execute the post startup script on controller 10.30.170.225
+ scp /w/workspace/controller-csit-3node-clustering-tell-all-calcium/post-startup-script.sh 10.30.170.225:/tmp/
Warning: Permanently added '10.30.170.225' (ECDSA) to the list of known hosts.
+ ssh 10.30.170.225 'bash /tmp/post-startup-script.sh 2'
Warning: Permanently added '10.30.170.225' (ECDSA) to the list of known hosts.
tcp6       0      0 :::8101                 :::*                    LISTEN     
Waiting up to 3 minutes for controller to come up, checking every 5 seconds...
2024-07-13T23:41:54,807 | INFO  | SystemReadyService-0 | SimpleSystemReadyMonitor         | 299 - org.opendaylight.infrautils.ready-api - 6.0.9 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.]
Controller is UP
2024-07-13T23:41:54,807 | INFO  | SystemReadyService-0 | SimpleSystemReadyMonitor         | 299 - org.opendaylight.infrautils.ready-api - 6.0.9 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.]
Listing all open ports on controller system...
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      -                   
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -                   
tcp        0    164 10.30.170.225:22        10.30.170.213:54874     ESTABLISHED -                   
tcp6       0      0 127.0.0.1:39427         :::*                    LISTEN      5741/java           
tcp6       0      0 :::8101                 :::*                    LISTEN      5741/java           
tcp6       0      0 :::6633                 :::*                    LISTEN      5741/java           
tcp6       0      0 127.0.0.1:1099          :::*                    LISTEN      5741/java           
tcp6       0      0 :::111                  :::*                    LISTEN      -                   
tcp6       0      0 :::6640                 :::*                    LISTEN      5741/java           
tcp6       0      0 :::5555                 :::*                    LISTEN      -                   
tcp6       0      0 :::8181                 :::*                    LISTEN      5741/java           
tcp6       0      0 10.30.170.225:2550      :::*                    LISTEN      5741/java           
tcp6       0      0 :::22                   :::*                    LISTEN      -                   
tcp6       0      0 127.0.0.1:44444         :::*                    LISTEN      5741/java           
tcp6       0      0 :::6653                 :::*                    LISTEN      5741/java           
tcp6       0      0 :::1790                 :::*                    LISTEN      5741/java           
tcp6       0      0 10.30.170.225:59894     10.30.170.154:2550      ESTABLISHED 5741/java           
tcp6      32      0 10.30.170.225:54292     199.204.45.87:443       CLOSE_WAIT  5741/java           
tcp6       0      0 10.30.170.225:56124     10.30.171.141:2550      ESTABLISHED 5741/java           
tcp6       0      0 127.0.0.1:34178         127.0.0.1:44444         TIME_WAIT   -                   
tcp6       0      0 10.30.170.225:2550      10.30.171.141:52822     ESTABLISHED 5741/java           
tcp6    5133      0 10.30.170.225:2550      10.30.171.141:52830     ESTABLISHED 5741/java           
tcp6       0      0 10.30.170.225:56134     10.30.171.141:2550      ESTABLISHED 5741/java           
tcp6       0      0 127.0.0.1:47060         127.0.0.1:1099          TIME_WAIT   -                   
tcp6       0      0 127.0.0.1:34192         127.0.0.1:44444         TIME_WAIT   -                   
tcp6       0      0 10.30.170.225:59882     10.30.170.154:2550      ESTABLISHED 5741/java           
tcp6       0      0 10.30.170.225:2550      10.30.170.154:50940     ESTABLISHED 5741/java           
tcp6       0      0 10.30.170.225:2550      10.30.170.154:50954     ESTABLISHED 5741/java           
udp        0      0 10.30.170.225:68        10.30.170.3:67          ESTABLISHED -                   
udp        0      0 0.0.0.0:111             0.0.0.0:*                           -                   
udp        0      0 127.0.0.1:323           0.0.0.0:*                           -                   
udp6       0      0 :::4342                 :::*                                5741/java           
udp6       0      0 :::111                  :::*                                -                   
udp6       0      0 ::1:323                 :::*                                -                   
looking for "BindException: Address already in use" in log file
looking for "server is unhealthy" in log file
+ '[' 2 == 0 ']'
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_3_IP
+ echo 'Execute the post startup script on controller 10.30.170.154'
Execute the post startup script on controller 10.30.170.154
+ scp /w/workspace/controller-csit-3node-clustering-tell-all-calcium/post-startup-script.sh 10.30.170.154:/tmp/
Warning: Permanently added '10.30.170.154' (ECDSA) to the list of known hosts.
+ ssh 10.30.170.154 'bash /tmp/post-startup-script.sh 3'
Warning: Permanently added '10.30.170.154' (ECDSA) to the list of known hosts.
tcp6       0      0 :::8101                 :::*                    LISTEN     
Waiting up to 3 minutes for controller to come up, checking every 5 seconds...
2024-07-13T23:41:54,166 | INFO  | SystemReadyService-0 | SimpleSystemReadyMonitor         | 299 - org.opendaylight.infrautils.ready-api - 6.0.9 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.]
Controller is UP
2024-07-13T23:41:54,166 | INFO  | SystemReadyService-0 | SimpleSystemReadyMonitor         | 299 - org.opendaylight.infrautils.ready-api - 6.0.9 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.]
Listing all open ports on controller system...
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      -                   
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -                   
tcp        0    164 10.30.170.154:22        10.30.170.213:33050     ESTABLISHED -                   
tcp6       0      0 :::111                  :::*                    LISTEN      -                   
tcp6       0      0 :::6640                 :::*                    LISTEN      5737/java           
tcp6       0      0 :::5555                 :::*                    LISTEN      -                   
tcp6       0      0 :::8181                 :::*                    LISTEN      5737/java           
tcp6       0      0 10.30.170.154:2550      :::*                    LISTEN      5737/java           
tcp6       0      0 :::22                   :::*                    LISTEN      -                   
tcp6       0      0 127.0.0.1:44444         :::*                    LISTEN      5737/java           
tcp6       0      0 :::6653                 :::*                    LISTEN      5737/java           
tcp6       0      0 :::1790                 :::*                    LISTEN      5737/java           
tcp6       0      0 127.0.0.1:45311         :::*                    LISTEN      5737/java           
tcp6       0      0 :::8101                 :::*                    LISTEN      5737/java           
tcp6       0      0 :::6633                 :::*                    LISTEN      5737/java           
tcp6       0      0 127.0.0.1:1099          :::*                    LISTEN      5737/java           
tcp6      32      0 10.30.170.154:43948     199.204.45.87:443       CLOSE_WAIT  5737/java           
tcp6       0      0 10.30.170.154:2550      10.30.171.141:36832     ESTABLISHED 5737/java           
tcp6       0      0 10.30.170.154:42640     10.30.171.141:2550      ESTABLISHED 5737/java           
tcp6       0      0 127.0.0.1:43772         127.0.0.1:44444         TIME_WAIT   -                   
tcp6       0      0 10.30.170.154:2550      10.30.170.225:59882     ESTABLISHED 5737/java           
tcp6       0      0 10.30.170.154:42646     10.30.171.141:2550      ESTABLISHED 5737/java           
tcp6       0      0 10.30.170.154:50954     10.30.170.225:2550      ESTABLISHED 5737/java           
tcp6       0      0 10.30.170.154:2550      10.30.171.141:36846     ESTABLISHED 5737/java           
tcp6       0      0 127.0.0.1:43774         127.0.0.1:44444         TIME_WAIT   -                   
tcp6       0      0 10.30.170.154:2550      10.30.170.225:59894     ESTABLISHED 5737/java           
tcp6       0      0 127.0.0.1:60704         127.0.0.1:1099          TIME_WAIT   -                   
tcp6       0      0 10.30.170.154:50940     10.30.170.225:2550      ESTABLISHED 5737/java           
udp        0      0 10.30.170.154:68        10.30.170.2:67          ESTABLISHED -                   
udp        0      0 0.0.0.0:111             0.0.0.0:*                           -                   
udp        0      0 127.0.0.1:323           0.0.0.0:*                           -                   
udp6       0      0 :::111                  :::*                                -                   
udp6       0      0 ::1:323                 :::*                                -                   
udp6       0      0 :::4342                 :::*                                5737/java           
looking for "BindException: Address already in use" in log file
looking for "server is unhealthy" in log file
+ '[' 0 == 0 ']'
+ seed_index=1
+ dump_controller_threads
++ seq 1 3
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_1_IP
+ echo 'Let'\''s take the karaf thread dump'
Let's take the karaf thread dump
+ ssh 10.30.171.141 'sudo ps aux'
Warning: Permanently added '10.30.171.141' (ECDSA) to the list of known hosts.
++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-tell-all-calcium/ps_before.log
++ cut -f2 '-d '
++ grep -v grep
++ tr -s ' '
+ pid=5757
+ echo 'karaf main: org.apache.karaf.main.Main, pid:5757'
karaf main: org.apache.karaf.main.Main, pid:5757
+ ssh 10.30.171.141 '/usr/lib/jvm/java-17-openjdk/bin/jstack -l 5757'
Warning: Permanently added '10.30.171.141' (ECDSA) to the list of known hosts.
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_2_IP
+ echo 'Let'\''s take the karaf thread dump'
Let's take the karaf thread dump
+ ssh 10.30.170.225 'sudo ps aux'
Warning: Permanently added '10.30.170.225' (ECDSA) to the list of known hosts.
++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-tell-all-calcium/ps_before.log
++ grep -v grep
++ tr -s ' '
++ cut -f2 '-d '
+ pid=5741
+ echo 'karaf main: org.apache.karaf.main.Main, pid:5741'
karaf main: org.apache.karaf.main.Main, pid:5741
+ ssh 10.30.170.225 '/usr/lib/jvm/java-17-openjdk/bin/jstack -l 5741'
Warning: Permanently added '10.30.170.225' (ECDSA) to the list of known hosts.
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_3_IP
+ echo 'Let'\''s take the karaf thread dump'
Let's take the karaf thread dump
+ ssh 10.30.170.154 'sudo ps aux'
Warning: Permanently added '10.30.170.154' (ECDSA) to the list of known hosts.
++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-tell-all-calcium/ps_before.log
++ grep -v grep
++ cut -f2 '-d '
++ tr -s ' '
+ pid=5737
+ echo 'karaf main: org.apache.karaf.main.Main, pid:5737'
karaf main: org.apache.karaf.main.Main, pid:5737
+ ssh 10.30.170.154 '/usr/lib/jvm/java-17-openjdk/bin/jstack -l 5737'
Warning: Permanently added '10.30.170.154' (ECDSA) to the list of known hosts.
+ '[' 0 -gt 0 ']'
+ echo 'Generating controller variables...'
Generating controller variables...
++ seq 1 3
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_1_IP
+ odl_variables=' -v ODL_SYSTEM_1_IP:10.30.171.141'
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_2_IP
+ odl_variables=' -v ODL_SYSTEM_1_IP:10.30.171.141 -v ODL_SYSTEM_2_IP:10.30.170.225'
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_3_IP
+ odl_variables=' -v ODL_SYSTEM_1_IP:10.30.171.141 -v ODL_SYSTEM_2_IP:10.30.170.225 -v ODL_SYSTEM_3_IP:10.30.170.154'
+ echo 'Generating mininet variables...'
Generating mininet variables...
++ seq 1 0
+ get_test_suites SUITES
+ local __suite_list=SUITES
+ echo 'Locating test plan to use...'
Locating test plan to use...
+ testplan_filepath=/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/testplans/controller-clustering-tell-calcium.txt
+ '[' '!' -f /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/testplans/controller-clustering-tell-calcium.txt ']'
+ testplan_filepath=/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/testplans/controller-clustering-tell.txt
+ '[' disabled '!=' disabled ']'
+ echo 'Changing the testplan path...'
Changing the testplan path...
+ sed s:integration:/w/workspace/controller-csit-3node-clustering-tell-all-calcium: /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/testplans/controller-clustering-tell.txt
+ cat testplan.txt
# Place the suites in run order:

# disabling all "prefix based" suites. This functionality was newly
# introduced with these tests in 2017. The assumption is that
# development stopped in that area, but may continue at a later
# time. Still, the tests are failing and no resources to solve the
# failures at this time. If/When this changes, we can uncomment
# those suites. Note that every suite is commented out with it's
# accompanying "restart_with_tell_based_true" setup suite.


# /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
# /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/ddb-sanity-prefix-based.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/ddb-sanity-module-based.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/clean_leader_shutdown.robot
# /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
# /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/clean_leader_shutdown_prefbasedshard.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement.robot
# /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
# /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement_prefbasedshard.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/leader_isolation.robot
# /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
# /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/leader_isolation_prefbasedshard.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/client_isolation.robot
# /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
# /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/client_isolation_prefbasedshard.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/listener_stability.robot
# /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
# /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/listener_stability_prefbasedshard.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/master_stability.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot
/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot
+ '[' -z '' ']'
++ grep -E -v '(^[[:space:]]*#|^[[:space:]]*$)' testplan.txt
++ tr '\012' ' '
+ suite_list='/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/ddb-sanity-module-based.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/clean_leader_shutdown.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/leader_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/client_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/listener_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot '
+ eval 'SUITES='\''/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/ddb-sanity-module-based.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/clean_leader_shutdown.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/leader_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/client_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/listener_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot '\'''
++ SUITES='/w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/ddb-sanity-module-based.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/clean_leader_shutdown.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/leader_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/client_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/listener_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot '
+ echo 'Starting Robot test suites /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/ddb-sanity-module-based.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/clean_leader_shutdown.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/leader_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/client_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/listener_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot  ...'
Starting Robot test suites /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/ddb-sanity-module-based.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/clean_leader_shutdown.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/leader_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/client_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/listener_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot  ...
+ robot -N controller-clustering-tell.txt --removekeywords wuks -e exclude -e skip_if_calcium -v BUNDLEFOLDER:karaf-0.20.2-SNAPSHOT -v BUNDLE_URL:https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240713.204807-83.zip -v CONTROLLER:10.30.171.141 -v CONTROLLER1:10.30.170.225 -v CONTROLLER2:10.30.170.154 -v CONTROLLER_USER:jenkins -v JAVA_HOME:/usr/lib/jvm/java-17-openjdk -v JDKVERSION:openjdk17 -v JENKINS_WORKSPACE:/w/workspace/controller-csit-3node-clustering-tell-all-calcium -v MININET: -v MININET1: -v MININET2: -v MININET_USER:jenkins -v NEXUSURL_PREFIX:https://nexus.opendaylight.org -v NUM_ODL_SYSTEM:3 -v NUM_TOOLS_SYSTEM:0 -v ODL_STREAM:calcium -v ODL_SYSTEM_IP:10.30.171.141 -v ODL_SYSTEM_1_IP:10.30.171.141 -v ODL_SYSTEM_2_IP:10.30.170.225 -v ODL_SYSTEM_3_IP:10.30.170.154 -v ODL_SYSTEM_USER:jenkins -v TOOLS_SYSTEM_IP: -v TOOLS_SYSTEM_USER:jenkins -v USER_HOME:/home/jenkins -v IS_KARAF_APPL:True -v WORKSPACE:/tmp /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/ddb-sanity-module-based.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/clean_leader_shutdown.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/explicit_leader_movement.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/leader_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/client_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/listener_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_true.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-tell-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot
==============================================================================
controller-clustering-tell.txt                                                
==============================================================================
controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell...
==============================================================================
Stop_All_Members :: Stop every odl node.                              | PASS |
------------------------------------------------------------------------------
Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS |
------------------------------------------------------------------------------
Start_All_And_Sync :: Start each member and wait for sync.            | PASS |
------------------------------------------------------------------------------
controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | PASS |
3 tests, 3 passed, 0 failed
==============================================================================
controller-clustering-tell.txt.Ddb-Sanity-Module-Based :: DOMDataBroker tes...
==============================================================================
Get_Shard_Role :: Get shard role.                                     | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_ddb_sanity_module_based_get_shard_role"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_ddb_sanity_module_based_get_shard_role&order=bug_status"

HTTPError: 404 Client Error: Not Found for url: http://10.30.171.141:8181/restconf/operations/cluster-admin:get-shard-role
------------------------------------------------------------------------------
Subscribe_Listener_To_Leader :: Subscribe listener to leader.         | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_ddb_sanity_module_based_subscribe_listener_to_leader"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_ddb_sanity_module_based_subscribe_listener_to_leader&order=bug_status"

HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.170.225:8181/rests/operations/odl-mdsal-lowlevel-control:unsubscribe-dtcl
------------------------------------------------------------------------------
Subscribe_Listener_To_Follower :: Subscribe listener to follower.     | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_ddb_sanity_module_based_subscribe_listener_to_follower"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_ddb_sanity_module_based_subscribe_listener_to_follower&order=bug_status"

HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/operations/odl-mdsal-lowlevel-control:unsubscribe-dtcl
------------------------------------------------------------------------------
Make_Leader_Local :: Make the loeader local and verify.               | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_ddb_sanity_module_based_make_leader_local"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_ddb_sanity_module_based_make_leader_local&order=bug_status"

HTTPError: 404 Client Error: Not Found for url: http://10.30.171.141:8181/restconf/operations/cluster-admin:make-leader-local
------------------------------------------------------------------------------
Remove_Leader_Shard_Replica_And_Add_It_Back :: Remove and add shar... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_ddb_sanity_module_based_remove_leader_shard_replica_and_add_it_back"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_ddb_sanity_module_based_remove_leader_shard_replica_and_add_it_back&order=bug_status"

HTTPError: 404 Client Error: Not Found for url: http://10.30.170.225:8181/restconf/operations/cluster-admin:remove-shard-replica
------------------------------------------------------------------------------
Remove_Follower_Shard_Replica_And_Add_It_Back :: Remove and add sh... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_ddb_sanity_module_based_remove_follower_shard_replica_and_add_it_back"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_ddb_sanity_module_based_remove_follower_shard_replica_and_add_it_back&order=bug_status"

HTTPError: 404 Client Error: Not Found for url: http://10.30.171.141:8181/restconf/operations/cluster-admin:remove-shard-replica
------------------------------------------------------------------------------
Write_Transactions_One_Node_Leader :: Write transactions.             | PASS |
------------------------------------------------------------------------------
Write_Transactions_One_Node_Follower :: Write transactions.           | PASS |
------------------------------------------------------------------------------
Write_Transactions_All_Nodes :: Write transactions.                   | PASS |
------------------------------------------------------------------------------
controller-clustering-tell.txt.Ddb-Sanity-Module-Based :: DOMDataB... | FAIL |
9 tests, 3 passed, 6 failed
==============================================================================
controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell...
==============================================================================
Stop_All_Members :: Stop every odl node.                              | FAIL |
Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS |
------------------------------------------------------------------------------
Start_All_And_Sync :: Start each member and wait for sync.            | FAIL |
Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.154:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore
------------------------------------------------------------------------------
controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL |
3 tests, 1 passed, 2 failed
==============================================================================
controller-clustering-tell.txt.Clean Leader Shutdown :: DOMDataBroker testi...
==============================================================================
Local_Leader_Shutdown :: Shutdown the leader on the same node as t... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_clean_leader_shutdown_local_leader_shutdown"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_clean_leader_shutdown_local_leader_shutdown&order=bug_status"

HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.154:8181/jolokia/read/org.opendaylight.controller:Category=Shards,name=member-3-shard-default-config,type=DistributedConfigDatastore
------------------------------------------------------------------------------
Restart :: Restart odl                                                | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_clean_leader_shutdown_restart"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_clean_leader_shutdown_restart&order=bug_status"

Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Remote_Leader_Shutdown :: Shutdown the leader on different node as... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_clean_leader_shutdown_remote_leader_shutdown"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_clean_leader_shutdown_remote_leader_shutdown&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.141', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fce85237d10>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Clean Leader Shutdown :: DOMDataBro... | FAIL |
3 tests, 0 passed, 3 failed
==============================================================================
controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell...
==============================================================================
Stop_All_Members :: Stop every odl node.                              | FAIL |
Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS |
------------------------------------------------------------------------------
Start_All_And_Sync :: Start each member and wait for sync.            | FAIL |
Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.154:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore
------------------------------------------------------------------------------
controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL |
3 tests, 1 passed, 2 failed
==============================================================================
controller-clustering-tell.txt.Explicit Leader Movement :: DOMDataBroker te...
==============================================================================
Local_To_Remote_Movement :: Leader moves from local to remote node... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_explicit_leader_movement_local_to_remote_movement"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_explicit_leader_movement_local_to_remote_movement&order=bug_status"

HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.154:8181/jolokia/read/org.opendaylight.controller:Category=Shards,name=member-3-shard-default-config,type=DistributedConfigDatastore
------------------------------------------------------------------------------
Restart1 :: Restart odl                                               | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_explicit_leader_movement_restart1"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_explicit_leader_movement_restart1&order=bug_status"

Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Remote_To_Remote_Movement :: Leader moves from one remote to other... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_explicit_leader_movement_remote_to_remote_movement"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_explicit_leader_movement_remote_to_remote_movement&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.141', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fce84df8f90>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Restart2 :: Restart odl                                               | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_explicit_leader_movement_restart2"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_explicit_leader_movement_restart2&order=bug_status"

Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Remote_To_Local_Movement :: Leader moves from remote to local node... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_explicit_leader_movement_remote_to_local_movement"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_explicit_leader_movement_remote_to_local_movement&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.141', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fce8422b2d0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Explicit Leader Movement :: DOMData... | FAIL |
5 tests, 0 passed, 5 failed
==============================================================================
controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell...
==============================================================================
Stop_All_Members :: Stop every odl node.                              | FAIL |
Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS |
------------------------------------------------------------------------------
Start_All_And_Sync :: Start each member and wait for sync.            | FAIL |
Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.154:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore
------------------------------------------------------------------------------
controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL |
3 tests, 1 passed, 2 failed
==============================================================================
controller-clustering-tell.txt.Leader Isolation :: DOMDataBroker testing: L...
==============================================================================
Healing_Within_Request_Timeout :: The isolated node (leader) is re... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_leader_isolation_healing_within_request_timeout"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_leader_isolation_healing_within_request_timeout&order=bug_status"

HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.154:8181/jolokia/read/org.opendaylight.controller:Category=Shards,name=member-3-shard-default-config,type=DistributedConfigDatastore
------------------------------------------------------------------------------
Restart :: Restart odl                                                | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_leader_isolation_restart"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_leader_isolation_restart&order=bug_status"

Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Healing_After_Request_Timeout :: The isolated node (leader) is rej... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_leader_isolation_healing_after_request_timeout"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_leader_isolation_healing_after_request_timeout&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.141', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fce8417c450>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Leader Isolation :: DOMDataBroker t... | FAIL |
3 tests, 0 passed, 3 failed
==============================================================================
controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell...
==============================================================================
Stop_All_Members :: Stop every odl node.                              | FAIL |
Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS |
------------------------------------------------------------------------------
Start_All_And_Sync :: Start each member and wait for sync.            | FAIL |
Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.154:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore
------------------------------------------------------------------------------
controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL |
3 tests, 1 passed, 2 failed
==============================================================================
controller-clustering-tell.txt.Client Isolation :: DOMDataBroker testing: C...
==============================================================================
Producer_On_Shard_Leader_Node_ChainedTx :: Client isolation with p... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_client_isolation_producer_on_shard_leader_node_chainedtx"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_client_isolation_producer_on_shard_leader_node_chainedtx&order=bug_status"

HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.154:8181/jolokia/read/org.opendaylight.controller:Category=Shards,name=member-3-shard-default-config,type=DistributedConfigDatastore

Also keyword teardown failed:
Several failures occurred:

1) Variable '${client_node_dst}' not found.

2) Variable '${client_node_dst}' not found.

3) Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 1 minute 10 seconds. The last error was: Several failures occurred:

1) HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.154:8181/jolokia/read/org.opendaylight.controller:Category=Shards,name=member-3-shard-default-config,type=DistributedConfigDatastore

2) Resolving variable '${response.text}' failed: AttributeError: 'NoneType' object has no attribute 'text'
    [ Message content over the limit has been removed. ]

65) Resolving variable '${response.text}' failed: AttributeError: 'NoneType' object has no attribute 'text'

66) Evaluating expression "json.loads('''None''')" failed: JSONDecodeError: Expecting value: line 1 column 1 (char 0)

67) TypeError: Expected argument 1 to be a dictionary, got None instead.

68) TypeError: Expected argument 1 to be a dictionary, got None instead.

69) Unrecognized Raft state: None

70) No leader found.

71) Too many Leaders.

72) IndexError: Given index 0 is out of the range 0--1.

4) KeyError: 'threads'
------------------------------------------------------------------------------
Restart1 :: Restart odl                                               | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_client_isolation_restart1"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_client_isolation_restart1&order=bug_status"

Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Producer_On_Shard_Leader_Node_SimpleTx :: Client isolation with pr... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_client_isolation_producer_on_shard_leader_node_simpletx"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_client_isolation_producer_on_shard_leader_node_simpletx&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.141', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fce840c6050>: Failed to establish a new connection: [Errno 111] Connection refused'))

Also keyword teardown failed:
Several failures occurred:

1) Variable '${client_node_dst}' not found.

2) Variable '${client_node_dst}' not found.

3) Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 1 minute 10 seconds. The last error was: Several failures occurred:

1) ConnectionError: HTTPConnectionPool(host='10.30.171.141', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f...
    [ Message content over the limit has been removed. ]

173) Resolving variable '${response.text}' failed: AttributeError: 'NoneType' object has no attribute 'text'

174) Evaluating expression "json.loads('''None''')" failed: JSONDecodeError: Expecting value: line 1 column 1 (char 0)

175) TypeError: Expected argument 1 to be a dictionary, got None instead.

176) TypeError: Expected argument 1 to be a dictionary, got None instead.

177) Unrecognized Raft state: None

178) No leader found.

179) Too many Leaders.

180) IndexError: Given index 0 is out of the range 0--1.

4) KeyError: 'threads'
------------------------------------------------------------------------------
Restart2 :: Restart odl                                               | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_client_isolation_restart2"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_client_isolation_restart2&order=bug_status"

Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Producer_On_Shard_Non_Leader_Node_ChainedTx :: Client isolation wi... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_client_isolation_producer_on_shard_non_leader_node_chainedtx"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_client_isolation_producer_on_shard_non_leader_node_chainedtx&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.141', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fce85a3de50>: Failed to establish a new connection: [Errno 111] Connection refused'))

Also keyword teardown failed:
Several failures occurred:

1) Variable '${client_node_dst}' not found.

2) Variable '${client_node_dst}' not found.

3) Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 1 minute 10 seconds. The last error was: Several failures occurred:

1) ConnectionError: HTTPConnectionPool(host='10.30.171.141', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f...
    [ Message content over the limit has been removed. ]

173) Resolving variable '${response.text}' failed: AttributeError: 'NoneType' object has no attribute 'text'

174) Evaluating expression "json.loads('''None''')" failed: JSONDecodeError: Expecting value: line 1 column 1 (char 0)

175) TypeError: Expected argument 1 to be a dictionary, got None instead.

176) TypeError: Expected argument 1 to be a dictionary, got None instead.

177) Unrecognized Raft state: None

178) No leader found.

179) Too many Leaders.

180) IndexError: Given index 0 is out of the range 0--1.

4) KeyError: 'threads'
------------------------------------------------------------------------------
Restart3 :: Restart odl                                               | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_client_isolation_restart3"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_client_isolation_restart3&order=bug_status"

Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Producer_On_Shard_Non_Leader_Node_SimpleTx :: Client isolation wit... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_client_isolation_producer_on_shard_non_leader_node_simpletx"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_client_isolation_producer_on_shard_non_leader_node_simpletx&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.141', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fce874941d0>: Failed to establish a new connection: [Errno 111] Connection refused'))

Also keyword teardown failed:
Several failures occurred:

1) Variable '${client_node_dst}' not found.

2) Variable '${client_node_dst}' not found.

3) Keyword 'ShardStability.Shards_Stability_Get_Details' failed after retrying for 1 minute 10 seconds. The last error was: Several failures occurred:

1) ConnectionError: HTTPConnectionPool(host='10.30.171.141', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f...
    [ Message content over the limit has been removed. ]

173) Resolving variable '${response.text}' failed: AttributeError: 'NoneType' object has no attribute 'text'

174) Evaluating expression "json.loads('''None''')" failed: JSONDecodeError: Expecting value: line 1 column 1 (char 0)

175) TypeError: Expected argument 1 to be a dictionary, got None instead.

176) TypeError: Expected argument 1 to be a dictionary, got None instead.

177) Unrecognized Raft state: None

178) No leader found.

179) Too many Leaders.

180) IndexError: Given index 0 is out of the range 0--1.

4) KeyError: 'threads'
------------------------------------------------------------------------------
controller-clustering-tell.txt.Client Isolation :: DOMDataBroker t... | FAIL |
7 tests, 0 passed, 7 failed
==============================================================================
controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell...
==============================================================================
Stop_All_Members :: Stop every odl node.                              | FAIL |
Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS |
------------------------------------------------------------------------------
Start_All_And_Sync :: Start each member and wait for sync.            | FAIL |
Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.154:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore
------------------------------------------------------------------------------
controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL |
3 tests, 1 passed, 2 failed
==============================================================================
controller-clustering-tell.txt.Listener Stability :: DOMDataBroker testing:...
==============================================================================
Move_Leader_From_Listener_Local_To_Remote :: Listener runs on lead... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_listener_stability_move_leader_from_listener_local_to_remote"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_listener_stability_move_leader_from_listener_local_to_remote&order=bug_status"

HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.154:8181/jolokia/read/org.opendaylight.controller:Category=Shards,name=member-3-shard-default-config,type=DistributedConfigDatastore
------------------------------------------------------------------------------
Restart_1 :: Restart odl.                                             | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_listener_stability_restart_1"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_listener_stability_restart_1&order=bug_status"

Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Move_Leader_From_Listener_Remote_To_Other_Remote :: Listener runs ... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_listener_stability_move_leader_from_listener_remote_to_other_remote"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_listener_stability_move_leader_from_listener_remote_to_other_remote&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.141', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fce84e42fd0>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
Restart_2 :: Restart odl.                                             | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_listener_stability_restart_2"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_listener_stability_restart_2&order=bug_status"

Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Move_Leader_From_Listener_Remote_To_Local :: Listener runs on foll... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_listener_stability_move_leader_from_listener_remote_to_local"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_listener_stability_move_leader_from_listener_remote_to_local&order=bug_status"

ConnectionError: HTTPConnectionPool(host='10.30.171.141', port=8181): Max retries exceeded with url: /jolokia/read/org.opendaylight.controller:Category=Shards,name=member-1-shard-default-config,type=DistributedConfigDatastore (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fce85858b10>: Failed to establish a new connection: [Errno 111] Connection refused'))
------------------------------------------------------------------------------
controller-clustering-tell.txt.Listener Stability :: DOMDataBroker... | FAIL |
5 tests, 0 passed, 5 failed
==============================================================================
controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell...
==============================================================================
Stop_All_Members :: Stop every odl node.                              | FAIL |
Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS |
------------------------------------------------------------------------------
Start_All_And_Sync :: Start each member and wait for sync.            | FAIL |
Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.154:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore
------------------------------------------------------------------------------
controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL |
3 tests, 1 passed, 2 failed
==============================================================================
controller-clustering-tell.txt.Master Stability :: Cluster Singleton testin...
==============================================================================
Register_Singleton_Constant_On_Each_Node_And_Verify :: Register a ... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_master_stability_register_singleton_constant_on_each_node_and_verify"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_master_stability_register_singleton_constant_on_each_node_and_verify&order=bug_status"

HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.154:8181/rests/operations/odl-mdsal-lowlevel-control:register-singleton-constant
------------------------------------------------------------------------------
Unregister_Singleton_Constant_On_Non_Master_Node :: Unregister the... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_master_stability_unregister_singleton_constant_on_non_master_node"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_master_stability_unregister_singleton_constant_on_non_master_node&order=bug_status"

Variable '@{cs_candidates}' not found. Did you mean:
    @{cs_exp_candidates}
------------------------------------------------------------------------------
Monitor_Stability_While_Unregistered :: Verify that the owner rema... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_master_stability_monitor_stability_while_unregistered"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_master_stability_monitor_stability_while_unregistered&order=bug_status"

Variable '${cs_owner}' not found.
------------------------------------------------------------------------------
Reregister_Singleton_Constant :: Re-register the unregistered cand... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_master_stability_reregister_singleton_constant"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_master_stability_reregister_singleton_constant&order=bug_status"

Variable '${unregistered_node}' not found.
------------------------------------------------------------------------------
Verify_Stability_After_Reregistration :: Verify that the owner rem... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_master_stability_verify_stability_after_reregistration"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_master_stability_verify_stability_after_reregistration&order=bug_status"

Variable '${cs_owner}' not found.
------------------------------------------------------------------------------
Unregister_Singleton_Constant_On_Each_Node :: Unregister the appli... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_master_stability_unregister_singleton_constant_on_each_node"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_master_stability_unregister_singleton_constant_on_each_node&order=bug_status"

HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.154:8181/rests/operations/odl-mdsal-lowlevel-control:unregister-singleton-constant
------------------------------------------------------------------------------
controller-clustering-tell.txt.Master Stability :: Cluster Singlet... | FAIL |
6 tests, 0 passed, 6 failed
==============================================================================
controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell...
==============================================================================
Stop_All_Members :: Stop every odl node.                              | FAIL |
Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS |
------------------------------------------------------------------------------
Start_All_And_Sync :: Start each member and wait for sync.            | FAIL |
Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.154:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore
------------------------------------------------------------------------------
controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL |
3 tests, 1 passed, 2 failed
==============================================================================
controller-clustering-tell.txt.Partition And Heal :: Cluster Singleton test...
==============================================================================
Register_Singleton_Constant_On_Each_Node :: Register a candidate a... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_partition_and_heal_register_singleton_constant_on_each_node"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_partition_and_heal_register_singleton_constant_on_each_node&order=bug_status"

HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.154:8181/rests/operations/odl-mdsal-lowlevel-control:register-singleton-constant
------------------------------------------------------------------------------
Verify_Singleton_Constant_On_Each_Node :: Store the owner and cand... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_partition_and_heal_verify_singleton_constant_on_each_node"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_partition_and_heal_verify_singleton_constant_on_each_node&order=bug_status"

Could not parse owner and candidates for device get-singleton-constant-service']
------------------------------------------------------------------------------
Isolate_Owner_Node :: Isolate the cluster node which is the owner.... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_partition_and_heal_isolate_owner_node"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_partition_and_heal_isolate_owner_node&order=bug_status"

Variable '${cs_owner}' not found.
------------------------------------------------------------------------------
Monitor_Stability_While_Isolated :: Monitor the stability of the s... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_partition_and_heal_monitor_stability_while_isolated"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_partition_and_heal_monitor_stability_while_isolated&order=bug_status"

Invalid IF condition: Evaluating expression '"${index}" == "${cs_isolated_index}"' failed: Variable '${cs_isolated_index}' not found.
------------------------------------------------------------------------------
Rejoin_Isolated_node :: Rejoin isolated node.                         | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_partition_and_heal_rejoin_isolated_node"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_partition_and_heal_rejoin_isolated_node&order=bug_status"

Variable '${cs_isolated_index}' not found.
------------------------------------------------------------------------------
Unregister_Singleton_Constant_On_Each_Node :: Unregister the appli... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_partition_and_heal_unregister_singleton_constant_on_each_node"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_partition_and_heal_unregister_singleton_constant_on_each_node&order=bug_status"

HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.154:8181/rests/operations/odl-mdsal-lowlevel-control:unregister-singleton-constant
------------------------------------------------------------------------------
controller-clustering-tell.txt.Partition And Heal :: Cluster Singl... | FAIL |
6 tests, 0 passed, 6 failed
==============================================================================
controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell...
==============================================================================
Stop_All_Members :: Stop every odl node.                              | FAIL |
Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS |
------------------------------------------------------------------------------
Start_All_And_Sync :: Start each member and wait for sync.            | FAIL |
Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.154:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore
------------------------------------------------------------------------------
controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL |
3 tests, 1 passed, 2 failed
==============================================================================
controller-clustering-tell.txt.Chasing The Leader :: Cluster Singleton test...
==============================================================================
Register_Candidates :: Register a candidate application on each no... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_chasing_the_leader_register_candidates"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_chasing_the_leader_register_candidates&order=bug_status"

HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.154:8181/rests/operations/odl-mdsal-lowlevel-control:register-flapping-singleton
------------------------------------------------------------------------------
Do_Nothing :: Do nothing for the time of the test duration, becaus... | PASS |
------------------------------------------------------------------------------
Unregister_Candidates_And_Validate_Criteria :: Unregister the test... | FAIL |
... click for list of related bugs or create a new one if needed (with the
"controller_clustering_tell_txt_chasing_the_leader_unregister_candidates_and_validate_criteria"
reference somewhere inside)
"https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_tell_txt_chasing_the_leader_unregister_candidates_and_validate_criteria&order=bug_status"

HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.154:8181/rests/operations/odl-mdsal-lowlevel-control:unregister-flapping-singleton
------------------------------------------------------------------------------
controller-clustering-tell.txt.Chasing The Leader :: Cluster Singl... | FAIL |
3 tests, 1 passed, 2 failed
==============================================================================
controller-clustering-tell.txt.Restart Odl With Tell Based True :: Set tell...
==============================================================================
Stop_All_Members :: Stop every odl node.                              | FAIL |
Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1
------------------------------------------------------------------------------
Set_Tell_Based_Protocol_Usage :: Un-comment the flag usage in conf... | PASS |
------------------------------------------------------------------------------
Start_All_And_Sync :: Start each member and wait for sync.            | FAIL |
Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.154:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore
------------------------------------------------------------------------------
controller-clustering-tell.txt.Restart Odl With Tell Based True ::... | FAIL |
3 tests, 1 passed, 2 failed
==============================================================================
controller-clustering-tell.txt.Carpeople Crud :: Suite for performing basic...
==============================================================================
Add_Cars_To_Leader :: Add 30 cars to car Leader by one big PUT.       | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
See_Added_Cars_On_Leader :: GET response from Leader should match ... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
See_Added_Cars_On_Followers :: The same check on other members.       | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Add_People_To_First_Follower :: Add 30 people to people first Foll... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
See_Added_People_On_Leader :: GET response from Leader should matc... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
See_Added_People_On_Followers :: The same check on other members.     | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Buy_Cars_On_Leader :: Buy some cars on car-people Leader, loop of ... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Buy_Cars_On_Followers :: On each Follower buy corresponding ID seg... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
See_Added_CarPeople_On_Leader :: GET car-person mappings from Lead... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
See_Added_CarPeople_On_Followers :: The same check on other members.  | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Delete_All_CarPeople_On_Leader :: DELETE car-people container. No ... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Delete_All_People_On_Leader :: DELETE people container. No verific... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Delete_All_Cars_On_Leader :: DELETE cars container. No verificatio... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
controller-clustering-tell.txt.Carpeople Crud :: Suite for perform... | FAIL |
Suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig

13 tests, 0 passed, 13 failed
==============================================================================
controller-clustering-tell.txt.Car Failover Crud :: Suite mixing basic oper...
==============================================================================
Add_Original_Cars_On_Old_Leader_And_Verify :: Add initial cars on ... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Stop_Original_Car_Leader :: Stop the car Leader to cause a new lea... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Wait_For_New_Leader :: Wait until new car Leader is elected.          | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
See_Original_Cars_On_New_Leader :: GET cars from new Leader, shoul... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
See_Original_Cars_On_New_Followers :: The same check on other exis... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Delete_Original_Cars_On_New_Leader :: Delete cars on the new Leader.  | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Add_Leader_Cars_On_New_Leader :: Add cars on the new Leader.          | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
See_Leader_Cars_On_New_Leader :: GET cars from new Leader, should ... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
See_Leader_Cars_On_New_Followers :: The same check on other existi... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Delete_Leader_Cars_On_New_First_Follower :: Delete cars in new fir... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Add_Follower_Cars_On_New_First_Follower :: Add cars on the new fir... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
See_Folower_Cars_On_New_Leader :: Get cars from the new Leader, sh... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
See_Follower_Cars_On_New_Followers :: The same check on other exis... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Start_Old_Car_Leader :: Start the stopped member without deleting ... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
See_Folower_Cars_On_Old_Leader :: GET cars from the restarted memb... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Delete_Follower_Cars_On_New_Leader :: Delete cars on the last Leader. | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
controller-clustering-tell.txt.Car Failover Crud :: Suite mixing b... | FAIL |
Suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig

16 tests, 0 passed, 16 failed
==============================================================================
controller-clustering-tell.txt.Car Persistence Recovery :: This test restar...
==============================================================================
Add_Cars_On_Leader_And_Verify :: Single big PUT to datastore to ad... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Stop_All_Members :: Stop all controllers.                             | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Start_All_Members :: Start all controllers (should restore the per... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Memorize_Leader_And_Followers :: Locate current Leader of car Shard.  | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
See_Cars_On_Leader :: GET cars from Leader, should match the PUT d... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
See_Cars_On_Followers :: The same check on other members.             | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Delete_Cars_On_Leader :: Delete cars on the new Leader.               | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
controller-clustering-tell.txt.Car Persistence Recovery :: This te... | FAIL |
Suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig

7 tests, 0 passed, 7 failed
==============================================================================
controller-clustering-tell.txt.Buycar Failover :: This test focuses on test...
==============================================================================
Add_Cars_To_Leader_And_Verify :: Add all needed cars to car Leader... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Add_People_To_First_Follower_And_Verify :: Add all needed people t... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Buy_Cars_On_Leader_And_Verify :: Buy some cars on the leader member.  | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Buy_Cars_On_Follower_And_Verify :: Buy some cars on the first foll... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Reboot_People_Leader :: Previous people Leader is rebooted. We sho... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Buy_Cars_On_Leader_After_Reboot_And_Verify :: Buy some cars on the... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Buy_Cars_On_Follower_After_Reboot_And_Verify :: Buy some cars on t... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Delete_All_CarPeople :: DELETE car-people container. No verificati... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Delete_All_People :: DELETE people container. No verification beyo... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Delete_All_Cars :: DELETE cars container. No verification beyond h... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
controller-clustering-tell.txt.Buycar Failover :: This test focuse... | FAIL |
Suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig

10 tests, 0 passed, 10 failed
==============================================================================
controller-clustering-tell.txt.Entity Isolate :: Suite for performing membe...
==============================================================================
Check All Shards Before Isolate :: Check all shards in controller.    | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Isolate Entity Leader :: Isolate the entity-ownership Leader to ca... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Check All Shards After Isolate :: Check all shards in controller.     | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Rejoin Entity Leader :: Rejoin the entity-ownership Leader.           | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Check All Shards After Rejoin :: Check all shards in controller.      | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
controller-clustering-tell.txt.Entity Isolate :: Suite for perform... | FAIL |
Suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig

5 tests, 0 passed, 5 failed
==============================================================================
controller-clustering-tell.txt.Buycar Failover Isolation :: This test focus...
==============================================================================
Add_Cars_To_Leader_And_Verify :: Add all needed cars to car Leader... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Add_People_To_First_Follower_And_Verify :: Add all needed people t... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Buy_Cars_On_Leader_And_Verify :: Buy some cars on the leader member.  | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Buy_Cars_On_Follower_And_Verify :: Buy some cars on the first foll... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Isolate_and_Rejoin_People_Leader :: Previous people Leader is isol... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Buy_Cars_On_Leader_After_Rejoin_And_Verify :: Buy some cars on the... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Buy_Cars_On_Follower_After_Rejoin_And_Verify :: Buy some cars on t... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Delete_All_CarPeople :: DELETE car-people container. No verificati... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Delete_All_People :: DELETE people container. No verification beyo... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Delete_All_Cars :: DELETE cars container. No verification beyond h... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
controller-clustering-tell.txt.Buycar Failover Isolation :: This t... | FAIL |
Suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig

10 tests, 0 passed, 10 failed
==============================================================================
controller-clustering-tell.txt.Car Failover Crud Isolation :: Suite mixing ...
==============================================================================
Add_Original_Cars_On_Old_Leader_And_Verify :: Add initial cars on ... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Isolate_Original_Car_Leader :: Isolate the car Leader to cause a n... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Wait_For_New_Leader :: Wait until new car Leader is elected.          | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
See_Original_Cars_On_New_Leader :: GET cars from new Leader, shoul... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
See_Original_Cars_On_New_Followers :: The same check on other exis... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Delete_Original_Cars_On_New_Leader :: Delete cars on the new Leader.  | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Add_Leader_Cars_On_New_Leader :: Add cars on the new Leader.          | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
See_Leader_Cars_On_New_Leader :: GET cars from new Leader, should ... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
See_Leader_Cars_On_New_Followers :: The same check on other existi... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Delete_Leader_Cars_On_New_First_Follower :: Delete cars in new fir... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Add_Follower_Cars_On_New_First_Follower :: Add cars on the new fir... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
See_Folower_Cars_On_New_Leader :: Get cars from the new Leader, sh... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
See_Follower_Cars_On_New_Followers :: The same check on other exis... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Rejoin_Old_Car_Leader :: Rejoin the isolated member without deleti... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
See_Folower_Cars_On_Old_Leader :: GET cars from the restarted memb... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Delete_Follower_Cars_On_New_Leader :: Delete cars on the last Leader. | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
controller-clustering-tell.txt.Car Failover Crud Isolation :: Suit... | FAIL |
Suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig

16 tests, 0 passed, 16 failed
==============================================================================
controller-clustering-tell.txt.Car Outage Corners :: Cluster suite for test...
==============================================================================
Stop_Majority_Of_The_Followers :: Stop half plus one car Follower ... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Attempt_To_Add_Cars_To_Leader :: Adding cars should fail, as major... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Start_Tipping_Follower :: Start one Follower member without persis... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Add_Cars_On_Tipping_Follower :: Add cars on the tipping Follower.     | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
See_Cars_On_Existing_Members :: On each up member: GET cars, shoul... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Start_Other_Followers :: Start other followers without persisted d... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
See_Cars_On_New_Follower_Leader :: GET cars from a new follower to... | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
Delete_Cars_On_Leader :: Delete cars on Leader.                       | FAIL |
Parent suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig
------------------------------------------------------------------------------
controller-clustering-tell.txt.Car Outage Corners :: Cluster suite... | FAIL |
Suite setup failed:
HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.141:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig

8 tests, 0 passed, 8 failed
==============================================================================
controller-clustering-tell.txt                                        | FAIL |
162 tests, 16 passed, 146 failed
==============================================================================
Output:  /w/workspace/controller-csit-3node-clustering-tell-all-calcium/output.xml
Log:     /w/workspace/controller-csit-3node-clustering-tell-all-calcium/log.html
Report:  /w/workspace/controller-csit-3node-clustering-tell-all-calcium/report.html
+ true
+ echo 'Examining the files in data/log and checking filesize'
Examining the files in data/log and checking filesize
+ ssh 10.30.171.141 'ls -altr /tmp/karaf-0.20.2-SNAPSHOT/data/log/'
Warning: Permanently added '10.30.171.141' (ECDSA) to the list of known hosts.
total 78212
drwxrwxr-x. 2 jenkins jenkins       48 Jul 13 23:40 .
-rw-rw-r--. 1 jenkins jenkins     1720 Jul 13 23:40 karaf_console.log
drwxrwxr-x. 9 jenkins jenkins      149 Jul 14 03:07 ..
-rw-rw-r--. 1 jenkins jenkins 52889203 Jul 14 03:17 karaf.log
+ ssh 10.30.171.141 'du -hs /tmp/karaf-0.20.2-SNAPSHOT/data/log/*'
Warning: Permanently added '10.30.171.141' (ECDSA) to the list of known hosts.
4.0K	/tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf_console.log
77M	/tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log
+ ssh 10.30.170.225 'ls -altr /tmp/karaf-0.20.2-SNAPSHOT/data/log/'
Warning: Permanently added '10.30.170.225' (ECDSA) to the list of known hosts.
total 106884
drwxrwxr-x. 2 jenkins jenkins       48 Jul 13 23:40 .
-rw-rw-r--. 1 jenkins jenkins     2042 Jul 13 23:41 karaf_console.log
drwxrwxr-x. 9 jenkins jenkins      149 Jul 14 03:07 ..
-rw-rw-r--. 1 jenkins jenkins 48566633 Jul 14 03:17 karaf.log
+ ssh 10.30.170.225 'du -hs /tmp/karaf-0.20.2-SNAPSHOT/data/log/*'
Warning: Permanently added '10.30.170.225' (ECDSA) to the list of known hosts.
4.0K	/tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf_console.log
105M	/tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log
+ ssh 10.30.170.154 'ls -altr /tmp/karaf-0.20.2-SNAPSHOT/data/log/'
Warning: Permanently added '10.30.170.154' (ECDSA) to the list of known hosts.
total 338564
drwxrwxr-x. 2 jenkins jenkins        48 Jul 13 23:40 .
-rw-rw-r--. 1 jenkins jenkins      1720 Jul 13 23:40 karaf_console.log
drwxrwxr-x. 4 jenkins jenkins        28 Jul 14 03:06 ..
-rw-rw-r--. 1 jenkins jenkins 279633060 Jul 14 03:17 karaf.log
+ ssh 10.30.170.154 'du -hs /tmp/karaf-0.20.2-SNAPSHOT/data/log/*'
Warning: Permanently added '10.30.170.154' (ECDSA) to the list of known hosts.
4.0K	/tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf_console.log
331M	/tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log
+ set +e
++ seq 1 3
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_1_IP
+ echo 'Let'\''s take the karaf thread dump again'
Let's take the karaf thread dump again
+ ssh 10.30.171.141 'sudo ps aux'
Warning: Permanently added '10.30.171.141' (ECDSA) to the list of known hosts.
++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-tell-all-calcium/ps_after.log
++ grep -v grep
++ cut -f2 '-d '
++ tr -s ' '
+ pid=31158
+ echo 'karaf main: org.apache.karaf.main.Main, pid:31158'
karaf main: org.apache.karaf.main.Main, pid:31158
+ ssh 10.30.171.141 '/usr/lib/jvm/java-17-openjdk/bin/jstack -l 31158'
Warning: Permanently added '10.30.171.141' (ECDSA) to the list of known hosts.
+ echo 'killing karaf process...'
killing karaf process...
+ ssh 10.30.171.141 bash -c 'ps axf | grep karaf | grep -v grep | awk '\''{print "kill -9 " $1}'\'' | sh'
Warning: Permanently added '10.30.171.141' (ECDSA) to the list of known hosts.
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_2_IP
+ echo 'Let'\''s take the karaf thread dump again'
Let's take the karaf thread dump again
+ ssh 10.30.170.225 'sudo ps aux'
Warning: Permanently added '10.30.170.225' (ECDSA) to the list of known hosts.
++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-tell-all-calcium/ps_after.log
++ grep -v grep
++ tr -s ' '
++ cut -f2 '-d '
+ pid=30352
+ echo 'karaf main: org.apache.karaf.main.Main, pid:30352'
karaf main: org.apache.karaf.main.Main, pid:30352
+ ssh 10.30.170.225 '/usr/lib/jvm/java-17-openjdk/bin/jstack -l 30352'
Warning: Permanently added '10.30.170.225' (ECDSA) to the list of known hosts.
+ echo 'killing karaf process...'
killing karaf process...
+ ssh 10.30.170.225 bash -c 'ps axf | grep karaf | grep -v grep | awk '\''{print "kill -9 " $1}'\'' | sh'
Warning: Permanently added '10.30.170.225' (ECDSA) to the list of known hosts.
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_3_IP
+ echo 'Let'\''s take the karaf thread dump again'
Let's take the karaf thread dump again
+ ssh 10.30.170.154 'sudo ps aux'
Warning: Permanently added '10.30.170.154' (ECDSA) to the list of known hosts.
++ tr -s ' '
++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-tell-all-calcium/ps_after.log
++ cut -f2 '-d '
++ grep -v grep
+ pid=14107
+ echo 'karaf main: org.apache.karaf.main.Main, pid:14107'
karaf main: org.apache.karaf.main.Main, pid:14107
+ ssh 10.30.170.154 '/usr/lib/jvm/java-17-openjdk/bin/jstack -l 14107'
Warning: Permanently added '10.30.170.154' (ECDSA) to the list of known hosts.
+ echo 'killing karaf process...'
killing karaf process...
+ ssh 10.30.170.154 bash -c 'ps axf | grep karaf | grep -v grep | awk '\''{print "kill -9 " $1}'\'' | sh'
Warning: Permanently added '10.30.170.154' (ECDSA) to the list of known hosts.
+ sleep 5
++ seq 1 3
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_1_IP
+ echo 'Compressing karaf.log 1'
Compressing karaf.log 1
+ ssh 10.30.171.141 gzip --best /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log
Warning: Permanently added '10.30.171.141' (ECDSA) to the list of known hosts.
gzip: /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log: file size changed while zipping
+ echo 'Fetching compressed karaf.log 1'
Fetching compressed karaf.log 1
+ scp 10.30.171.141:/tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log.gz odl1_karaf.log.gz
Warning: Permanently added '10.30.171.141' (ECDSA) to the list of known hosts.
+ ssh 10.30.171.141 rm -f /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log.gz
Warning: Permanently added '10.30.171.141' (ECDSA) to the list of known hosts.
+ scp 10.30.171.141:/tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf_console.log odl1_karaf_console.log
Warning: Permanently added '10.30.171.141' (ECDSA) to the list of known hosts.
+ ssh 10.30.171.141 rm -f /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf_console.log
Warning: Permanently added '10.30.171.141' (ECDSA) to the list of known hosts.
+ echo 'Fetch GC logs'
Fetch GC logs
+ mkdir -p gclogs-1
+ scp '10.30.171.141:/tmp/karaf-0.20.2-SNAPSHOT/data/log/*.log' gclogs-1/
Warning: Permanently added '10.30.171.141' (ECDSA) to the list of known hosts.
scp: /tmp/karaf-0.20.2-SNAPSHOT/data/log/*.log: No such file or directory
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_2_IP
+ echo 'Compressing karaf.log 2'
Compressing karaf.log 2
+ ssh 10.30.170.225 gzip --best /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log
Warning: Permanently added '10.30.170.225' (ECDSA) to the list of known hosts.
gzip: /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log: file size changed while zipping
+ echo 'Fetching compressed karaf.log 2'
Fetching compressed karaf.log 2
+ scp 10.30.170.225:/tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log.gz odl2_karaf.log.gz
Warning: Permanently added '10.30.170.225' (ECDSA) to the list of known hosts.
+ ssh 10.30.170.225 rm -f /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log.gz
Warning: Permanently added '10.30.170.225' (ECDSA) to the list of known hosts.
+ scp 10.30.170.225:/tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf_console.log odl2_karaf_console.log
Warning: Permanently added '10.30.170.225' (ECDSA) to the list of known hosts.
+ ssh 10.30.170.225 rm -f /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf_console.log
Warning: Permanently added '10.30.170.225' (ECDSA) to the list of known hosts.
+ echo 'Fetch GC logs'
Fetch GC logs
+ mkdir -p gclogs-2
+ scp '10.30.170.225:/tmp/karaf-0.20.2-SNAPSHOT/data/log/*.log' gclogs-2/
Warning: Permanently added '10.30.170.225' (ECDSA) to the list of known hosts.
scp: /tmp/karaf-0.20.2-SNAPSHOT/data/log/*.log: No such file or directory
+ for i in $(seq 1 "${NUM_ODL_SYSTEM}")
+ CONTROLLERIP=ODL_SYSTEM_3_IP
+ echo 'Compressing karaf.log 3'
Compressing karaf.log 3
+ ssh 10.30.170.154 gzip --best /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log
Warning: Permanently added '10.30.170.154' (ECDSA) to the list of known hosts.
+ echo 'Fetching compressed karaf.log 3'
Fetching compressed karaf.log 3
+ scp 10.30.170.154:/tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log.gz odl3_karaf.log.gz
Warning: Permanently added '10.30.170.154' (ECDSA) to the list of known hosts.
+ ssh 10.30.170.154 rm -f /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log.gz
Warning: Permanently added '10.30.170.154' (ECDSA) to the list of known hosts.
+ scp 10.30.170.154:/tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf_console.log odl3_karaf_console.log
Warning: Permanently added '10.30.170.154' (ECDSA) to the list of known hosts.
+ ssh 10.30.170.154 rm -f /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf_console.log
Warning: Permanently added '10.30.170.154' (ECDSA) to the list of known hosts.
+ echo 'Fetch GC logs'
Fetch GC logs
+ mkdir -p gclogs-3
+ scp '10.30.170.154:/tmp/karaf-0.20.2-SNAPSHOT/data/log/*.log' gclogs-3/
Warning: Permanently added '10.30.170.154' (ECDSA) to the list of known hosts.
scp: /tmp/karaf-0.20.2-SNAPSHOT/data/log/*.log: No such file or directory
+ echo 'Examine copied files'
Examine copied files
+ ls -lt
total 111884
drwxrwxr-x. 2 jenkins jenkins         6 Jul 14 03:18 gclogs-3
-rw-rw-r--. 1 jenkins jenkins      1720 Jul 14 03:18 odl3_karaf_console.log
-rw-rw-r--. 1 jenkins jenkins   3474887 Jul 14 03:18 odl3_karaf.log.gz
drwxrwxr-x. 2 jenkins jenkins         6 Jul 14 03:18 gclogs-2
-rw-rw-r--. 1 jenkins jenkins      2042 Jul 14 03:18 odl2_karaf_console.log
-rw-rw-r--. 1 jenkins jenkins   1776237 Jul 14 03:18 odl2_karaf.log.gz
drwxrwxr-x. 2 jenkins jenkins         6 Jul 14 03:18 gclogs-1
-rw-rw-r--. 1 jenkins jenkins      1720 Jul 14 03:18 odl1_karaf_console.log
-rw-rw-r--. 1 jenkins jenkins   1940785 Jul 14 03:18 odl1_karaf.log.gz
-rw-rw-r--. 1 jenkins jenkins    141195 Jul 14 03:17 karaf_3_14107_threads_after.log
-rw-rw-r--. 1 jenkins jenkins     16331 Jul 14 03:17 ps_after.log
-rw-rw-r--. 1 jenkins jenkins    131517 Jul 14 03:17 karaf_2_30352_threads_after.log
-rw-rw-r--. 1 jenkins jenkins    125634 Jul 14 03:17 karaf_1_31158_threads_after.log
-rw-rw-r--. 1 jenkins jenkins    305293 Jul 14 03:17 report.html
-rw-rw-r--. 1 jenkins jenkins   3749495 Jul 14 03:17 log.html
-rw-rw-r--. 1 jenkins jenkins 102318386 Jul 14 03:17 output.xml
-rw-rw-r--. 1 jenkins jenkins      6030 Jul 13 23:42 testplan.txt
-rw-rw-r--. 1 jenkins jenkins    159375 Jul 13 23:42 karaf_3_5737_threads_before.log
-rw-rw-r--. 1 jenkins jenkins     16459 Jul 13 23:42 ps_before.log
-rw-rw-r--. 1 jenkins jenkins    150710 Jul 13 23:42 karaf_2_5741_threads_before.log
-rw-rw-r--. 1 jenkins jenkins    157387 Jul 13 23:42 karaf_1_5757_threads_before.log
-rw-rw-r--. 1 jenkins jenkins      3106 Jul 13 23:40 post-startup-script.sh
-rw-rw-r--. 1 jenkins jenkins       252 Jul 13 23:40 startup-script.sh
-rw-rw-r--. 1 jenkins jenkins      3452 Jul 13 23:40 configuration-script.sh
-rw-rw-r--. 1 jenkins jenkins       695 Jul 13 23:40 custom_shard_config.txt
-rw-rw-r--. 1 jenkins jenkins       135 Jul 13 23:40 scriptplan.txt
-rw-rw-r--. 1 jenkins jenkins       335 Jul 13 23:40 detect_variables.env
-rw-rw-r--. 1 jenkins jenkins      2619 Jul 13 23:40 pom.xml
-rw-rw-r--. 1 jenkins jenkins        86 Jul 13 23:40 set_variables.env
-rw-rw-r--. 1 jenkins jenkins       314 Jul 13 23:40 slave_addresses.txt
-rw-rw-r--. 1 jenkins jenkins       570 Jul 13 23:40 requirements.txt
-rw-rw-r--. 1 jenkins jenkins        26 Jul 13 23:40 env.properties
-rw-rw-r--. 1 jenkins jenkins       335 Jul 13 23:38 stack-parameters.yaml
drwxrwxr-x. 7 jenkins jenkins      4096 Jul 13 23:36 test
drwxrwxr-x. 2 jenkins jenkins         6 Jul 13 23:36 test@tmp
-rw-rw-r--. 1 jenkins jenkins      1404 Jul 13 20:48 maven-metadata.xml
+ true
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/sh /tmp/jenkins282464119457224632.sh
Cleaning up Robot installation...
$ ssh-agent -k
unset SSH_AUTH_SOCK;
unset SSH_AGENT_PID;
echo Agent pid 5274 killed;
[ssh-agent] Stopped.
Recording plot data
Robot results publisher started...
INFO: Checking test criticality is deprecated and will be dropped in a future release!
-Parsing output xml:
Done!
-Copying log files to build dir:
Done!
-Assigning results to build:
Done!
-Checking thresholds:
Done!
Done publishing Robot results.
Build step 'Publish Robot Framework test results' changed build result to UNSTABLE
[PostBuildScript] - [INFO] Executing post build scripts.
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins9423614797089393798.sh
Archiving csit artifacts
mv: cannot stat '*_1.png': No such file or directory
mv: cannot stat '/tmp/odl1_*': No such file or directory
mv: cannot stat '*_2.png': No such file or directory
mv: cannot stat '/tmp/odl2_*': No such file or directory
mv: cannot stat '*_3.png': No such file or directory
mv: cannot stat '/tmp/odl3_*': No such file or directory
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 1832k    0 1832k    0     0  4512k      0 --:--:-- --:--:-- --:--:-- 4501k
100 6696k    0 6696k    0     0  4769k      0 --:--:--  0:00:01 --:--:-- 4765k
100 8092k    0 8092k    0     0  4866k      0 --:--:--  0:00:01 --:--:-- 4863k
Archive:  robot-plugin.zip
  inflating: ./archives/robot-plugin/log.html  
  inflating: ./archives/robot-plugin/output.xml  
  inflating: ./archives/robot-plugin/report.html  
mv: cannot stat '*.log.gz': No such file or directory
mv: cannot stat '*.csv': No such file or directory
mv: cannot stat '*.png': No such file or directory
[PostBuildScript] - [INFO] Executing post build scripts.
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins13939413314127951260.sh
[PostBuildScript] - [INFO] Executing post build scripts.
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties content 
OS_CLOUD=vex
OS_STACK_NAME=releng-controller-csit-3node-clustering-tell-all-calcium-44

[EnvInject] - Variables injected successfully.
provisioning config files...
copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins12770884133493116676.sh
---> openstack-stack-delete.sh
Setup pyenv:
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-tell-all-calcium/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-2KoL from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes python-heatclient python-openstackclient 
lf-activate-venv(): INFO: Adding /tmp/venv-2KoL/bin to PATH
INFO: Retrieving stack cost for: releng-controller-csit-3node-clustering-tell-all-calcium-44
DEBUG: Successfully retrieved stack cost: total: 1.32
INFO: Deleting stack releng-controller-csit-3node-clustering-tell-all-calcium-44
Successfully deleted stack releng-controller-csit-3node-clustering-tell-all-calcium-44
[PostBuildScript] - [INFO] Executing post build scripts.
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins14034713625971691889.sh
---> sysstat.sh
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins15009622892192314146.sh
---> package-listing.sh
++ facter osfamily
++ tr '[:upper:]' '[:lower:]'
+ OS_FAMILY=redhat
+ workspace=/w/workspace/controller-csit-3node-clustering-tell-all-calcium
+ START_PACKAGES=/tmp/packages_start.txt
+ END_PACKAGES=/tmp/packages_end.txt
+ DIFF_PACKAGES=/tmp/packages_diff.txt
+ PACKAGES=/tmp/packages_start.txt
+ '[' /w/workspace/controller-csit-3node-clustering-tell-all-calcium ']'
+ PACKAGES=/tmp/packages_end.txt
+ case "${OS_FAMILY}" in
+ rpm -qa
+ sort
+ '[' -f /tmp/packages_start.txt ']'
+ '[' -f /tmp/packages_end.txt ']'
+ diff /tmp/packages_start.txt /tmp/packages_end.txt
+ '[' /w/workspace/controller-csit-3node-clustering-tell-all-calcium ']'
+ mkdir -p /w/workspace/controller-csit-3node-clustering-tell-all-calcium/archives/
+ cp -f /tmp/packages_diff.txt /tmp/packages_end.txt /tmp/packages_start.txt /w/workspace/controller-csit-3node-clustering-tell-all-calcium/archives/
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins10826257454609235774.sh
---> capture-instance-metadata.sh
Setup pyenv:
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-tell-all-calcium/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-2KoL from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing: lftools 
lf-activate-venv(): INFO: Adding /tmp/venv-2KoL/bin to PATH
INFO: Running in OpenStack, capturing instance metadata
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins4859505166646051268.sh
provisioning config files...
Could not find credentials [logs] for controller-csit-3node-clustering-tell-all-calcium #44
copy managed file [jenkins-log-archives-settings] to file:/w/workspace/controller-csit-3node-clustering-tell-all-calcium@tmp/config15246394984659914315tmp
Regular expression run condition: Expression=[^.*logs-s3.*], Label=[odl-logs-s3-cloudfront-index]
Run condition [Regular expression match] enabling perform for step [Provide Configuration files]
provisioning config files...
copy managed file [jenkins-s3-log-ship] to file:/home/jenkins/.aws/credentials
[EnvInject] - Injecting environment variables from a build step.
[EnvInject] - Injecting as environment variables the properties content 
SERVER_ID=logs

[EnvInject] - Variables injected successfully.
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins15672678502454438413.sh
---> create-netrc.sh
WARN: Log server credential not found.
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins6348205999454517242.sh
---> python-tools-install.sh
Setup pyenv:
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-tell-all-calcium/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-2KoL from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing: lftools 
lf-activate-venv(): INFO: Adding /tmp/venv-2KoL/bin to PATH
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins9597101431298180773.sh
---> sudo-logs.sh
Archiving 'sudo' log..
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash /tmp/jenkins10823509923030072603.sh
---> job-cost.sh
Setup pyenv:
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-tell-all-calcium/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-2KoL from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing: zipp==1.1.0 python-openstackclient urllib3~=1.26.15 
lf-activate-venv(): INFO: Adding /tmp/venv-2KoL/bin to PATH
DEBUG: total: 1.32
INFO: Retrieving Stack Cost...
INFO: Retrieving Pricing Info for: v3-standard-2
INFO: Archiving Costs
[controller-csit-3node-clustering-tell-all-calcium] $ /bin/bash -l /tmp/jenkins672873784773915434.sh
---> logs-deploy.sh
Setup pyenv:
  system
  3.8.13
  3.9.13
  3.10.13
* 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-tell-all-calcium/.python-version)
lf-activate-venv(): INFO: Reuse venv:/tmp/venv-2KoL from file:/tmp/.os_lf_venv
lf-activate-venv(): INFO: Installing: lftools 
lf-activate-venv(): INFO: Adding /tmp/venv-2KoL/bin to PATH
WARNING: Nexus logging server not set
INFO: S3 path logs/releng/vex-yul-odl-jenkins-1/controller-csit-3node-clustering-tell-all-calcium/44/
INFO: archiving logs to S3
---> uname -a:
Linux prd-centos8-robot-2c-8g-20286.novalocal 4.18.0-553.5.1.el8.x86_64 #1 SMP Tue May 21 05:46:01 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux


---> lscpu:
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              2
On-line CPU(s) list: 0,1
Thread(s) per core:  1
Core(s) per socket:  1
Socket(s):           2
NUMA node(s):        1
Vendor ID:           AuthenticAMD
CPU family:          23
Model:               49
Model name:          AMD EPYC-Rome Processor
Stepping:            0
CPU MHz:             2799.998
BogoMIPS:            5599.99
Virtualization:      AMD-V
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           32K
L1i cache:           32K
L2 cache:            512K
L3 cache:            16384K
NUMA node0 CPU(s):   0,1
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr wbnoinvd arat npt nrip_save umip rdpid arch_capabilities


---> nproc:
2


---> df -h:
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs        3.8G     0  3.8G   0% /dev
tmpfs           3.8G     0  3.8G   0% /dev/shm
tmpfs           3.8G   17M  3.8G   1% /run
tmpfs           3.8G     0  3.8G   0% /sys/fs/cgroup
/dev/vda1        40G  8.5G   32G  22% /
tmpfs           770M     0  770M   0% /run/user/1001


---> free -m:
              total        used        free      shared  buff/cache   available
Mem:           7697         684        4653          19        2359        6716
Swap:          1023           0        1023


---> ip addr:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1458 qdisc mq state UP group default qlen 1000
    link/ether fa:16:3e:05:96:fe brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    altname ens3
    inet 10.30.170.213/23 brd 10.30.171.255 scope global dynamic noprefixroute eth0
       valid_lft 72986sec preferred_lft 72986sec
    inet6 fe80::f816:3eff:fe05:96fe/64 scope link 
       valid_lft forever preferred_lft forever


---> sar -b -r -n DEV:
Linux 4.18.0-553.5.1.el8.x86_64 (prd-centos8-robot-2c-8g-20286.novalocal) 	07/14/2024 	_x86_64_	(2 CPU)

12:00:01 AM       tps      rtps      wtps   bread/s   bwrtn/s
12:01:01 AM      0.98      0.02      0.97      0.13    213.06
12:02:01 AM      0.63      0.07      0.57      1.73     23.50
12:03:01 AM      0.13      0.00      0.13      0.00     11.60
12:04:01 AM      0.15      0.00      0.15      0.00     23.06
12:05:01 AM      0.25      0.00      0.25      0.00     11.90
12:06:01 AM      0.20      0.00      0.20      0.00      2.28
12:07:01 AM      0.33      0.00      0.33      0.00      4.90
12:08:01 AM      0.43      0.07      0.37      7.60     20.98
12:09:01 AM      0.22      0.00      0.22      0.00     10.68
12:10:01 AM      0.13      0.00      0.13      0.00     14.26
12:11:01 AM      0.42      0.00      0.42      0.00     17.63
12:12:01 AM      0.30      0.00      0.30      0.00     17.96
12:13:01 AM      0.15      0.00      0.15      0.00     14.15
12:14:01 AM      0.22      0.00      0.22      0.00      8.53
12:15:01 AM      0.13      0.00      0.13      0.00      2.80
12:16:01 AM      0.10      0.00      0.10      0.00      0.95
12:17:01 AM      0.40      0.00      0.40      0.00     18.91
12:18:01 AM      0.13      0.00      0.13      0.00     14.13
12:19:01 AM      0.25      0.00      0.25      0.00      9.02
12:20:01 AM      0.32      0.00      0.32      0.00     20.04
12:21:01 AM      0.58      0.00      0.58      0.00     17.93
12:22:01 AM      0.60      0.00      0.60      0.00     18.36
12:23:01 AM      0.25      0.00      0.25      0.00     21.84
12:24:01 AM      0.35      0.00      0.35      0.00     16.81
12:25:01 AM      0.12      0.00      0.12      0.00     12.00
12:26:01 AM      0.25      0.00      0.25      0.00     23.99
12:27:01 AM      0.38      0.00      0.38      0.00     16.53
12:28:01 AM      0.17      0.00      0.17      0.00     22.81
12:29:02 AM      0.13      0.00      0.13      0.00     12.00
12:30:01 AM      0.10      0.00      0.10      0.00      9.22
12:31:01 AM      1.25      0.32      0.93      6.93     35.30
12:32:01 AM      0.42      0.00      0.42      0.00     20.13
12:33:01 AM      0.13      0.00      0.13      0.00      8.81
12:34:01 AM      0.20      0.00      0.20      0.00     14.28
12:35:01 AM      0.13      0.00      0.13      0.00     15.06
12:36:01 AM      0.13      0.00      0.13      0.00     14.01
12:37:01 AM      0.37      0.00      0.37      0.00     13.58
12:38:01 AM      0.15      0.00      0.15      0.00     14.40
12:39:01 AM      0.27      0.02      0.25      0.13      8.96
12:40:01 AM      8.81      5.73      3.08    555.41     85.84
12:41:01 AM      0.30      0.00      0.30      0.00     18.34
12:42:01 AM      0.40      0.00      0.40      0.00     18.80
12:43:01 AM      0.12      0.00      0.12      0.00      8.13
12:44:01 AM      0.15      0.00      0.15      0.00     15.08
12:45:01 AM      0.17      0.00      0.17      0.00     16.95
12:46:01 AM      0.13      0.00      0.13      0.00      7.33
12:47:01 AM      0.37      0.00      0.37      0.00     18.38
12:48:01 AM      0.30      0.00      0.30      0.00     19.36
12:49:01 AM      0.18      0.00      0.18      0.00     14.95
12:50:01 AM      0.13      0.00      0.13      0.00      7.43
12:51:01 AM      0.23      0.00      0.23      0.00     28.54
12:52:01 AM      0.40      0.00      0.40      0.00     11.01
12:53:01 AM      0.42      0.00      0.42      0.00     28.00
12:54:01 AM      0.17      0.00      0.17      0.00     12.25
12:55:01 AM      0.20      0.00      0.20      0.00     23.13
12:56:01 AM      0.20      0.00      0.20      0.00     12.86
12:57:01 AM      0.27      0.00      0.27      0.00     25.66
12:58:01 AM      0.30      0.00      0.30      0.00     10.30
12:59:01 AM      0.15      0.00      0.15      0.00     16.83
01:00:01 AM      0.20      0.00      0.20      0.00     14.30
01:01:01 AM      0.35      0.03      0.32      0.27     12.34
01:02:01 AM      0.45      0.00      0.45      0.00     22.51
01:03:01 AM      0.15      0.00      0.15      0.00      7.20
01:04:01 AM      0.12      0.00      0.12      0.00     14.66
01:05:01 AM      0.20      0.00      0.20      0.00     12.31
01:06:01 AM      0.23      0.00      0.23      0.00     11.06
01:07:01 AM      0.42      0.00      0.42      0.00     20.54
01:08:01 AM      0.18      0.00      0.18      0.00     15.21
01:09:01 AM      0.13      0.00      0.13      0.00     14.40
01:10:01 AM      0.13      0.00      0.13      0.00      8.27
01:11:01 AM      0.40      0.00      0.40      0.00     18.55
01:12:01 AM      0.38      0.00      0.38      0.00     26.94
01:13:01 AM      0.13      0.00      0.13      0.00      9.47
01:14:01 AM      0.15      0.00      0.15      0.00     20.13
01:15:01 AM      0.13      0.00      0.13      0.00     17.73
01:16:01 AM      0.23      0.00      0.23      0.00     16.03
01:17:01 AM      0.37      0.00      0.37      0.00     20.76
01:18:01 AM      0.15      0.00      0.15      0.00     19.35
01:19:01 AM      0.15      0.00      0.15      0.00      9.48
01:20:01 AM      0.32      0.00      0.32      0.00    106.93
01:21:01 AM      0.20      0.00      0.20      0.00     39.26
01:22:01 AM      0.35      0.00      0.35      0.00     19.84
01:23:01 AM      0.13      0.00      0.13      0.00     14.26
01:24:01 AM      0.15      0.00      0.15      0.00     14.40
01:25:01 AM      0.12      0.00      0.12      0.00      7.60
01:26:01 AM      0.13      0.00      0.13      0.00     14.56
01:27:01 AM      0.80      0.00      0.80      0.00     24.26
01:28:01 AM      0.33      0.00      0.33      0.00     62.10
01:29:01 AM      0.17      0.00      0.17      0.00     73.37
01:30:01 AM      0.15      0.00      0.15      0.00     34.14
01:31:01 AM      0.28      0.00      0.28      0.00     16.69
01:32:01 AM      0.42      0.00      0.42      0.00     15.23
01:33:01 AM      0.17      0.00      0.17      0.00     18.26
01:34:01 AM      0.22      0.00      0.22      0.00      9.90
01:35:02 AM      0.32      0.00      0.32      0.00     20.03
01:36:01 AM      0.15      0.00      0.15      0.00     27.64
01:37:01 AM      0.37      0.00      0.37      0.00     81.97
01:38:01 AM      0.15      0.00      0.15      0.00     26.81
01:39:01 AM      0.15      0.00      0.15      0.00     13.16
01:40:01 AM      0.30      0.00      0.30      0.00     17.71
01:41:01 AM      0.30      0.00      0.30      0.00     16.31
01:42:01 AM      0.45      0.00      0.45      0.00     19.45
01:43:01 AM      0.20      0.00      0.20      0.00     13.20
01:44:01 AM      0.18      0.00      0.18      0.00     10.25
01:45:01 AM      0.18      0.00      0.18      0.00     97.32
01:46:01 AM      0.20      0.00      0.20      0.00     70.55
01:47:01 AM      0.32      0.00      0.32      0.00     12.10
01:48:01 AM      0.13      0.00      0.13      0.00     15.10
01:49:01 AM      0.18      0.00      0.18      0.00     14.85
01:50:01 AM      0.15      0.00      0.15      0.00      8.15
01:51:01 AM      0.27      0.00      0.27      0.00     16.69
01:52:01 AM      0.42      0.00      0.42      0.00     30.94
01:53:01 AM      0.17      0.00      0.17      0.00     12.78
01:54:01 AM      0.13      0.00      0.13      0.00     11.73
01:55:01 AM      0.13      0.00      0.13      0.00     22.13
01:56:01 AM      0.12      0.00      0.12      0.00     11.33
01:57:01 AM      0.33      0.00      0.33      0.00     25.56
01:58:01 AM      0.30      0.00      0.30      0.00     15.16
01:59:01 AM      0.15      0.00      0.15      0.00     14.93
02:00:01 AM      0.15      0.00      0.15      0.00     15.59
02:01:01 AM      0.33      0.00      0.33      0.00     12.83
02:02:01 AM      0.43      0.00      0.43      0.00     20.11
02:03:01 AM      0.20      0.00      0.20      0.00     15.41
02:04:01 AM      0.83      0.00      0.83      0.00     26.26
02:05:01 AM      0.20      0.00      0.20      0.00      9.73
02:06:01 AM      0.28      0.00      0.28      0.00     15.86
02:07:01 AM      0.52      0.00      0.52      0.00     17.40
02:08:01 AM      0.37      0.00      0.37      0.00     12.86
02:09:01 AM      0.40      0.00      0.40      0.00     17.08
02:10:01 AM      0.13      0.00      0.13      0.00      9.23
02:11:01 AM      0.27      0.00      0.27      0.00     18.68
02:12:01 AM      0.27      0.00      0.27      0.00     15.98
02:13:01 AM      0.23      0.00      0.23      0.00     12.60
02:14:01 AM      0.18      0.00      0.18      0.00     16.95
02:15:01 AM      0.38      0.00      0.38      0.00     16.66
02:16:01 AM      0.58      0.00      0.58      0.00     17.21
02:17:01 AM      0.65      0.00      0.65      0.00     18.48
02:18:01 AM      0.15      0.00      0.15      0.00     13.06
02:19:01 AM      0.13      0.00      0.13      0.00     10.13
02:20:01 AM      0.15      0.00      0.15      0.00     25.76
02:21:01 AM      0.17      0.00      0.17      0.00     10.70
02:22:01 AM      0.48      0.00      0.48      0.00     24.36
02:23:01 AM      0.18      0.00      0.18      0.00      9.75
02:24:01 AM      0.15      0.00      0.15      0.00     19.59
02:25:01 AM      0.17      0.00      0.17      0.00     20.93
02:26:01 AM      0.13      0.00      0.13      0.00     18.13
02:27:01 AM      0.40      0.00      0.40      0.00     12.28
02:28:01 AM      0.17      0.00      0.17      0.00     12.98
02:29:01 AM      0.18      0.00      0.18      0.00     14.96
02:30:01 AM      0.12      0.00      0.12      0.00      9.46
02:31:01 AM      0.30      0.00      0.30      0.00     17.00
02:32:01 AM      0.38      0.00      0.38      0.00     14.51
02:33:01 AM      0.17      0.00      0.17      0.00     14.40
02:34:01 AM      0.25      0.00      0.25      0.00     16.53
02:35:01 AM      0.25      0.00      0.25      0.00     14.65
02:36:01 AM      0.22      0.00      0.22      0.00     23.45
02:37:01 AM      0.43      0.00      0.43      0.00     11.16
02:38:01 AM      0.17      0.00      0.17      0.00     22.84
02:39:01 AM      0.23      0.00      0.23      0.00     13.36
02:40:01 AM      0.22      0.00      0.22      0.00     23.23
02:41:01 AM      0.15      0.00      0.15      0.00      9.10
02:42:01 AM      0.32      0.00      0.32      0.00     21.03
02:43:01 AM      0.15      0.00      0.15      0.00     17.73
02:44:01 AM      0.17      0.00      0.17      0.00      5.62
02:45:02 AM      0.20      0.00      0.20      0.00     18.54
02:46:01 AM      0.37      0.00      0.37      0.00     16.56
02:47:01 AM      0.43      0.00      0.43      0.00     11.70
02:48:01 AM      0.28      0.00      0.28      0.00     20.93
02:49:01 AM      0.28      0.00      0.28      0.00     10.45
02:50:01 AM      0.13      0.00      0.13      0.00     14.16
02:51:01 AM      0.22      0.00      0.22      0.00     10.71
02:52:01 AM      0.52      0.00      0.52      0.00     32.63
02:53:01 AM      0.42      0.00      0.42      0.00     15.96
02:54:01 AM      0.13      0.00      0.13      0.00     11.46
02:55:01 AM      0.13      0.00      0.13      0.00     22.26
02:56:01 AM      0.15      0.00      0.15      0.00     11.76
02:57:01 AM      0.33      0.00      0.33      0.00     26.10
02:58:01 AM      0.15      0.00      0.15      0.00     17.73
02:59:01 AM      0.13      0.00      0.13      0.00      9.52
03:00:01 AM      0.17      0.00      0.17      0.00      1.52
03:01:01 AM      0.28      0.00      0.28      0.00     21.01
03:02:01 AM      0.40      0.00      0.40      0.00     13.60
03:03:01 AM      0.17      0.00      0.17      0.00     15.86
03:04:01 AM      0.13      0.00      0.13      0.00     15.33
03:05:01 AM      0.18      0.00      0.18      0.00     10.38
03:06:01 AM      0.22      0.00      0.22      0.00     15.91
03:07:01 AM      0.62      0.00      0.62      0.00     33.04
03:08:01 AM      0.43      0.00      0.43      0.00     11.06
03:09:01 AM      0.28      0.00      0.28      0.00     16.11
03:10:01 AM      0.27      0.00      0.27      0.00     18.29
03:11:01 AM      0.20      0.00      0.20      0.00     12.23
03:12:01 AM      0.47      0.00      0.47      0.00     28.26
03:13:01 AM      0.13      0.00      0.13      0.00     11.46
03:14:01 AM      0.27      0.00      0.27      0.00     19.61
03:15:01 AM      0.18      0.00      0.18      0.00     14.45
03:16:01 AM      0.17      0.00      0.17      0.00     15.71
03:17:01 AM      0.22      0.00      0.22      0.00     12.05
03:18:01 AM      0.75      0.00      0.75      0.00    154.39
03:19:01 AM     17.51      0.33     17.18     39.99   4540.72
Average:         0.40      0.03      0.36      3.08     42.88

12:00:01 AM kbmemfree   kbavail kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit  kbactive   kbinact   kbdirty
12:01:01 AM   4927548   6972132   2954876     37.49      2688   2214496    746684      8.36    265028   2312408         4
12:02:01 AM   4927568   6972692   2954856     37.49      2688   2215044    746684      8.36    265112   2312536        96
12:03:01 AM   4927392   6972992   2955032     37.49      2688   2215520    746684      8.36    265112   2312484       252
12:04:01 AM   4926736   6972900   2955688     37.50      2688   2216076    746684      8.36    265336   2313084       144
12:05:01 AM   4926528   6972836   2955896     37.50      2688   2216220    746684      8.36    265336   2313344         4
12:06:01 AM   4926620   6972932   2955804     37.50      2688   2216228    746684      8.36    265336   2313352        16
12:07:01 AM   4926000   6972364   2956424     37.51      2688   2216280    730004      8.17    265336   2313836        64
12:08:01 AM   4925812   6972868   2956612     37.51      2688   2216972    748148      8.38    265376   2313996        68
12:09:01 AM   4925584   6972988   2956840     37.51      2688   2217328    748148      8.38    265376   2314344       176
12:10:01 AM   4924472   6972236   2957952     37.53      2688   2217688    738628      8.27    265376   2315216       140
12:11:01 AM   4924436   6972564   2957988     37.53      2688   2218044    738628      8.27    265380   2315340       136
12:12:01 AM   4923976   6972444   2958448     37.53      2688   2218384    738628      8.27    265376   2315716        88
12:13:01 AM   4923740   6972564   2958684     37.54      2688   2218748    738628      8.27    265376   2315892        60
12:14:01 AM   4923616   6972632   2958808     37.54      2688   2218932    738628      8.27    265412   2316048        56
12:15:01 AM   4923760   6972784   2958664     37.53      2688   2218940    738628      8.27    265412   2316056         8
12:16:01 AM   4923648   6972748   2958776     37.54      2688   2219072    738628      8.27    265412   2316132       136
12:17:01 AM   4922716   6972188   2959708     37.55      2688   2219396    738628      8.27    265412   2317028        56
12:18:01 AM   4922312   6972128   2960112     37.55      2688   2219740    738628      8.27    265412   2317464         4
12:19:01 AM   4922288   6972468   2960136     37.55      2688   2220104    738628      8.27    265412   2317560       184
12:20:01 AM   4921424   6971960   2961000     37.56      2688   2220452    740700      8.29    265412   2318276        24
12:21:01 AM   4920988   6971880   2961436     37.57      2688   2220808    740700      8.29    265412   2318880        16
12:22:01 AM   4920556   6971800   2961868     37.58      2688   2221160    740700      8.29    265412   2319184        40
12:23:01 AM   4919824   6971728   2962600     37.58      2688   2221828    740700      8.29    265412   2319784       136
12:24:01 AM   4919632   6971816   2962792     37.59      2688   2222100    740700      8.29    265412   2320084         4
12:25:01 AM   4918448   6971112   2963976     37.60      2688   2222580    740700      8.29    265412   2321428       168
12:26:01 AM   4917864   6971004   2964560     37.61      2688   2223056    740700      8.29    265412   2321984         4
12:27:01 AM   4917340   6970960   2965084     37.62      2688   2223536    740700      8.29    265412   2322616       172
12:28:01 AM   4916888   6970984   2965536     37.62      2688   2224012    740700      8.29    265412   2322972         4
12:29:02 AM   4916140   6970776   2966284     37.63      2688   2224544    740700      8.29    265412   2323452       232
12:30:01 AM   4915828   6970660   2966596     37.64      2688   2224740    732788      8.20    265412   2323744       168
12:31:01 AM   4915220   6970596   2967204     37.64      2688   2225284    732788      8.20    265548   2324304       104
12:32:01 AM   4914716   6970444   2967708     37.65      2688   2225644    732788      8.20    265548   2324724        72
12:33:01 AM   4914476   6970564   2967948     37.65      2688   2226004    732788      8.20    265548   2325268       212
12:34:01 AM   4914224   6970656   2968200     37.66      2688   2226340    732788      8.20    265548   2325484       160
12:35:01 AM   4913964   6970756   2968460     37.66      2688   2226700    732788      8.20    265548   2325548        96
12:36:01 AM   4913604   6970756   2968820     37.66      2688   2227060    732788      8.20    265548   2325936        64
12:37:01 AM   4913200   6970576   2969224     37.67      2688   2227284    732788      8.20    265548   2326380        48
12:38:01 AM   4912592   6970328   2969832     37.68      2688   2227644    732788      8.20    265548   2326924        20
12:39:01 AM   4911824   6969920   2970600     37.69      2688   2228004    732844      8.21    265548   2327524       200
12:40:01 AM   4894952   6970092   2987472     37.90      2688   2245004    732860      8.21    267812   2342148       112
12:41:01 AM   4894160   6969672   2988264     37.91      2688   2245372    732860      8.21    267812   2342840        60
12:42:01 AM   4894268   6970136   2988156     37.91      2688   2245720    732860      8.21    267812   2342868        40
12:43:01 AM   4893820   6970052   2988604     37.91      2688   2246080    732860      8.21    267812   2343368       184
12:44:01 AM   4893676   6970340   2988748     37.92      2688   2246512    732860      8.21    267812   2343500       208
12:45:01 AM   4893632   6970608   2988792     37.92      2688   2246824    748732      8.38    267812   2343676        52
12:46:01 AM   4892924   6970240   2989500     37.93      2688   2247164    748732      8.38    267812   2344216       204
12:47:01 AM   4892656   6970340   2989768     37.93      2688   2247532    748732      8.38    267812   2344652       184
12:48:01 AM   4892116   6970152   2990308     37.94      2688   2247892    748732      8.38    267812   2344872        56
12:49:01 AM   4891760   6970144   2990664     37.94      2688   2248232    748732      8.38    267812   2345484        12
12:50:01 AM   4891400   6970140   2991024     37.95      2688   2248588    767136      8.59    267812   2345860       176
12:51:01 AM   4890848   6970232   2991576     37.95      2688   2249240    739632      8.28    267840   2346336        24
12:52:01 AM   4890472   6970204   2991952     37.96      2688   2249588    739632      8.28    267840   2346752       244
12:53:01 AM   4890072   6970212   2992352     37.96      2688   2249988    739632      8.28    267840   2347012         4
12:54:01 AM   4889948   6970564   2992476     37.96      2688   2250464    739632      8.28    267840   2347372       168
12:55:01 AM   4889368   6970464   2993056     37.97      2688   2250944    739632      8.28    267840   2347968         4
12:56:01 AM   4888892   6970468   2993532     37.98      2688   2251420    739632      8.28    267840   2348328       164
12:57:01 AM   4887748   6969936   2994676     37.99      2688   2252032    739632      8.28    267840   2349220       116
12:58:01 AM   4887792   6970184   2994632     37.99      2688   2252236    739632      8.28    267840   2349404       120
12:59:01 AM   4887472   6970228   2994952     38.00      2688   2252604    739632      8.28    267840   2349788        24
01:00:01 AM   4886492   6969604   2995932     38.01      2688   2252952    734584      8.23    267840   2350576         4
01:01:01 AM   4884480   6967956   2997944     38.03      2688   2253308    734584      8.23    267844   2352316       104
01:02:01 AM   4884992   6968820   2997432     38.03      2688   2253668    734584      8.23    267848   2351952        24
01:03:01 AM   4884948   6969136   2997476     38.03      2688   2254020    734584      8.23    267848   2352108       192
01:04:01 AM   4884700   6969248   2997724     38.03      2688   2254380    734584      8.23    267848   2352320       136
01:05:01 AM   4884400   6969148   2998024     38.03      2688   2254580    734584      8.23    267848   2352764        24
01:06:01 AM   4884264   6969424   2998160     38.04      2688   2255000    734584      8.23    267848   2353004       200
01:07:01 AM   4883752   6969276   2998672     38.04      2688   2255364    734584      8.23    267848   2353452       136
01:08:01 AM   4883056   6968928   2999368     38.05      2688   2255704    734584      8.23    267848   2354092        72
01:09:01 AM   4883004   6969228   2999420     38.05      2688   2256056    734584      8.23    267848   2354224        24
01:10:01 AM   4883392   6969980   2999032     38.05      2688   2256420    754436      8.45    267852   2354212       168
01:11:01 AM   4882696   6969632   2999728     38.06      2688   2256768    754436      8.45    267844   2354700       136
01:12:01 AM   4881788   6969428   3000636     38.07      2688   2257476    738132      8.26    267844   2355480       236
01:13:01 AM   4881740   6969592   3000684     38.07      2688   2257680    738132      8.26    267844   2355624       184
01:14:01 AM   4881084   6969416   3001340     38.08      2688   2258160    738132      8.26    267844   2356104        88
01:15:01 AM   4880760   6969564   3001664     38.08      2688   2258640    738132      8.26    267844   2356576        64
01:16:01 AM   4880160   6969440   3002264     38.09      2688   2259116    738132      8.26    267844   2357168       136
01:17:01 AM   4879532   6969288   3002892     38.10      2688   2259592    738132      8.26    267844   2357596       152
01:18:01 AM   4879388   6969556   3003036     38.10      2688   2259996    738132      8.26    267844   2357940         4
01:19:01 AM   4877800   6968836   3004624     38.12      2688   2260872    738132      8.26    267844   2359636       632
01:20:01 AM   4871904   6965436   3010520     38.19      2688   2263372    756036      8.47    267844   2365188         4
01:21:01 AM   4870092   6964908   3012332     38.22      2688   2264644    756036      8.47    267852   2367088       160
01:22:01 AM   4869692   6964856   3012732     38.22      2688   2265000    756036      8.47    267852   2367596       108
01:23:01 AM   4868728   6964256   3013696     38.23      2688   2265364    756036      8.47    267852   2368480        76
01:24:01 AM   4867756   6963640   3014668     38.25      2688   2265708    756036      8.47    267852   2369268        16
01:25:01 AM   4867560   6963796   3014864     38.25      2688   2266056    756036      8.47    267852   2369728       160
01:26:01 AM   4867032   6963632   3015392     38.25      2688   2266420    756036      8.47    267852   2370304       128
01:27:01 AM   4866836   6963752   3015588     38.26      2688   2266736    756036      8.47    267852   2370368         4
01:28:01 AM   4864252   6962940   3018172     38.29      2688   2268508    756036      8.47    267852   2373000         4
01:29:01 AM   4859612   6961224   3022812     38.35      2688   2271432    756036      8.47    267852   2377564       780
01:30:01 AM   4858768   6960760   3023656     38.36      2688   2271808    749560      8.39    267852   2378460       156
01:31:01 AM   4858460   6960808   3023964     38.36      2688   2272172    749560      8.39    267852   2378764        80
01:32:01 AM   4858144   6960848   3024280     38.37      2688   2272528    749560      8.39    267852   2379072       180
01:33:01 AM   4857604   6960668   3024820     38.37      2688   2272880    749560      8.39    267852   2379580        32
01:34:01 AM   4857048   6960468   3025376     38.38      2688   2273244    749560      8.39    267852   2380084       184
01:35:02 AM   4857148   6960908   3025276     38.38      2688   2273584    749560      8.39    267852   2379900        40
01:36:01 AM   4855360   6961012   3027064     38.40      2688   2275464    749560      8.39    267852   2381808      1148
01:37:01 AM   4853436   6960856   3028988     38.43      2688   2277232    749560      8.39    267852   2383792       608
01:38:01 AM   4852396   6960136   3030028     38.44      2688   2277552    749560      8.39    267852   2384860       168
01:39:01 AM   4852248   6960348   3030176     38.44      2688   2277912    749560      8.39    267852   2384832       172
01:40:01 AM   4851476   6959912   3030948     38.45      2688   2278256    751612      8.42    267852   2385516        80
01:41:01 AM   4851004   6959808   3031420     38.46      2688   2278616    751612      8.42    267852   2386040        56
01:42:01 AM   4850796   6959960   3031628     38.46      2688   2278968    751612      8.42    267852   2386336        32
01:43:01 AM   4850816   6960332   3031608     38.46      2688   2279328    751612      8.42    267852   2386368        40
01:44:01 AM   4848996   6959900   3033428     38.48      2688   2280708    751612      8.42    267852   2388184      1188
01:45:01 AM   4846364   6960116   3036060     38.52      2688   2283552    751612      8.42    267852   2390620      1144
01:46:01 AM   4843940   6958660   3038484     38.55      2688   2284520    751612      8.42    267852   2393424        32
01:47:01 AM   4842636   6957720   3039788     38.56      2688   2284880    751612      8.42    267852   2394460       188
01:48:01 AM   4842544   6957976   3039880     38.57      2688   2285236    751612      8.42    267852   2394672       124
01:49:01 AM   4842192   6957988   3040232     38.57      2688   2285592    751612      8.42    267852   2395036        72
01:50:01 AM   4841468   6957608   3040956     38.58      2688   2285936    744112      8.33    267852   2395736       208
01:51:01 AM   4841212   6957704   3041212     38.58      2688   2286296    744112      8.33    267852   2395936       152
01:52:01 AM   4840592   6957792   3041832     38.59      2688   2286996    744112      8.33    267852   2396544       128
01:53:01 AM   4840416   6957892   3042008     38.59      2688   2287280    744112      8.33    267852   2396868        92
01:54:01 AM   4840180   6958060   3042244     38.60      2688   2287684    744112      8.33    267852   2397156       168
01:55:01 AM   4839416   6957772   3043008     38.60      2688   2288160    744112      8.33    267852   2397632         4
01:56:01 AM   4839124   6957956   3043300     38.61      2688   2288636    744112      8.33    267852   2398108       168
01:57:01 AM   4838440   6957752   3043984     38.62      2688   2289116    744112      8.33    267852   2398724        32
01:58:01 AM   4837688   6957588   3044736     38.63      2688   2289704    759580      8.50    267852   2399468       312
01:59:01 AM   4837748   6957880   3044676     38.63      2688   2289928    759580      8.50    267852   2399436       128
02:00:01 AM   4836748   6957228   3045676     38.64      2688   2290280    752336      8.42    267852   2400120        40
02:01:01 AM   4836148   6956996   3046276     38.65      2688   2290636    752336      8.42    267852   2400832       120
02:02:01 AM   4836440   6957640   3045984     38.64      2688   2290988    752336      8.42    267856   2400552        88
02:03:01 AM   4834516   6956064   3047908     38.67      2688   2291344    752748      8.43    267856   2402312        52
02:04:01 AM   4835644   6957560   3046780     38.65      2688   2291700    767764      8.60    277052   2392184        68
02:05:01 AM   4835292   6957564   3047132     38.66      2688   2292064    751636      8.42    277052   2392752       176
02:06:01 AM   4835296   6957796   3047128     38.66      2688   2292284    751636      8.42    277052   2392600         4
02:07:01 AM   4834808   6957676   3047616     38.66      2688   2292652    751636      8.42    277052   2393036         4
02:08:01 AM   4834580   6957812   3047844     38.67      2688   2293016    751636      8.42    277052   2393332       132
02:09:01 AM   4834204   6957736   3048220     38.67      2688   2293324    751636      8.42    277052   2393748        24
02:10:01 AM   4832812   6956740   3049612     38.69      2688   2293684    744452      8.34    277052   2394784       148
02:11:01 AM   4831940   6956276   3050484     38.70      2688   2294040    744452      8.34    277052   2395816        68
02:12:01 AM   4831428   6956116   3050996     38.71      2688   2294396    744452      8.34    277052   2396348        48
02:13:01 AM   4831080   6956180   3051344     38.71      2688   2294840    744452      8.34    277052   2396672       212
02:14:01 AM   4830776   6956216   3051648     38.71      2688   2295148    744452      8.34    277052   2396944        48
02:15:01 AM   4830468   6956260   3051956     38.72      2688   2295492    744452      8.34    277052   2397364        48
02:16:01 AM   4830312   6956460   3052112     38.72      2688   2295848    744452      8.34    277052   2397608        16
02:17:01 AM   4829580   6956084   3052844     38.73      2688   2296204    744452      8.34    277052   2398032        56
02:18:01 AM   4829288   6956128   3053136     38.73      2688   2296548    744452      8.34    277052   2398300        40
02:19:01 AM   4828972   6956184   3053452     38.74      2688   2296912    760052      8.51    277052   2398788       128
02:20:01 AM   4827924   6955780   3054500     38.75      2688   2297552    753092      8.43    277052   2399660        24
02:21:01 AM   4827892   6956100   3054532     38.75      2688   2297900    753092      8.43    277052   2399840        88
02:22:01 AM   4827204   6955892   3055220     38.76      2688   2298372    753092      8.43    277052   2400388        80
02:23:01 AM   4826992   6956080   3055432     38.76      2688   2298780    753092      8.43    277052   2400720       248
02:24:01 AM   4826432   6955996   3055992     38.77      2688   2299256    753092      8.43    277052   2401196       168
02:25:01 AM   4826292   6956336   3056132     38.77      2688   2299736    753092      8.43    277052   2401488        56
02:26:01 AM   4825368   6956016   3057056     38.78      2688   2300332    753092      8.43    277052   2402216       136
02:27:01 AM   4825224   6956160   3057200     38.79      2688   2300620    753092      8.43    277060   2402448       240
02:28:01 AM   4825096   6956192   3057328     38.79      2688   2300780    753092      8.43    277060   2402540        60
02:29:01 AM   4824496   6955940   3057928     38.79      2688   2301128    753092      8.43    277064   2403088        84
02:30:01 AM   4824272   6956076   3058152     38.80      2688   2301484    752800      8.43    277064   2403580       124
02:31:01 AM   4823696   6955864   3058728     38.80      2688   2301848    752800      8.43    277064   2403892        64
02:32:01 AM   4823352   6955864   3059072     38.81      2688   2302192    752800      8.43    277064   2404188       176
02:33:01 AM   4823068   6955944   3059356     38.81      2688   2302552    752800      8.43    277064   2404616       140
02:34:01 AM   4822884   6956104   3059540     38.81      2688   2302888    752800      8.43    277064   2404772        48
02:35:01 AM   4822448   6956020   3059976     38.82      2688   2303248    752800      8.43    277064   2405104        48
02:36:01 AM   4822052   6956224   3060372     38.83      2688   2303840    752800      8.43    277064   2405716         4
02:37:01 AM   4821568   6956152   3060856     38.83      2688   2304252    752800      8.43    277064   2406196       260
02:38:01 AM   4821356   6956408   3061068     38.83      2688   2304728    752800      8.43    277064   2406480        92
02:39:01 AM   4820676   6956208   3061748     38.84      2688   2305208    752800      8.43    277064   2407076       244
02:40:01 AM   4819728   6955668   3062696     38.85      2688   2305608    754780      8.45    277064   2407716         4
02:41:01 AM   4819276   6955704   3063148     38.86      2688   2306088    754780      8.45    277064   2408044       248
02:42:01 AM   4818940   6955864   3063484     38.86      2688   2306580    754780      8.45    277064   2408612       272
02:43:01 AM   4818892   6956120   3063532     38.87      2688   2306884    754780      8.45    277072   2408648        76
02:44:01 AM   4818716   6956116   3063708     38.87      2688   2307056    754780      8.45    277072   2408820       108
02:45:02 AM   4818032   6955864   3064392     38.88      2688   2307496    754780      8.45    277072   2409456        48
02:46:01 AM   4817844   6956020   3064580     38.88      2688   2307840    754780      8.45    277072   2409596        48
02:47:01 AM   4817408   6955928   3065016     38.88      2688   2308184    770548      8.63    277072   2410128       208
02:48:01 AM   4817096   6955976   3065328     38.89      2688   2308536    755156      8.46    277072   2410516        24
02:49:01 AM   4817160   6956392   3065264     38.89      2688   2308896    755156      8.46    277072   2410668       176
02:50:01 AM   4815920   6955504   3066504     38.90      2688   2309240    752804      8.43    277072   2411372       132
02:51:01 AM   4815768   6955736   3066656     38.90      2688   2309616    752804      8.43    277072   2411724       232
02:52:01 AM   4815392   6955888   3067032     38.91      2688   2310152    752804      8.43    277072   2412000         4
02:53:01 AM   4815132   6956104   3067292     38.91      2688   2310628    752804      8.43    277072   2412408       168
02:54:01 AM   4814472   6955924   3067952     38.92      2688   2311108    752804      8.43    277072   2412888       328
02:55:01 AM   4814152   6956084   3068272     38.93      2688   2311588    752804      8.43    277072   2413504       172
02:56:01 AM   4813608   6955944   3068816     38.93      2688   2311984    752804      8.43    277072   2413792       244
02:57:01 AM   4813176   6955992   3069248     38.94      2688   2312460    752804      8.43    277072   2414472        88
02:58:01 AM   4812832   6956096   3069592     38.94      2688   2312908    752804      8.43    277072   2414728        40
02:59:01 AM   4812444   6955920   3069980     38.95      2688   2313120    752804      8.43    277076   2415140        16
03:00:01 AM   4811988   6955512   3070436     38.95      2688   2313168    768680      8.61    277076   2415332        56
03:01:01 AM   4810380   6954412   3072044     38.97      2688   2313684    768680      8.61    277080   2416488        44
03:02:01 AM   4811688   6956060   3070736     38.96      2688   2314024    768680      8.61    277080   2416008       168
03:03:01 AM   4811316   6956048   3071108     38.96      2688   2314376    768680      8.61    277080   2416184        80
03:04:01 AM   4810756   6955844   3071668     38.97      2688   2314732    753000      8.43    277080   2416744        20
03:05:01 AM   4810508   6955936   3071916     38.97      2688   2315080    753000      8.43    277080   2417032       132
03:06:01 AM   4810368   6956148   3072056     38.97      2688   2315424    753000      8.43    277080   2417252        56
03:07:01 AM   4809620   6956108   3072804     38.98      2688   2316140    753000      8.43    277080   2418028        32
03:08:01 AM   4809312   6955980   3073112     38.99      2688   2316312    753000      8.43    277080   2418140         4
03:09:01 AM   4808996   6956140   3073428     38.99      2688   2316788    753000      8.43    277080   2418616        88
03:10:01 AM   4808048   6955672   3074376     39.00      2688   2317268    752868      8.43    277080   2419444        88
03:11:01 AM   4807992   6956104   3074432     39.00      2688   2317748    752868      8.43    277080   2419576       244
03:12:01 AM   4807432   6956020   3074992     39.01      2688   2318232    752868      8.43    277080   2420120        88
03:13:01 AM   4807184   6956248   3075240     39.01      2688   2318708    752868      8.43    277080   2420532       248
03:14:01 AM   4806492   6955992   3075932     39.02      2688   2319136    752868      8.43    277080   2421012       184
03:15:01 AM   4805420   6955268   3077004     39.04      2688   2319476    794748      8.90    277080   2421624       144
03:16:01 AM   4802292   6952536   3080132     39.08      2688   2319880    828128      9.27    277080   2424388       112
03:17:01 AM   4801680   6952172   3080744     39.08      2688   2320112    843508      9.44    277080   2424876        64
03:18:01 AM   4868416   7023532   3014008     38.24      2688   2324744    652596      7.31    278080   2357984       276
03:19:01 AM   4769972   6878108   3112452     39.49      2688   2282564    770676      8.63    555004   2201076     14048
Average:      4860048   6962719   3022376     38.34      2688   2272482    748026      8.38    272290   2372673       200

12:00:01 AM     IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
12:01:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:01:01 AM      eth0      1.07      0.98      0.35      0.21      0.00      0.00      0.00      0.00
12:02:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:02:01 AM      eth0      1.48      1.23      0.46      0.29      0.00      0.00      0.00      0.00
12:03:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:03:01 AM      eth0      1.25      1.08      0.39      0.22      0.00      0.00      0.00      0.00
12:04:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:04:01 AM      eth0      8.66      8.76      1.82      1.13      0.00      0.00      0.00      0.00
12:05:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:05:01 AM      eth0      2.73      2.55      0.61      0.27      0.00      0.00      0.00      0.00
12:06:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:06:01 AM      eth0      0.70      0.13      0.15      0.01      0.00      0.00      0.00      0.00
12:07:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:07:01 AM      eth0      3.13      2.83      0.59      0.31      0.00      0.00      0.00      0.00
12:08:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:08:01 AM      eth0     11.05     11.01      2.37      2.04      0.00      0.00      0.00      0.00
12:09:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:09:01 AM      eth0      9.55      9.80      2.03      1.82      0.00      0.00      0.00      0.00
12:10:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:10:01 AM      eth0      9.60      9.73      2.03      1.81      0.00      0.00      0.00      0.00
12:11:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:11:01 AM      eth0      9.51      9.75      2.02      1.81      0.00      0.00      0.00      0.00
12:12:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:12:01 AM      eth0      9.58      9.57      2.17      1.83      0.00      0.00      0.00      0.00
12:13:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:13:01 AM      eth0     10.73      9.60      2.11      1.78      0.00      0.00      0.00      0.00
12:14:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:14:01 AM      eth0      3.08      2.85      0.57      0.52      0.00      0.00      0.00      0.00
12:15:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:15:01 AM      eth0      0.22      0.02      0.01      0.00      0.00      0.00      0.00      0.00
12:16:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:16:01 AM      eth0      1.67      1.37      0.30      0.26      0.00      0.00      0.00      0.00
12:17:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:17:01 AM      eth0      7.73      7.08      1.59      1.36      0.00      0.00      0.00      0.00
12:18:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:18:01 AM      eth0      9.66      9.53      2.24      1.93      0.00      0.00      0.00      0.00
12:19:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:19:01 AM      eth0      9.52      9.72      2.02      1.81      0.00      0.00      0.00      0.00
12:20:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:20:01 AM      eth0     13.31      9.70      2.53      1.81      0.00      0.00      0.00      0.00
12:21:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:21:01 AM      eth0      9.48      9.53      2.01      1.79      0.00      0.00      0.00      0.00
12:22:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:22:01 AM      eth0      9.73      9.58      2.11      1.82      0.00      0.00      0.00      0.00
12:23:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:23:01 AM      eth0     17.13     17.11      3.82      3.17      0.00      0.00      0.00      0.00
12:24:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:24:01 AM      eth0      4.53      0.83      0.64      0.14      0.00      0.00      0.00      0.00
12:25:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:25:01 AM      eth0      1.87      1.43      0.73      0.49      0.00      0.00      0.00      0.00
12:26:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:26:01 AM      eth0      2.38      1.27      0.76      0.41      0.00      0.00      0.00      0.00
12:27:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:27:01 AM      eth0      6.32      2.25      1.53      3.32      0.00      0.00      0.00      0.00
12:28:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:28:01 AM      eth0      8.35      1.17      1.34      0.41      0.00      0.00      0.00      0.00
12:29:02 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:29:02 AM      eth0      7.50      4.88      1.57      1.13      0.00      0.00      0.00      0.00
12:30:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:30:01 AM      eth0     12.93      9.41      2.32      0.77      0.00      0.00      0.00      0.00
12:31:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:31:01 AM      eth0      9.51      9.25      1.93      1.70      0.00      0.00      0.00      0.00
12:32:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:32:01 AM      eth0     11.18      9.88      2.29      1.88      0.00      0.00      0.00      0.00
12:33:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:33:01 AM      eth0     10.91     10.25      2.72      2.27      0.00      0.00      0.00      0.00
12:34:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:34:01 AM      eth0      9.63      9.62      2.03      1.80      0.00      0.00      0.00      0.00
12:35:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:35:01 AM      eth0      9.53      9.37      2.01      1.74      0.00      0.00      0.00      0.00
12:36:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:36:01 AM      eth0     12.55      9.81      2.15      1.82      0.00      0.00      0.00      0.00
12:37:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:37:01 AM      eth0     11.01      5.17      2.00      1.22      0.00      0.00      0.00      0.00
12:38:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:38:01 AM      eth0      8.05      7.72      2.04      1.77      0.00      0.00      0.00      0.00
12:39:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:39:01 AM      eth0     10.33      9.78      2.13      1.81      0.00      0.00      0.00      0.00
12:40:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:40:01 AM      eth0     15.73     10.53      2.80      1.88      0.00      0.00      0.00      0.00
12:41:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:41:01 AM      eth0     10.91      9.43      2.39      1.92      0.00      0.00      0.00      0.00
12:42:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:42:01 AM      eth0     10.56      9.78      2.18      1.87      0.00      0.00      0.00      0.00
12:43:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:43:01 AM      eth0     10.53      9.85      2.40      2.00      0.00      0.00      0.00      0.00
12:44:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:44:01 AM      eth0      8.35      7.43      1.79      1.55      0.00      0.00      0.00      0.00
12:45:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:45:01 AM      eth0      7.85      7.10      1.61      1.31      0.00      0.00      0.00      0.00
12:46:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:46:01 AM      eth0     10.05      9.67      2.25      1.96      0.00      0.00      0.00      0.00
12:47:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:47:01 AM      eth0     13.28     10.05      2.75      2.05      0.00      0.00      0.00      0.00
12:48:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:48:01 AM      eth0     10.51      9.93      2.35      2.01      0.00      0.00      0.00      0.00
12:49:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:49:01 AM      eth0     12.98      9.65      2.64      1.98      0.00      0.00      0.00      0.00
12:50:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:50:01 AM      eth0     11.73      9.45      2.18      1.76      0.00      0.00      0.00      0.00
12:51:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:51:01 AM      eth0     16.88     16.38      3.69      3.01      0.00      0.00      0.00      0.00
12:52:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:52:01 AM      eth0      1.35      1.08      0.37      0.24      0.00      0.00      0.00      0.00
12:53:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:53:01 AM      eth0      1.35      1.08      0.48      0.22      0.00      0.00      0.00      0.00
12:54:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:54:01 AM      eth0      1.15      0.83      0.30      0.18      0.00      0.00      0.00      0.00
12:55:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:55:01 AM      eth0      1.25      1.00      0.36      0.22      0.00      0.00      0.00      0.00
12:56:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:56:01 AM      eth0      1.60      1.07      0.40      0.22      0.00      0.00      0.00      0.00
12:57:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:57:01 AM      eth0      8.37      4.70      1.83      1.12      0.00      0.00      0.00      0.00
12:58:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:58:01 AM      eth0     11.31      8.98      2.23      0.90      0.00      0.00      0.00      0.00
12:59:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
12:59:01 AM      eth0     10.10      9.83      2.26      1.99      0.00      0.00      0.00      0.00
01:00:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:00:01 AM      eth0     13.15      9.66      2.49      1.81      0.00      0.00      0.00      0.00
01:01:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:01:01 AM      eth0     10.29      9.56      2.14      1.80      0.00      0.00      0.00      0.00
01:02:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:02:01 AM      eth0     10.08      9.63      2.32      2.02      0.00      0.00      0.00      0.00
01:03:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:03:01 AM      eth0      9.53      9.58      2.00      1.78      0.00      0.00      0.00      0.00
01:04:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:04:01 AM      eth0     10.51      9.82      2.16      1.82      0.00      0.00      0.00      0.00
01:05:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:05:01 AM      eth0      4.78      4.27      1.16      1.00      0.00      0.00      0.00      0.00
01:06:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:06:01 AM      eth0      7.10      6.93      1.40      1.28      0.00      0.00      0.00      0.00
01:07:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:07:01 AM      eth0     10.03      9.81      2.14      1.87      0.00      0.00      0.00      0.00
01:08:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:08:01 AM      eth0     11.30      9.63      2.35      1.96      0.00      0.00      0.00      0.00
01:09:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:09:01 AM      eth0     12.41      9.55      2.12      1.78      0.00      0.00      0.00      0.00
01:10:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:10:01 AM      eth0     10.14      9.66      2.09      1.81      0.00      0.00      0.00      0.00
01:11:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:11:01 AM      eth0      9.51      9.69      2.02      1.81      0.00      0.00      0.00      0.00
01:12:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:12:01 AM      eth0     18.51     18.18      4.15      3.42      0.00      0.00      0.00      0.00
01:13:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:13:01 AM      eth0      0.82      0.62      0.16      0.10      0.00      0.00      0.00      0.00
01:14:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:14:01 AM      eth0      1.13      1.03      0.36      0.22      0.00      0.00      0.00      0.00
01:15:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:15:01 AM      eth0      1.40      1.15      0.39      0.25      0.00      0.00      0.00      0.00
01:16:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:16:01 AM      eth0      1.18      1.10      0.38      0.22      0.00      0.00      0.00      0.00
01:17:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:17:01 AM      eth0      1.52      1.23      0.51      0.31      0.00      0.00      0.00      0.00
01:18:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:18:01 AM      eth0      1.02      0.87      0.31      0.18      0.00      0.00      0.00      0.00
01:19:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:19:01 AM      eth0     11.48     12.00      2.62      1.44      0.00      0.00      0.00      0.00
01:20:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:20:01 AM      eth0      2.83      4.00      1.89      0.72      0.00      0.00      0.00      0.00
01:21:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:21:01 AM      eth0      9.02      9.72      2.37      1.56      0.00      0.00      0.00      0.00
01:22:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:22:01 AM      eth0      9.68      9.75      2.10      1.87      0.00      0.00      0.00      0.00
01:23:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:23:01 AM      eth0      9.73      9.65      2.08      1.81      0.00      0.00      0.00      0.00
01:24:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:24:01 AM      eth0      9.67      9.65      2.18      1.80      0.00      0.00      0.00      0.00
01:25:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:25:01 AM      eth0      9.75      9.46      2.02      1.75      0.00      0.00      0.00      0.00
01:26:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:26:01 AM      eth0      9.93      9.98      2.28      2.01      0.00      0.00      0.00      0.00
01:27:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:27:01 AM      eth0      7.80      7.95      1.72      1.56      0.00      0.00      0.00      0.00
01:28:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:28:01 AM      eth0      1.83      1.83      0.14      0.25      0.00      0.00      0.00      0.00
01:29:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:29:01 AM      eth0      2.57      2.82      0.44      0.60      0.00      0.00      0.00      0.00
01:30:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:30:01 AM      eth0      9.95     10.10      2.11      1.88      0.00      0.00      0.00      0.00
01:31:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:31:01 AM      eth0      9.61      9.73      2.03      1.81      0.00      0.00      0.00      0.00
01:32:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:32:01 AM      eth0     10.18      9.68      2.16      1.87      0.00      0.00      0.00      0.00
01:33:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:33:01 AM      eth0     10.38      9.93      2.44      2.01      0.00      0.00      0.00      0.00
01:34:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:34:01 AM      eth0      9.83      9.85      2.28      2.00      0.00      0.00      0.00      0.00
01:35:02 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:35:02 AM      eth0      9.77      9.55      2.08      1.79      0.00      0.00      0.00      0.00
01:36:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:36:01 AM      eth0      3.41      3.52      0.50      0.58      0.00      0.00      0.00      0.00
01:37:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:37:01 AM      eth0      3.40      1.70      0.44      0.36      0.00      0.00      0.00      0.00
01:38:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:38:01 AM      eth0      9.23      8.88      2.12      1.85      0.00      0.00      0.00      0.00
01:39:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:39:01 AM      eth0      9.75      9.80      2.09      1.82      0.00      0.00      0.00      0.00
01:40:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:40:01 AM      eth0     12.53      9.43      2.33      1.78      0.00      0.00      0.00      0.00
01:41:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:41:01 AM      eth0     11.98      9.96      2.78      2.21      0.00      0.00      0.00      0.00
01:42:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:42:01 AM      eth0     10.16      9.97      2.36      2.07      0.00      0.00      0.00      0.00
01:43:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:43:01 AM      eth0     14.46      9.62      2.68      1.80      0.00      0.00      0.00      0.00
01:44:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:44:01 AM      eth0      6.60      5.31      1.31      1.11      0.00      0.00      0.00      0.00
01:45:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:45:01 AM      eth0      2.25      2.15      0.15      0.30      0.00      0.00      0.00      0.00
01:46:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:46:01 AM      eth0      7.68      7.15      1.43      1.34      0.00      0.00      0.00      0.00
01:47:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:47:01 AM      eth0     11.50     10.10      2.82      2.31      0.00      0.00      0.00      0.00
01:48:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:48:01 AM      eth0     10.93      9.93      2.24      1.83      0.00      0.00      0.00      0.00
01:49:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:49:01 AM      eth0     11.98      9.75      2.30      1.81      0.00      0.00      0.00      0.00
01:50:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:50:01 AM      eth0     10.60      9.55      2.17      1.79      0.00      0.00      0.00      0.00
01:51:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:51:01 AM      eth0      9.50      9.48      1.98      1.75      0.00      0.00      0.00      0.00
01:52:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:52:01 AM      eth0     18.46     18.21      4.19      3.42      0.00      0.00      0.00      0.00
01:53:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:53:01 AM      eth0      1.12      0.85      0.31      0.14      0.00      0.00      0.00      0.00
01:54:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:54:01 AM      eth0      1.10      0.87      0.30      0.18      0.00      0.00      0.00      0.00
01:55:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:55:01 AM      eth0      1.22      1.12      0.38      0.22      0.00      0.00      0.00      0.00
01:56:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:56:01 AM      eth0      1.48      1.03      0.37      0.22      0.00      0.00      0.00      0.00
01:57:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:57:01 AM      eth0      5.55      1.67      1.28      0.58      0.00      0.00      0.00      0.00
01:58:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:58:01 AM      eth0      5.20      4.43      1.86      1.33      0.00      0.00      0.00      0.00
01:59:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
01:59:01 AM      eth0     11.85      9.77      2.24      0.85      0.00      0.00      0.00      0.00
02:00:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:00:01 AM      eth0     15.00      9.36      2.66      1.73      0.00      0.00      0.00      0.00
02:01:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:01:01 AM      eth0      9.95      9.72      2.05      1.81      0.00      0.00      0.00      0.00
02:02:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:02:01 AM      eth0     10.33      9.73      2.15      1.87      0.00      0.00      0.00      0.00
02:03:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:03:01 AM      eth0     10.38      9.60      2.47      2.02      0.00      0.00      0.00      0.00
02:04:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:04:01 AM      eth0     10.08      9.65      2.05      1.79      0.00      0.00      0.00      0.00
02:05:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:05:01 AM      eth0     10.78     10.08      2.48      2.08      0.00      0.00      0.00      0.00
02:06:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:06:01 AM      eth0      5.23      4.80      1.04      0.89      0.00      0.00      0.00      0.00
02:07:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:07:01 AM      eth0      7.88      7.48      1.65      1.46      0.00      0.00      0.00      0.00
02:08:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:08:01 AM      eth0      9.56      9.63      2.02      1.80      0.00      0.00      0.00      0.00
02:09:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:09:01 AM      eth0      9.20      8.82      1.96      1.63      0.00      0.00      0.00      0.00
02:10:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:10:01 AM      eth0     47.25     28.77     28.40     18.96      0.00      0.00      0.00      0.00
02:11:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:11:01 AM      eth0     38.56     35.71     18.52     67.26      0.00      0.00      0.00      0.00
02:12:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:12:01 AM      eth0     10.13      9.96      2.38      2.07      0.00      0.00      0.00      0.00
02:13:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:13:01 AM      eth0      7.22      7.25      1.46      1.42      0.00      0.00      0.00      0.00
02:14:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:14:01 AM      eth0      7.08      7.05      1.45      1.31      0.00      0.00      0.00      0.00
02:15:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:15:01 AM      eth0      9.36      9.50      2.09      1.77      0.00      0.00      0.00      0.00
02:16:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:16:01 AM      eth0      9.50      9.63      2.00      1.78      0.00      0.00      0.00      0.00
02:17:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:17:01 AM      eth0      9.76      9.83      2.14      1.90      0.00      0.00      0.00      0.00
02:18:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:18:01 AM      eth0      9.17      9.27      1.94      1.73      0.00      0.00      0.00      0.00
02:19:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:19:01 AM      eth0      9.56      9.70      2.08      1.81      0.00      0.00      0.00      0.00
02:20:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:20:01 AM      eth0     16.45     16.35      3.66      3.00      0.00      0.00      0.00      0.00
02:21:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:21:01 AM      eth0      1.07      0.95      0.27      0.17      0.00      0.00      0.00      0.00
02:22:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:22:01 AM      eth0      1.47      1.10      0.48      0.28      0.00      0.00      0.00      0.00
02:23:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:23:01 AM      eth0      0.93      0.90      0.30      0.19      0.00      0.00      0.00      0.00
02:24:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:24:01 AM      eth0      1.30      1.13      0.38      0.23      0.00      0.00      0.00      0.00
02:25:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:25:01 AM      eth0      1.12      1.00      0.35      0.22      0.00      0.00      0.00      0.00
02:26:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:26:01 AM      eth0      4.22      4.30      1.08      0.82      0.00      0.00      0.00      0.00
02:27:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:27:01 AM      eth0     11.28     11.11      2.14      0.96      0.00      0.00      0.00      0.00
02:28:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:28:01 AM      eth0      7.30      6.65      1.65      0.61      0.00      0.00      0.00      0.00
02:29:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:29:01 AM      eth0     10.85     10.65      2.24      1.34      0.00      0.00      0.00      0.00
02:30:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:30:01 AM      eth0      9.63      9.86      2.04      1.83      0.00      0.00      0.00      0.00
02:31:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:31:01 AM      eth0      9.48      9.70      2.02      1.81      0.00      0.00      0.00      0.00
02:32:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:32:01 AM      eth0      9.93      9.40      2.08      1.80      0.00      0.00      0.00      0.00
02:33:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:33:01 AM      eth0     10.51      9.97      2.36      2.03      0.00      0.00      0.00      0.00
02:34:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:34:01 AM      eth0      9.73      9.58      2.29      2.00      0.00      0.00      0.00      0.00
02:35:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:35:01 AM      eth0      9.51      9.60      2.06      1.80      0.00      0.00      0.00      0.00
02:36:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:36:01 AM      eth0     14.88     14.30      3.44      2.64      0.00      0.00      0.00      0.00
02:37:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:37:01 AM      eth0      1.32      1.10      0.40      0.26      0.00      0.00      0.00      0.00
02:38:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:38:01 AM      eth0      1.15      1.00      0.35      0.22      0.00      0.00      0.00      0.00
02:39:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:39:01 AM      eth0      1.08      1.03      0.36      0.22      0.00      0.00      0.00      0.00
02:40:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:40:01 AM      eth0      0.97      0.83      0.29      0.18      0.00      0.00      0.00      0.00
02:41:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:41:01 AM      eth0      1.20      1.05      0.36      0.22      0.00      0.00      0.00      0.00
02:42:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:42:01 AM      eth0      8.37      8.17      1.85      1.16      0.00      0.00      0.00      0.00
02:43:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:43:01 AM      eth0      8.75      8.68      1.68      0.73      0.00      0.00      0.00      0.00
02:44:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:44:01 AM      eth0     11.83     11.30      1.91      0.92      0.00      0.00      0.00      0.00
02:45:02 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:45:02 AM      eth0     11.05     10.83      2.50      1.82      0.00      0.00      0.00      0.00
02:46:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:46:01 AM      eth0      9.37      9.51      1.98      1.77      0.00      0.00      0.00      0.00
02:47:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:47:01 AM      eth0      9.95      9.67      2.21      1.84      0.00      0.00      0.00      0.00
02:48:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:48:01 AM      eth0      9.75      9.83      2.25      1.98      0.00      0.00      0.00      0.00
02:49:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:49:01 AM      eth0      9.58      9.75      2.05      1.81      0.00      0.00      0.00      0.00
02:50:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:50:01 AM      eth0      9.31      9.43      1.95      1.74      0.00      0.00      0.00      0.00
02:51:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:51:01 AM      eth0     10.00     10.36      2.15      1.96      0.00      0.00      0.00      0.00
02:52:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:52:01 AM      eth0     11.51     10.88      2.70      1.98      0.00      0.00      0.00      0.00
02:53:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:53:01 AM      eth0      1.05      1.10      0.35      0.22      0.00      0.00      0.00      0.00
02:54:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:54:01 AM      eth0      1.25      1.13      0.37      0.23      0.00      0.00      0.00      0.00
02:55:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:55:01 AM      eth0      1.47      1.00      0.44      0.22      0.00      0.00      0.00      0.00
02:56:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:56:01 AM      eth0      1.18      1.00      0.35      0.22      0.00      0.00      0.00      0.00
02:57:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:57:01 AM      eth0      1.85      1.25      0.75      0.51      0.00      0.00      0.00      0.00
02:58:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:58:01 AM      eth0     11.31     11.25      2.30      1.29      0.00      0.00      0.00      0.00
02:59:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
02:59:01 AM      eth0      3.95      3.15      0.96      0.57      0.00      0.00      0.00      0.00
03:00:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:00:01 AM      eth0      4.30      2.88      0.78      0.24      0.00      0.00      0.00      0.00
03:01:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:01:01 AM      eth0     13.46     13.46      2.60      1.94      0.00      0.00      0.00      0.00
03:02:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:02:01 AM      eth0     10.15      9.78      2.19      1.87      0.00      0.00      0.00      0.00
03:03:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:03:01 AM      eth0     11.88      9.35      2.25      1.74      0.00      0.00      0.00      0.00
03:04:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:04:01 AM      eth0     12.28     10.10      2.62      2.08      0.00      0.00      0.00      0.00
03:05:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:05:01 AM      eth0      9.68      9.70      2.27      1.94      0.00      0.00      0.00      0.00
03:06:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:06:01 AM      eth0     12.25      9.70      2.40      1.81      0.00      0.00      0.00      0.00
03:07:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:07:01 AM      eth0     22.83     18.23      4.66      3.42      0.00      0.00      0.00      0.00
03:08:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:08:01 AM      eth0      1.42      0.58      0.19      0.09      0.00      0.00      0.00      0.00
03:09:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:09:01 AM      eth0      1.65      1.07      0.48      0.22      0.00      0.00      0.00      0.00
03:10:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:10:01 AM      eth0      1.30      1.07      0.37      0.22      0.00      0.00      0.00      0.00
03:11:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:11:01 AM      eth0      1.62      1.00      0.39      0.22      0.00      0.00      0.00      0.00
03:12:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:12:01 AM      eth0      2.48      1.15      0.52      0.28      0.00      0.00      0.00      0.00
03:13:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:13:01 AM      eth0      2.23      1.00      0.44      0.22      0.00      0.00      0.00      0.00
03:14:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:14:01 AM      eth0     14.68     14.15      3.04      1.82      0.00      0.00      0.00      0.00
03:15:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:15:01 AM      eth0      8.17      8.10      1.81      1.38      0.00      0.00      0.00      0.00
03:16:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:16:01 AM      eth0     15.48     15.21      2.90      1.90      0.00      0.00      0.00      0.00
03:17:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:17:01 AM      eth0     14.63     11.35      2.59      1.24      0.00      0.00      0.00      0.00
03:18:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:18:01 AM      eth0     32.58     22.38     12.89      3.47      0.00      0.00      0.00      0.00
03:19:01 AM        lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
03:19:01 AM      eth0    130.82     97.43    336.98    441.83      0.00      0.00      0.00      0.00
Average:           lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
Average:         eth0      9.05      7.98      3.66      3.98      0.00      0.00      0.00      0.00


---> sar -P ALL:
Linux 4.18.0-553.5.1.el8.x86_64 (prd-centos8-robot-2c-8g-20286.novalocal) 	07/14/2024 	_x86_64_	(2 CPU)

12:00:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
12:01:01 AM     all      1.13      0.02      0.29      0.02      0.07     98.48
12:01:01 AM       0      0.89      0.03      0.40      0.02      0.08     98.58
12:01:01 AM       1      1.37      0.00      0.18      0.02      0.05     98.38
12:02:01 AM     all      1.34      0.00      0.13      0.03      0.07     98.44
12:02:01 AM       0      0.99      0.00      0.12      0.05      0.07     98.78
12:02:01 AM       1      1.68      0.00      0.15      0.00      0.07     98.10
12:03:01 AM     all      1.23      0.00      0.16      0.00      0.05     98.56
12:03:01 AM       0      0.70      0.00      0.13      0.00      0.03     99.13
12:03:01 AM       1      1.75      0.00      0.18      0.00      0.07     98.00
12:04:01 AM     all      3.33      0.00      0.22      0.00      0.07     96.38
12:04:01 AM       0      1.64      0.00      0.17      0.00      0.05     98.14
12:04:01 AM       1      5.04      0.00      0.27      0.00      0.09     94.60
12:05:01 AM     all      4.47      0.00      0.39      0.00      0.07     95.07
12:05:01 AM       0      4.28      0.00      0.40      0.00      0.09     95.24
12:05:01 AM       1      4.67      0.00      0.38      0.00      0.05     94.90
12:06:01 AM     all      0.51      0.00      0.08      0.00      0.08     99.33
12:06:01 AM       0      0.40      0.00      0.12      0.00      0.10     99.38
12:06:01 AM       1      0.62      0.00      0.05      0.00      0.05     99.28
12:07:01 AM     all      2.25      0.00      0.23      0.01      0.08     97.43
12:07:01 AM       0      0.52      0.00      0.15      0.00      0.08     99.25
12:07:01 AM       1      4.04      0.00      0.31      0.02      0.07     95.56
12:08:01 AM     all      4.88      0.00      0.24      0.00      0.07     94.81
12:08:01 AM       0      1.52      0.00      0.20      0.00      0.05     98.23
12:08:01 AM       1      8.26      0.00      0.29      0.00      0.08     91.37
12:09:01 AM     all      4.22      0.00      0.18      0.00      0.08     95.52
12:09:01 AM       0      1.35      0.00      0.15      0.00      0.05     98.45
12:09:01 AM       1      7.11      0.00      0.20      0.00      0.10     92.59
12:10:01 AM     all      4.27      0.00      0.20      0.00      0.07     95.47
12:10:01 AM       0      5.98      0.00      0.22      0.00      0.08     93.72
12:10:01 AM       1      2.56      0.00      0.18      0.00      0.05     97.21
12:11:01 AM     all      4.32      0.00      0.18      0.00      0.09     95.41
12:11:01 AM       0      1.02      0.00      0.15      0.00      0.05     98.78
12:11:01 AM       1      7.62      0.00      0.22      0.00      0.13     92.03

12:11:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
12:12:01 AM     all      4.19      0.00      0.19      0.00      0.07     95.55
12:12:01 AM       0      3.11      0.00      0.17      0.00      0.07     96.65
12:12:01 AM       1      5.26      0.00      0.22      0.00      0.07     94.46
12:13:01 AM     all      4.29      0.00      0.19      0.00      0.07     95.45
12:13:01 AM       0      2.59      0.00      0.17      0.00      0.07     97.17
12:13:01 AM       1      5.99      0.00      0.22      0.00      0.07     93.72
12:14:01 AM     all      5.74      0.00      0.47      0.00      0.05     93.74
12:14:01 AM       0      3.08      0.00      0.36      0.00      0.03     96.53
12:14:01 AM       1      8.48      0.00      0.58      0.00      0.07     90.88
12:15:01 AM     all      0.53      0.00      0.08      0.00      0.06     99.33
12:15:01 AM       0      0.58      0.00      0.05      0.00      0.03     99.33
12:15:01 AM       1      0.47      0.00      0.12      0.00      0.08     99.33
12:16:01 AM     all      1.09      0.00      0.11      0.00      0.06     98.75
12:16:01 AM       0      1.24      0.00      0.13      0.00      0.05     98.58
12:16:01 AM       1      0.94      0.00      0.08      0.00      0.07     98.91
12:17:01 AM     all      5.88      0.00      0.41      0.01      0.06     93.65
12:17:01 AM       0      1.90      0.00      0.20      0.02      0.07     97.81
12:17:01 AM       1     10.05      0.00      0.63      0.00      0.05     89.26
12:18:01 AM     all      4.12      0.00      0.29      0.00      0.07     95.52
12:18:01 AM       0      3.04      0.00      0.23      0.00      0.05     96.67
12:18:01 AM       1      5.21      0.00      0.35      0.00      0.08     94.36
12:19:01 AM     all      4.19      0.00      0.30      0.00      0.06     95.45
12:19:01 AM       0      3.35      0.00      0.23      0.00      0.05     96.37
12:19:01 AM       1      5.04      0.00      0.37      0.00      0.07     94.53
12:20:01 AM     all      4.29      0.00      0.30      0.00      0.08     95.33
12:20:01 AM       0      4.85      0.00      0.32      0.00      0.08     94.74
12:20:01 AM       1      3.73      0.00      0.28      0.00      0.07     95.92
12:21:01 AM     all      4.24      0.00      0.31      0.02      0.06     95.37
12:21:01 AM       0      3.68      0.00      0.30      0.03      0.05     95.93
12:21:01 AM       1      4.80      0.00      0.32      0.00      0.07     94.81
12:22:01 AM     all      4.10      0.00      0.28      0.00      0.05     95.57
12:22:01 AM       0      5.36      0.00      0.37      0.00      0.05     94.22
12:22:01 AM       1      2.84      0.00      0.20      0.00      0.05     96.91

12:22:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
12:23:01 AM     all      6.41      0.00      0.34      0.00      0.07     93.19
12:23:01 AM       0      5.79      0.00      0.35      0.00      0.07     93.79
12:23:01 AM       1      7.03      0.00      0.32      0.00      0.07     92.59
12:24:01 AM     all      0.92      0.00      0.13      0.00      0.06     98.90
12:24:01 AM       0      0.55      0.00      0.08      0.00      0.05     99.32
12:24:01 AM       1      1.29      0.00      0.17      0.00      0.07     98.48
12:25:01 AM     all      1.35      0.00      0.11      0.00      0.04     98.50
12:25:01 AM       0      0.99      0.00      0.10      0.00      0.05     98.86
12:25:01 AM       1      1.72      0.00      0.12      0.00      0.03     98.13
12:26:01 AM     all      1.35      0.00      0.13      0.00      0.03     98.48
12:26:01 AM       0      1.40      0.00      0.17      0.00      0.03     98.39
12:26:01 AM       1      1.30      0.00      0.10      0.00      0.03     98.57
12:27:01 AM     all      1.20      0.00      0.13      0.00      0.06     98.60
12:27:01 AM       0      1.49      0.00      0.13      0.00      0.07     98.31
12:27:01 AM       1      0.92      0.00      0.13      0.00      0.05     98.90
12:28:01 AM     all      1.35      0.00      0.13      0.00      0.04     98.47
12:28:01 AM       0      1.40      0.00      0.13      0.00      0.03     98.43
12:28:01 AM       1      1.30      0.00      0.13      0.00      0.05     98.51
12:29:02 AM     all      2.65      0.00      0.18      0.00      0.06     97.12
12:29:02 AM       0      1.85      0.00      0.12      0.00      0.05     97.98
12:29:02 AM       1      3.44      0.00      0.24      0.00      0.07     96.25
12:30:01 AM     all      6.63      0.00      0.52      0.00      0.05     92.80
12:30:01 AM       0      7.16      0.00      0.52      0.00      0.05     92.27
12:30:01 AM       1      6.11      0.00      0.52      0.00      0.05     93.33
12:31:01 AM     all      5.02      0.01      0.26      0.02      0.06     94.64
12:31:01 AM       0      1.69      0.02      0.18      0.03      0.05     98.03
12:31:01 AM       1      8.40      0.00      0.34      0.00      0.07     91.19
12:32:01 AM     all      4.30      0.00      0.20      0.00      0.05     95.45
12:32:01 AM       0      1.24      0.00      0.18      0.00      0.05     98.53
12:32:01 AM       1      7.36      0.00      0.22      0.00      0.05     92.37
12:33:01 AM     all      4.29      0.00      0.20      0.00      0.05     95.45
12:33:01 AM       0      1.91      0.00      0.17      0.00      0.03     97.89
12:33:01 AM       1      6.69      0.00      0.23      0.00      0.07     93.01

12:33:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
12:34:01 AM     all      4.30      0.00      0.17      0.00      0.05     95.48
12:34:01 AM       0      3.73      0.00      0.15      0.00      0.05     96.07
12:34:01 AM       1      4.87      0.00      0.18      0.00      0.05     94.89
12:35:01 AM     all      4.09      0.00      0.17      0.00      0.06     95.68
12:35:01 AM       0      2.07      0.00      0.12      0.00      0.05     97.76
12:35:01 AM       1      6.11      0.00      0.22      0.00      0.07     93.60
12:36:01 AM     all      4.21      0.00      0.18      0.00      0.07     95.54
12:36:01 AM       0      3.20      0.00      0.18      0.00      0.07     96.55
12:36:01 AM       1      5.23      0.00      0.18      0.00      0.07     94.52
12:37:01 AM     all      5.71      0.00      0.38      0.00      0.07     93.85
12:37:01 AM       0      2.04      0.00      0.20      0.00      0.07     97.69
12:37:01 AM       1      9.56      0.00      0.56      0.00      0.07     89.81
12:38:01 AM     all      5.87      0.00      0.34      0.00      0.06     93.73
12:38:01 AM       0      1.27      0.00      0.12      0.00      0.05     98.56
12:38:01 AM       1     10.67      0.00      0.57      0.00      0.07     88.69
12:39:01 AM     all      4.28      0.00      0.18      0.01      0.08     95.46
12:39:01 AM       0      3.48      0.00      0.17      0.00      0.08     96.27
12:39:01 AM       1      5.09      0.00      0.18      0.02      0.07     94.65
12:40:01 AM     all      4.23      0.24      0.32      0.31      0.06     94.84
12:40:01 AM       0      2.12      0.00      0.23      0.15      0.05     97.44
12:40:01 AM       1      6.35      0.49      0.40      0.47      0.07     92.23
12:41:01 AM     all      4.16      0.00      0.20      0.00      0.05     95.59
12:41:01 AM       0      5.04      0.00      0.22      0.00      0.05     94.69
12:41:01 AM       1      3.28      0.00      0.18      0.00      0.05     96.49
12:42:01 AM     all      4.24      0.00      0.20      0.00      0.05     95.51
12:42:01 AM       0      3.44      0.00      0.20      0.00      0.05     96.31
12:42:01 AM       1      5.04      0.00      0.20      0.00      0.05     94.71
12:43:01 AM     all      4.29      0.00      0.18      0.00      0.05     95.48
12:43:01 AM       0      2.54      0.00      0.12      0.00      0.03     97.31
12:43:01 AM       1      6.04      0.00      0.23      0.00      0.07     93.65
12:44:01 AM     all      5.57      0.00      0.30      0.00      0.06     94.08
12:44:01 AM       0      1.57      0.00      0.13      0.00      0.05     98.25
12:44:01 AM       1      9.71      0.00      0.47      0.00      0.07     89.75

12:44:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
12:45:01 AM     all      6.00      0.00      0.38      0.00      0.08     93.54
12:45:01 AM       0      2.22      0.00      0.20      0.00      0.07     97.51
12:45:01 AM       1      9.94      0.00      0.58      0.00      0.09     89.40
12:46:01 AM     all      4.19      0.00      0.33      0.00      0.08     95.41
12:46:01 AM       0      4.94      0.00      0.35      0.00      0.07     94.65
12:46:01 AM       1      3.44      0.00      0.30      0.00      0.08     96.17
12:47:01 AM     all      4.25      0.00      0.32      0.00      0.05     95.38
12:47:01 AM       0      2.32      0.00      0.25      0.00      0.05     97.38
12:47:01 AM       1      6.18      0.00      0.39      0.00      0.05     93.38
12:48:01 AM     all      4.37      0.00      0.30      0.00      0.07     95.26
12:48:01 AM       0      3.75      0.00      0.25      0.00      0.07     95.93
12:48:01 AM       1      4.99      0.00      0.35      0.00      0.07     94.60
12:49:01 AM     all      4.35      0.00      0.34      0.00      0.06     95.25
12:49:01 AM       0      2.22      0.00      0.20      0.00      0.05     97.53
12:49:01 AM       1      6.48      0.00      0.49      0.00      0.07     92.97
12:50:01 AM     all      4.20      0.00      0.31      0.00      0.06     95.43
12:50:01 AM       0      4.01      0.00      0.35      0.00      0.05     95.59
12:50:01 AM       1      4.39      0.00      0.27      0.00      0.07     95.28
12:51:01 AM     all      6.42      0.00      0.35      0.03      0.07     93.13
12:51:01 AM       0      3.66      0.00      0.25      0.00      0.05     96.03
12:51:01 AM       1      9.19      0.00      0.45      0.05      0.08     90.22
12:52:01 AM     all      1.10      0.00      0.13      0.00      0.06     98.71
12:52:01 AM       0      0.60      0.00      0.07      0.00      0.05     99.28
12:52:01 AM       1      1.60      0.00      0.18      0.00      0.07     98.15
12:53:01 AM     all      1.36      0.00      0.13      0.00      0.04     98.46
12:53:01 AM       0      0.67      0.00      0.10      0.00      0.03     99.20
12:53:01 AM       1      2.06      0.00      0.17      0.00      0.05     97.73
12:54:01 AM     all      1.11      0.00      0.14      0.00      0.05     98.70
12:54:01 AM       0      1.14      0.00      0.13      0.00      0.05     98.68
12:54:01 AM       1      1.09      0.00      0.15      0.00      0.05     98.71
12:55:01 AM     all      1.32      0.00      0.15      0.00      0.05     98.48
12:55:01 AM       0      0.87      0.00      0.13      0.00      0.05     98.95
12:55:01 AM       1      1.77      0.00      0.17      0.00      0.05     98.01

12:55:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
12:56:01 AM     all      1.38      0.00      0.13      0.00      0.06     98.44
12:56:01 AM       0      1.82      0.00      0.15      0.00      0.05     97.98
12:56:01 AM       1      0.93      0.00      0.10      0.00      0.07     98.90
12:57:01 AM     all      2.58      0.00      0.19      0.00      0.06     97.16
12:57:01 AM       0      2.71      0.00      0.23      0.00      0.07     96.99
12:57:01 AM       1      2.46      0.00      0.15      0.00      0.05     97.34
12:58:01 AM     all      6.57      0.00      0.52      0.00      0.05     92.85
12:58:01 AM       0      6.47      0.00      0.58      0.00      0.05     92.90
12:58:01 AM       1      6.67      0.00      0.47      0.00      0.05     92.81
12:59:01 AM     all      5.16      0.00      0.30      0.00      0.08     94.47
12:59:01 AM       0      2.43      0.00      0.22      0.00      0.07     97.29
12:59:01 AM       1      7.94      0.00      0.37      0.00      0.09     91.60
01:00:01 AM     all      4.28      0.00      0.18      0.00      0.04     95.50
01:00:01 AM       0      3.45      0.00      0.15      0.00      0.03     96.37
01:00:01 AM       1      5.10      0.00      0.22      0.00      0.05     94.63
01:01:01 AM     all      4.36      0.00      0.18      0.00      0.06     95.40
01:01:01 AM       0      6.26      0.00      0.22      0.00      0.07     93.45
01:01:01 AM       1      2.46      0.00      0.15      0.00      0.05     97.34
01:02:01 AM     all      4.18      0.00      0.20      0.00      0.06     95.56
01:02:01 AM       0      4.78      0.00      0.22      0.00      0.07     94.93
01:02:01 AM       1      3.58      0.00      0.18      0.00      0.05     96.18
01:03:01 AM     all      4.36      0.00      0.18      0.00      0.06     95.40
01:03:01 AM       0      4.67      0.00      0.20      0.00      0.05     95.08
01:03:01 AM       1      4.05      0.00      0.17      0.00      0.07     95.72
01:04:01 AM     all      4.36      0.00      0.20      0.00      0.05     95.39
01:04:01 AM       0      4.10      0.00      0.18      0.00      0.05     95.67
01:04:01 AM       1      4.62      0.00      0.22      0.00      0.05     95.11
01:05:01 AM     all      5.82      0.00      0.37      0.00      0.06     93.75
01:05:01 AM       0      0.40      0.00      0.15      0.00      0.05     99.40
01:05:01 AM       1     11.57      0.00      0.60      0.00      0.07     87.75
01:06:01 AM     all      6.58      0.00      0.38      0.00      0.06     92.98
01:06:01 AM       0      7.50      0.00      0.54      0.00      0.05     91.90
01:06:01 AM       1      5.70      0.00      0.22      0.00      0.07     94.02

01:06:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
01:07:01 AM     all      4.37      0.00      0.31      0.01      0.06     95.25
01:07:01 AM       0      1.90      0.00      0.20      0.02      0.05     97.83
01:07:01 AM       1      6.85      0.00      0.42      0.00      0.07     92.67
01:08:01 AM     all      4.35      0.00      0.28      0.00      0.07     95.31
01:08:01 AM       0      3.68      0.00      0.27      0.00      0.07     95.98
01:08:01 AM       1      5.02      0.00      0.28      0.00      0.07     94.63
01:09:01 AM     all      4.18      0.00      0.30      0.00      0.05     95.47
01:09:01 AM       0      1.97      0.00      0.20      0.00      0.05     97.78
01:09:01 AM       1      6.39      0.00      0.40      0.00      0.05     93.16
01:10:01 AM     all      4.26      0.00      0.32      0.00      0.07     95.35
01:10:01 AM       0      7.23      0.00      0.47      0.00      0.08     92.22
01:10:01 AM       1      1.30      0.00      0.17      0.00      0.05     98.48
01:11:01 AM     all      4.08      0.00      0.28      0.00      0.05     95.58
01:11:01 AM       0      3.28      0.00      0.27      0.00      0.05     96.40
01:11:01 AM       1      4.88      0.00      0.30      0.00      0.05     94.77
01:12:01 AM     all      7.05      0.00      0.34      0.00      0.07     92.54
01:12:01 AM       0      3.11      0.00      0.25      0.00      0.07     96.57
01:12:01 AM       1     11.01      0.00      0.44      0.00      0.07     88.48
01:13:01 AM     all      0.81      0.00      0.09      0.00      0.05     99.05
01:13:01 AM       0      0.92      0.00      0.12      0.00      0.07     98.90
01:13:01 AM       1      0.70      0.00      0.07      0.00      0.03     99.20
01:14:01 AM     all      1.30      0.00      0.11      0.00      0.06     98.53
01:14:01 AM       0      1.75      0.00      0.15      0.00      0.07     98.03
01:14:01 AM       1      0.85      0.00      0.07      0.00      0.05     99.03
01:15:01 AM     all      1.25      0.00      0.11      0.00      0.05     98.59
01:15:01 AM       0      0.82      0.00      0.08      0.00      0.03     99.06
01:15:01 AM       1      1.69      0.00      0.13      0.00      0.07     98.11
01:16:01 AM     all      1.42      0.00      0.12      0.00      0.07     98.40
01:16:01 AM       0      0.58      0.00      0.07      0.00      0.05     99.30
01:16:01 AM       1      2.25      0.00      0.17      0.00      0.08     97.49
01:17:01 AM     all      1.18      0.00      0.12      0.00      0.05     98.65
01:17:01 AM       0      1.04      0.00      0.10      0.00      0.05     98.81
01:17:01 AM       1      1.32      0.00      0.13      0.00      0.05     98.50

01:17:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
01:18:01 AM     all      1.10      0.00      0.13      0.00      0.05     98.73
01:18:01 AM       0      1.15      0.00      0.12      0.00      0.03     98.70
01:18:01 AM       1      1.04      0.00      0.13      0.00      0.07     98.76
01:19:01 AM     all      7.65      0.00      0.55      0.00      0.05     91.74
01:19:01 AM       0     10.48      0.00      0.80      0.00      0.05     88.66
01:19:01 AM       1      4.96      0.00      0.31      0.00      0.05     94.68
01:20:01 AM     all      4.28      0.00      0.40      0.00      0.06     95.26
01:20:01 AM       0      0.62      0.00      0.13      0.00      0.05     99.20
01:20:01 AM       1      7.95      0.00      0.67      0.00      0.07     91.32
01:21:01 AM     all      6.15      0.00      0.40      0.00      0.06     93.40
01:21:01 AM       0      9.02      0.00      0.46      0.00      0.07     90.45
01:21:01 AM       1      3.36      0.00      0.33      0.00      0.05     96.26
01:22:01 AM     all      4.18      0.00      0.18      0.00      0.06     95.58
01:22:01 AM       0      3.87      0.00      0.18      0.00      0.05     95.90
01:22:01 AM       1      4.50      0.00      0.17      0.00      0.07     95.26
01:23:01 AM     all      4.22      0.00      0.18      0.00      0.06     95.55
01:23:01 AM       0      1.52      0.00      0.15      0.00      0.05     98.28
01:23:01 AM       1      6.92      0.00      0.20      0.00      0.07     92.81
01:24:01 AM     all      4.22      0.00      0.18      0.00      0.06     95.55
01:24:01 AM       0      6.40      0.00      0.20      0.00      0.07     93.33
01:24:01 AM       1      2.04      0.00      0.15      0.00      0.05     97.76
01:25:01 AM     all      4.09      0.00      0.16      0.00      0.06     95.69
01:25:01 AM       0      5.84      0.00      0.20      0.00      0.07     93.89
01:25:01 AM       1      2.34      0.00      0.12      0.00      0.05     97.49
01:26:01 AM     all      4.30      0.00      0.19      0.00      0.06     95.45
01:26:01 AM       0      5.24      0.00      0.20      0.00      0.07     94.50
01:26:01 AM       1      3.36      0.00      0.18      0.00      0.05     96.41
01:27:01 AM     all      4.91      0.00      0.23      0.00      0.04     94.82
01:27:01 AM       0      7.23      0.00      0.31      0.00      0.05     92.41
01:27:01 AM       1      2.64      0.00      0.15      0.00      0.03     97.18
01:28:01 AM     all      5.35      0.00      0.42      0.03      0.09     94.11
01:28:01 AM       0      7.56      0.00      0.49      0.00      0.09     91.87
01:28:01 AM       1      3.24      0.00      0.35      0.07      0.08     96.26

01:28:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
01:29:01 AM     all      5.70      0.00      0.52      0.00      0.06     93.72
01:29:01 AM       0      6.65      0.00      0.61      0.00      0.05     92.69
01:29:01 AM       1      4.75      0.00      0.44      0.00      0.07     94.75
01:30:01 AM     all      5.04      0.00      0.24      0.00      0.05     94.67
01:30:01 AM       0      4.96      0.00      0.27      0.00      0.05     94.72
01:30:01 AM       1      5.12      0.00      0.22      0.00      0.05     94.61
01:31:01 AM     all      4.30      0.00      0.19      0.00      0.08     95.43
01:31:01 AM       0      1.02      0.00      0.12      0.00      0.07     98.80
01:31:01 AM       1      7.59      0.00      0.27      0.00      0.08     92.06
01:32:01 AM     all      4.28      0.00      0.18      0.00      0.06     95.47
01:32:01 AM       0      3.60      0.00      0.17      0.00      0.05     96.19
01:32:01 AM       1      4.97      0.00      0.20      0.00      0.07     94.76
01:33:01 AM     all      4.33      0.00      0.18      0.00      0.06     95.43
01:33:01 AM       0      1.91      0.00      0.17      0.00      0.05     97.88
01:33:01 AM       1      6.77      0.00      0.18      0.00      0.07     92.98
01:34:01 AM     all      4.27      0.00      0.18      0.00      0.07     95.49
01:34:01 AM       0      3.71      0.00      0.17      0.00      0.05     96.07
01:34:01 AM       1      4.82      0.00      0.18      0.00      0.08     94.91
01:35:02 AM     all      4.23      0.00      0.19      0.01      0.06     95.51
01:35:02 AM       0      4.47      0.00      0.18      0.02      0.07     95.27
01:35:02 AM       1      4.00      0.00      0.20      0.00      0.05     95.75
01:36:01 AM     all      5.35      0.00      0.43      0.00      0.05     94.17
01:36:01 AM       0      1.02      0.00      0.10      0.00      0.03     98.85
01:36:01 AM       1      9.83      0.00      0.77      0.00      0.07     89.33
01:37:01 AM     all      2.71      0.00      0.26      0.00      0.05     96.98
01:37:01 AM       0      0.62      0.00      0.10      0.00      0.05     99.23
01:37:01 AM       1      4.81      0.00      0.42      0.00      0.05     94.72
01:38:01 AM     all      5.58      0.00      0.28      0.00      0.06     94.08
01:38:01 AM       0      2.74      0.00      0.17      0.00      0.05     97.04
01:38:01 AM       1      8.52      0.00      0.40      0.00      0.07     91.02
01:39:01 AM     all      4.23      0.00      0.18      0.00      0.05     95.54
01:39:01 AM       0      4.37      0.00      0.18      0.00      0.05     95.40
01:39:01 AM       1      4.10      0.00      0.17      0.00      0.05     95.68

01:39:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
01:40:01 AM     all      4.19      0.00      0.17      0.00      0.05     95.59
01:40:01 AM       0      1.69      0.00      0.13      0.00      0.05     98.13
01:40:01 AM       1      6.70      0.00      0.20      0.00      0.05     93.05
01:41:01 AM     all      4.11      0.00      0.20      0.00      0.07     95.63
01:41:01 AM       0      5.12      0.00      0.23      0.00      0.07     94.58
01:41:01 AM       1      3.09      0.00      0.17      0.00      0.07     96.68
01:42:01 AM     all      4.21      0.00      0.17      0.00      0.06     95.57
01:42:01 AM       0      2.79      0.00      0.15      0.00      0.05     97.01
01:42:01 AM       1      5.62      0.00      0.18      0.00      0.07     94.12
01:43:01 AM     all      4.35      0.00      0.18      0.00      0.08     95.39
01:43:01 AM       0      2.86      0.00      0.18      0.00      0.07     96.89
01:43:01 AM       1      5.84      0.00      0.18      0.00      0.08     93.89
01:44:01 AM     all      5.79      0.00      0.38      0.00      0.06     93.77
01:44:01 AM       0      2.61      0.00      0.17      0.00      0.05     97.17
01:44:01 AM       1      9.07      0.00      0.60      0.00      0.07     90.26
01:45:01 AM     all      4.14      0.00      0.41      0.00      0.06     95.39
01:45:01 AM       0      4.64      0.00      0.47      0.00      0.07     94.82
01:45:01 AM       1      3.64      0.00      0.35      0.00      0.05     95.96
01:46:01 AM     all      6.92      0.00      0.47      0.00      0.06     92.55
01:46:01 AM       0      5.79      0.00      0.40      0.00      0.05     93.76
01:46:01 AM       1      8.04      0.00      0.54      0.00      0.07     91.35
01:47:01 AM     all      4.23      0.00      0.31      0.00      0.06     95.40
01:47:01 AM       0      3.10      0.00      0.22      0.00      0.07     96.62
01:47:01 AM       1      5.37      0.00      0.40      0.00      0.05     94.18
01:48:01 AM     all      4.23      0.00      0.28      0.00      0.06     95.43
01:48:01 AM       0      3.23      0.00      0.23      0.00      0.05     96.49
01:48:01 AM       1      5.24      0.00      0.32      0.00      0.07     94.38
01:49:01 AM     all      4.23      0.00      0.30      0.00      0.08     95.40
01:49:01 AM       0      3.55      0.00      0.25      0.00      0.07     96.14
01:49:01 AM       1      4.90      0.00      0.35      0.00      0.08     94.66
01:50:01 AM     all      4.20      0.00      0.28      0.00      0.05     95.47
01:50:01 AM       0      3.01      0.00      0.25      0.00      0.05     96.69
01:50:01 AM       1      5.39      0.00      0.32      0.00      0.05     94.24

01:50:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
01:51:01 AM     all      4.11      0.00      0.30      0.01      0.06     95.52
01:51:01 AM       0      5.89      0.00      0.42      0.00      0.07     93.62
01:51:01 AM       1      2.34      0.00      0.18      0.02      0.05     97.41
01:52:01 AM     all      6.98      0.00      0.39      0.00      0.06     92.57
01:52:01 AM       0      6.05      0.00      0.39      0.00      0.07     93.50
01:52:01 AM       1      7.90      0.00      0.40      0.00      0.05     91.65
01:53:01 AM     all      0.94      0.00      0.10      0.00      0.06     98.90
01:53:01 AM       0      1.32      0.00      0.17      0.00      0.07     98.45
01:53:01 AM       1      0.57      0.00      0.03      0.00      0.05     99.35
01:54:01 AM     all      1.07      0.00      0.12      0.00      0.05     98.76
01:54:01 AM       0      1.32      0.00      0.13      0.00      0.05     98.50
01:54:01 AM       1      0.82      0.00      0.10      0.00      0.05     99.03
01:55:01 AM     all      1.29      0.00      0.12      0.00      0.06     98.53
01:55:01 AM       0      1.54      0.00      0.12      0.00      0.07     98.28
01:55:01 AM       1      1.05      0.00      0.12      0.00      0.05     98.78
01:56:01 AM     all      1.30      0.00      0.11      0.00      0.06     98.53
01:56:01 AM       0      0.63      0.00      0.05      0.00      0.03     99.28
01:56:01 AM       1      1.97      0.00      0.17      0.00      0.08     97.78
01:57:01 AM     all      1.29      0.00      0.13      0.00      0.06     98.52
01:57:01 AM       0      1.52      0.00      0.15      0.00      0.07     98.26
01:57:01 AM       1      1.07      0.00      0.10      0.00      0.05     98.78
01:58:01 AM     all      2.30      0.00      0.14      0.00      0.06     97.50
01:58:01 AM       0      0.84      0.00      0.12      0.00      0.05     99.00
01:58:01 AM       1      3.77      0.00      0.17      0.00      0.07     96.00
01:59:01 AM     all      6.61      0.00      0.49      0.00      0.08     92.82
01:59:01 AM       0      0.70      0.00      0.17      0.00      0.08     99.05
01:59:01 AM       1     13.08      0.00      0.84      0.00      0.07     86.00
02:00:01 AM     all      5.27      0.00      0.31      0.00      0.06     94.36
02:00:01 AM       0      2.01      0.00      0.20      0.00      0.05     97.74
02:00:01 AM       1      8.60      0.00      0.43      0.00      0.07     90.91
02:01:01 AM     all      4.32      0.00      0.19      0.01      0.06     95.42
02:01:01 AM       0      1.55      0.00      0.15      0.02      0.05     98.23
02:01:01 AM       1      7.09      0.00      0.23      0.00      0.07     92.61

02:01:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
02:02:01 AM     all      4.38      0.00      0.22      0.00      0.06     95.34
02:02:01 AM       0      4.44      0.00      0.23      0.00      0.07     95.26
02:02:01 AM       1      4.33      0.00      0.20      0.00      0.05     95.42
02:03:01 AM     all      4.33      0.00      0.21      0.00      0.05     95.41
02:03:01 AM       0      4.50      0.00      0.20      0.00      0.05     95.25
02:03:01 AM       1      4.17      0.00      0.22      0.00      0.05     95.57
02:04:01 AM     all      4.12      0.17      0.25      0.00      0.06     95.40
02:04:01 AM       0      6.24      0.07      0.30      0.00      0.07     93.32
02:04:01 AM       1      2.01      0.27      0.20      0.00      0.05     97.48
02:05:01 AM     all      4.27      0.00      0.17      0.00      0.04     95.52
02:05:01 AM       0      4.40      0.00      0.15      0.00      0.03     95.41
02:05:01 AM       1      4.15      0.00      0.18      0.00      0.05     95.62
02:06:01 AM     all      5.64      0.00      0.37      0.00      0.06     93.94
02:06:01 AM       0     10.46      0.00      0.62      0.00      0.07     88.85
02:06:01 AM       1      1.07      0.00      0.13      0.00      0.05     98.75
02:07:01 AM     all      5.68      0.00      0.33      0.00      0.06     93.92
02:07:01 AM       0      9.13      0.00      0.50      0.00      0.05     90.31
02:07:01 AM       1      2.37      0.00      0.17      0.00      0.07     97.39
02:08:01 AM     all      4.28      0.00      0.17      0.01      0.07     95.47
02:08:01 AM       0      3.09      0.00      0.15      0.00      0.07     96.69
02:08:01 AM       1      5.47      0.00      0.18      0.02      0.07     94.26
02:09:01 AM     all      4.02      0.00      0.18      0.00      0.07     95.74
02:09:01 AM       0      5.24      0.00      0.18      0.00      0.08     94.49
02:09:01 AM       1      2.79      0.00      0.17      0.00      0.05     96.99
02:10:01 AM     all      4.90      0.00      0.56      0.00      0.07     94.47
02:10:01 AM       0      2.58      0.00      0.54      0.00      0.05     96.84
02:10:01 AM       1      7.23      0.00      0.59      0.00      0.08     92.10
02:11:01 AM     all      5.13      0.00      0.44      0.00      0.05     94.37
02:11:01 AM       0      5.95      0.00      0.50      0.00      0.05     93.50
02:11:01 AM       1      4.32      0.00      0.38      0.00      0.05     95.25
02:12:01 AM     all      4.34      0.00      0.19      0.00      0.06     95.41
02:12:01 AM       0      3.04      0.00      0.18      0.00      0.05     96.72
02:12:01 AM       1      5.64      0.00      0.20      0.00      0.07     94.10

02:12:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
02:13:01 AM     all      5.67      0.00      0.29      0.00      0.06     93.98
02:13:01 AM       0      9.02      0.00      0.41      0.00      0.07     90.50
02:13:01 AM       1      2.42      0.00      0.17      0.00      0.05     97.36
02:14:01 AM     all      6.10      0.00      0.39      0.00      0.05     93.46
02:14:01 AM       0     10.80      0.00      0.58      0.00      0.05     88.57
02:14:01 AM       1      1.62      0.00      0.22      0.00      0.05     98.11
02:15:01 AM     all      4.05      0.00      0.28      0.00      0.05     95.62
02:15:01 AM       0      5.32      0.00      0.33      0.00      0.07     94.27
02:15:01 AM       1      2.78      0.00      0.22      0.00      0.03     96.97
02:16:01 AM     all      4.16      0.00      0.28      0.02      0.04     95.50
02:16:01 AM       0      2.27      0.00      0.18      0.00      0.03     97.51
02:16:01 AM       1      6.05      0.00      0.39      0.03      0.05     93.48
02:17:01 AM     all      4.31      0.00      0.34      0.00      0.05     95.30
02:17:01 AM       0      3.58      0.00      0.30      0.00      0.05     96.07
02:17:01 AM       1      5.03      0.00      0.38      0.00      0.05     94.53
02:18:01 AM     all      4.13      0.00      0.28      0.00      0.04     95.55
02:18:01 AM       0      3.50      0.00      0.23      0.00      0.03     96.23
02:18:01 AM       1      4.75      0.00      0.33      0.00      0.05     94.86
02:19:01 AM     all      4.30      0.00      0.32      0.00      0.07     95.31
02:19:01 AM       0      5.38      0.00      0.37      0.00      0.07     94.19
02:19:01 AM       1      3.23      0.00      0.27      0.00      0.07     96.44
02:20:01 AM     all      6.40      0.00      0.39      0.00      0.06     93.16
02:20:01 AM       0      8.11      0.00      0.47      0.00      0.07     91.35
02:20:01 AM       1      4.69      0.00      0.30      0.00      0.05     94.96
02:21:01 AM     all      1.11      0.00      0.13      0.00      0.03     98.73
02:21:01 AM       0      1.32      0.00      0.17      0.00      0.03     98.48
02:21:01 AM       1      0.90      0.00      0.08      0.00      0.03     98.98
02:22:01 AM     all      1.40      0.00      0.13      0.00      0.04     98.42
02:22:01 AM       0      1.87      0.00      0.15      0.00      0.05     97.93
02:22:01 AM       1      0.94      0.00      0.12      0.00      0.03     98.91
02:23:01 AM     all      1.12      0.00      0.12      0.00      0.05     98.71
02:23:01 AM       0      1.71      0.00      0.15      0.00      0.05     98.09
02:23:01 AM       1      0.53      0.00      0.08      0.00      0.05     99.33

02:23:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
02:24:01 AM     all      1.35      0.00      0.11      0.00      0.05     98.50
02:24:01 AM       0      1.86      0.00      0.13      0.00      0.07     97.94
02:24:01 AM       1      0.84      0.00      0.08      0.00      0.03     99.05
02:25:01 AM     all      1.35      0.00      0.12      0.00      0.03     98.50
02:25:01 AM       0      2.07      0.00      0.13      0.00      0.03     97.76
02:25:01 AM       1      0.62      0.00      0.10      0.00      0.03     99.25
02:26:01 AM     all      2.33      0.00      0.17      0.00      0.06     97.44
02:26:01 AM       0      3.90      0.00      0.20      0.00      0.05     95.85
02:26:01 AM       1      0.77      0.00      0.13      0.00      0.07     99.03
02:27:01 AM     all      6.74      0.00      2.90      0.00      0.06     90.30
02:27:01 AM       0      8.47      0.00      5.49      0.00      0.07     85.98
02:27:01 AM       1      5.05      0.00      0.38      0.00      0.05     94.52
02:28:01 AM     all      6.47      0.00      0.56      0.00      0.06     92.91
02:28:01 AM       0      4.33      0.00      0.47      0.00      0.05     95.16
02:28:01 AM       1      8.68      0.00      0.66      0.00      0.07     90.59
02:29:01 AM     all      7.39      0.00      0.50      0.00      0.06     92.05
02:29:01 AM       0     10.86      0.00      0.64      0.00      0.07     88.42
02:29:01 AM       1      4.06      0.00      0.36      0.00      0.05     95.53
02:30:01 AM     all      4.85      0.00      0.38      0.00      0.06     94.72
02:30:01 AM       0      5.36      0.00      0.42      0.00      0.07     94.15
02:30:01 AM       1      4.32      0.00      0.34      0.00      0.05     95.28
02:31:01 AM     all      4.34      0.00      0.29      0.04      0.06     95.27
02:31:01 AM       0      6.33      0.00      0.37      0.00      0.05     93.25
02:31:01 AM       1      2.34      0.00      0.22      0.08      0.07     97.29
02:32:01 AM     all      4.15      0.00      0.30      0.00      0.05     95.50
02:32:01 AM       0      4.97      0.00      0.37      0.00      0.05     94.61
02:32:01 AM       1      3.33      0.00      0.23      0.00      0.05     96.39
02:33:01 AM     all      4.43      0.00      0.34      0.00      0.05     95.18
02:33:01 AM       0      3.01      0.00      0.27      0.00      0.05     96.67
02:33:01 AM       1      5.85      0.00      0.42      0.00      0.05     93.68
02:34:01 AM     all      4.23      0.00      0.33      0.00      0.06     95.39
02:34:01 AM       0      4.25      0.00      0.35      0.00      0.05     95.34
02:34:01 AM       1      4.20      0.00      0.30      0.00      0.07     95.43

02:34:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
02:35:01 AM     all      4.31      0.00      0.30      0.00      0.06     95.33
02:35:01 AM       0      6.42      0.00      0.40      0.00      0.07     93.11
02:35:01 AM       1      2.21      0.00      0.20      0.00      0.05     97.54
02:36:01 AM     all      5.66      0.00      0.28      0.00      0.05     94.01
02:36:01 AM       0      3.28      0.00      0.17      0.00      0.03     96.52
02:36:01 AM       1      8.05      0.00      0.40      0.00      0.07     91.48
02:37:01 AM     all      1.24      0.00      0.11      0.00      0.04     98.60
02:37:01 AM       0      1.87      0.00      0.13      0.00      0.05     97.94
02:37:01 AM       1      0.62      0.00      0.08      0.00      0.03     99.26
02:38:01 AM     all      1.30      0.00      0.13      0.00      0.03     98.54
02:38:01 AM       0      1.69      0.00      0.15      0.00      0.03     98.13
02:38:01 AM       1      0.92      0.00      0.10      0.00      0.03     98.95
02:39:01 AM     all      1.30      0.00      0.13      0.02      0.04     98.51
02:39:01 AM       0      1.82      0.00      0.15      0.00      0.03     97.99
02:39:01 AM       1      0.77      0.00      0.12      0.03      0.05     99.03
02:40:01 AM     all      1.07      0.00      0.12      0.00      0.04     98.77
02:40:01 AM       0      1.40      0.00      0.12      0.00      0.05     98.43
02:40:01 AM       1      0.74      0.00      0.12      0.00      0.03     99.11
02:41:01 AM     all      1.33      0.00      0.14      0.00      0.06     98.47
02:41:01 AM       0      1.89      0.00      0.15      0.00      0.07     97.89
02:41:01 AM       1      0.77      0.00      0.13      0.00      0.05     99.05
02:42:01 AM     all      4.42      0.00      2.61      0.01      0.05     92.92
02:42:01 AM       0      2.09      0.00      0.22      0.02      0.03     97.64
02:42:01 AM       1      6.79      0.00      5.05      0.00      0.07     88.09
02:43:01 AM     all      6.89      0.00      0.51      0.00      0.06     92.54
02:43:01 AM       0      0.45      0.00      0.15      0.00      0.05     99.35
02:43:01 AM       1     13.98      0.00      0.90      0.00      0.07     85.04
02:44:01 AM     all      6.42      0.00      0.57      0.00      0.07     92.94
02:44:01 AM       0      2.30      0.00      0.29      0.00      0.07     97.35
02:44:01 AM       1     10.88      0.00      0.88      0.00      0.07     88.18
02:45:02 AM     all      7.74      0.00      0.54      0.00      0.05     91.67
02:45:02 AM       0      7.90      0.00      0.54      0.00      0.05     91.51
02:45:02 AM       1      7.58      0.00      0.53      0.00      0.05     91.83

02:45:02 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
02:46:01 AM     all      4.23      0.00      0.32      0.00      0.07     95.38
02:46:01 AM       0      1.72      0.00      0.17      0.00      0.07     98.04
02:46:01 AM       1      6.75      0.00      0.48      0.00      0.07     92.71
02:47:01 AM     all      4.24      0.00      0.32      0.00      0.03     95.41
02:47:01 AM       0      3.89      0.00      0.33      0.00      0.03     95.75
02:47:01 AM       1      4.59      0.00      0.30      0.00      0.03     95.08
02:48:01 AM     all      4.32      0.00      0.28      0.00      0.05     95.35
02:48:01 AM       0      3.16      0.00      0.22      0.00      0.05     96.57
02:48:01 AM       1      5.47      0.00      0.35      0.00      0.05     94.12
02:49:01 AM     all      4.34      0.00      0.28      0.00      0.05     95.34
02:49:01 AM       0      3.65      0.00      0.25      0.00      0.03     96.07
02:49:01 AM       1      5.02      0.00      0.30      0.00      0.07     94.61
02:50:01 AM     all      4.22      0.00      0.33      0.00      0.05     95.40
02:50:01 AM       0      3.14      0.00      0.23      0.00      0.05     96.57
02:50:01 AM       1      5.30      0.00      0.42      0.00      0.05     94.23
02:51:01 AM     all      4.61      0.00      0.32      0.00      0.05     95.02
02:51:01 AM       0      3.75      0.00      0.30      0.00      0.05     95.90
02:51:01 AM       1      5.47      0.00      0.33      0.00      0.05     94.14
02:52:01 AM     all      3.94      0.00      0.23      0.00      0.05     95.78
02:52:01 AM       0      3.72      0.00      0.20      0.00      0.05     96.03
02:52:01 AM       1      4.15      0.00      0.27      0.00      0.05     95.53
02:53:01 AM     all      1.32      0.00      0.13      0.00      0.05     98.50
02:53:01 AM       0      1.82      0.00      0.18      0.00      0.07     97.93
02:53:01 AM       1      0.82      0.00      0.08      0.00      0.03     99.07
02:54:01 AM     all      1.28      0.00      0.12      0.00      0.04     98.56
02:54:01 AM       0      2.04      0.00      0.15      0.00      0.05     97.76
02:54:01 AM       1      0.52      0.00      0.08      0.00      0.03     99.36
02:55:01 AM     all      1.22      0.00      0.13      0.00      0.05     98.60
02:55:01 AM       0      2.02      0.00      0.17      0.00      0.05     97.76
02:55:01 AM       1      0.42      0.00      0.08      0.00      0.05     99.45
02:56:01 AM     all      1.34      0.00      0.13      0.00      0.05     98.48
02:56:01 AM       0      1.86      0.00      0.18      0.00      0.07     97.89
02:56:01 AM       1      0.82      0.00      0.08      0.00      0.03     99.07

02:56:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
02:57:01 AM     all      1.17      0.00      0.12      0.00      0.05     98.66
02:57:01 AM       0      1.74      0.00      0.15      0.00      0.05     98.06
02:57:01 AM       1      0.60      0.00      0.08      0.00      0.05     99.27
02:58:01 AM     all      5.38      0.00      2.26      0.00      0.05     92.31
02:58:01 AM       0      4.82      0.00      0.46      0.00      0.05     94.66
02:58:01 AM       1      5.94      0.00      4.08      0.00      0.05     89.93
02:59:01 AM     all      4.67      0.00      0.36      0.00      0.06     94.91
02:59:01 AM       0      3.08      0.00      0.27      0.00      0.05     96.59
02:59:01 AM       1      6.31      0.00      0.45      0.00      0.07     93.17
03:00:01 AM     all      2.60      0.00      0.29      0.00      0.04     97.07
03:00:01 AM       0      5.05      0.00      0.41      0.00      0.03     94.50
03:00:01 AM       1      0.23      0.00      0.17      0.00      0.05     99.55
03:01:01 AM     all      6.84      0.00      0.48      0.00      0.07     92.61
03:01:01 AM       0      9.52      0.00      0.61      0.00      0.09     89.78
03:01:01 AM       1      4.28      0.00      0.35      0.00      0.05     95.32
03:02:01 AM     all      4.35      0.00      0.33      0.00      0.06     95.27
03:02:01 AM       0      3.14      0.00      0.28      0.00      0.05     96.52
03:02:01 AM       1      5.56      0.00      0.37      0.00      0.07     94.01
03:03:01 AM     all      4.18      0.00      0.30      0.00      0.08     95.45
03:03:01 AM       0      2.91      0.00      0.22      0.00      0.08     96.79
03:03:01 AM       1      5.44      0.00      0.39      0.00      0.07     94.11
03:04:01 AM     all      4.38      0.00      0.35      0.00      0.06     95.21
03:04:01 AM       0      2.66      0.00      0.28      0.00      0.05     97.01
03:04:01 AM       1      6.11      0.00      0.42      0.00      0.07     93.40
03:05:01 AM     all      4.31      0.00      0.30      0.00      0.05     95.34
03:05:01 AM       0      4.52      0.00      0.30      0.00      0.05     95.13
03:05:01 AM       1      4.10      0.00      0.30      0.00      0.05     95.55
03:06:01 AM     all      4.30      0.00      0.30      0.00      0.04     95.36
03:06:01 AM       0      2.29      0.00      0.18      0.00      0.03     97.49
03:06:01 AM       1      6.30      0.00      0.42      0.00      0.05     93.23
03:07:01 AM     all      7.30      0.00      0.41      0.01      0.08     92.21
03:07:01 AM       0      3.75      0.00      0.23      0.00      0.05     95.97
03:07:01 AM       1     10.86      0.00      0.59      0.02      0.10     88.44

03:07:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
03:08:01 AM     all      0.71      0.00      0.11      0.00      0.03     99.15
03:08:01 AM       0      0.89      0.00      0.12      0.00      0.03     98.96
03:08:01 AM       1      0.53      0.00      0.10      0.00      0.03     99.33
03:09:01 AM     all      1.33      0.00      0.13      0.00      0.04     98.50
03:09:01 AM       0      2.06      0.00      0.18      0.00      0.05     97.71
03:09:01 AM       1      0.60      0.00      0.08      0.00      0.03     99.28
03:10:01 AM     all      1.33      0.00      0.10      0.00      0.04     98.53
03:10:01 AM       0      1.47      0.00      0.13      0.00      0.05     98.34
03:10:01 AM       1      1.19      0.00      0.07      0.00      0.03     98.71
03:11:01 AM     all      1.23      0.00      0.14      0.00      0.04     98.59
03:11:01 AM       0      0.37      0.00      0.12      0.00      0.03     99.48
03:11:01 AM       1      2.09      0.00      0.17      0.00      0.05     97.69
03:12:01 AM     all      1.29      0.00      0.12      0.00      0.06     98.54
03:12:01 AM       0      0.55      0.00      0.07      0.00      0.05     99.33
03:12:01 AM       1      2.02      0.00      0.17      0.00      0.07     97.74
03:13:01 AM     all      1.34      0.00      0.12      0.00      0.04     98.50
03:13:01 AM       0      0.58      0.00      0.05      0.00      0.03     99.33
03:13:01 AM       1      2.09      0.00      0.18      0.00      0.05     97.68
03:14:01 AM     all      6.73      0.00      3.09      0.00      0.07     90.10
03:14:01 AM       0      5.10      0.00      4.05      0.00      0.07     90.79
03:14:01 AM       1      8.38      0.00      2.13      0.00      0.07     89.42
03:15:01 AM     all      7.24      0.00      4.20      0.00      0.06     88.50
03:15:01 AM       0      7.51      0.00      4.80      0.00      0.07     87.62
03:15:01 AM       1      6.98      0.00      3.59      0.00      0.05     89.37
03:16:01 AM     all      7.67      0.00      5.23      0.00      0.06     87.04
03:16:01 AM       0      9.42      0.00      5.27      0.00      0.07     85.24
03:16:01 AM       1      5.97      0.00      5.19      0.00      0.05     88.79
03:17:01 AM     all      6.32      0.00      4.68      0.00      0.06     88.94
03:17:01 AM       0      7.11      0.00      8.69      0.00      0.05     84.15
03:17:01 AM       1      5.55      0.00      0.73      0.00      0.07     93.65
03:18:01 AM     all     17.60      0.00      3.05      0.00      0.08     79.27
03:18:01 AM       0      4.35      0.00      0.85      0.00      0.09     94.71
03:18:01 AM       1     30.92      0.00      5.26      0.00      0.07     63.75

03:18:01 AM     CPU     %user     %nice   %system   %iowait    %steal     %idle
03:19:01 AM     all     26.34      0.00      3.23      1.33      0.12     68.98
03:19:01 AM       0     21.55      0.00      3.26      1.59      0.12     73.49
03:19:01 AM       1     31.12      0.00      3.21      1.07      0.12     64.49
Average:        all      4.09      0.00      0.40      0.01      0.06     95.44
Average:          0      3.47      0.00      0.39      0.01      0.06     96.08
Average:          1      4.71      0.00      0.41      0.01      0.06     94.81