Started by upstream project "integration-distribution-mri-test-calcium" build number 43 originally caused by: Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on prd-centos8-robot-2c-8g-18399 (centos8-robot-2c-8g) in workspace /w/workspace/controller-csit-3node-clustering-ask-all-calcium [ssh-agent] Looking for ssh-agent implementation... [ssh-agent] Exec ssh-agent (binary ssh-agent on a remote machine) $ ssh-agent SSH_AUTH_SOCK=/tmp/ssh-B7uUusAIXVcV/agent.5292 SSH_AGENT_PID=5294 [ssh-agent] Started. Running ssh-add (command line suppressed) Identity added: /w/workspace/controller-csit-3node-clustering-ask-all-calcium@tmp/private_key_10622386785860279629.key (/w/workspace/controller-csit-3node-clustering-ask-all-calcium@tmp/private_key_10622386785860279629.key) [ssh-agent] Using credentials jenkins (Release Engineering Jenkins Key) The recommended git tool is: NONE using credential opendaylight-jenkins-ssh Wiping out workspace first. Cloning the remote Git repository Cloning repository git://devvexx.opendaylight.org/mirror/integration/test > git init /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test # timeout=10 Fetching upstream changes from git://devvexx.opendaylight.org/mirror/integration/test > git --version # timeout=10 > git --version # 'git version 2.43.0' using GIT_SSH to set credentials Release Engineering Jenkins Key [INFO] Currently running in a labeled security context [INFO] Currently SELinux is 'enforcing' on the host > /usr/bin/chcon --type=ssh_home_t /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test@tmp/jenkins-gitclient-ssh5203164362867978797.key Verifying host key using known hosts file You're using 'Known hosts file' strategy to verify ssh host keys, but your known_hosts file does not exist, please go to 'Manage Jenkins' -> 'Security' -> 'Git Host Key Verification Configuration' and configure host key verification. > git fetch --tags --force --progress -- git://devvexx.opendaylight.org/mirror/integration/test +refs/heads/*:refs/remotes/origin/* # timeout=10 > git config remote.origin.url git://devvexx.opendaylight.org/mirror/integration/test # timeout=10 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 > git config remote.origin.url git://devvexx.opendaylight.org/mirror/integration/test # timeout=10 Fetching upstream changes from git://devvexx.opendaylight.org/mirror/integration/test using GIT_SSH to set credentials Release Engineering Jenkins Key [INFO] Currently running in a labeled security context [INFO] Currently SELinux is 'enforcing' on the host > /usr/bin/chcon --type=ssh_home_t /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test@tmp/jenkins-gitclient-ssh15971611318123007493.key Verifying host key using known hosts file You're using 'Known hosts file' strategy to verify ssh host keys, but your known_hosts file does not exist, please go to 'Manage Jenkins' -> 'Security' -> 'Git Host Key Verification Configuration' and configure host key verification. > git fetch --tags --force --progress -- git://devvexx.opendaylight.org/mirror/integration/test master # timeout=10 > git rev-parse FETCH_HEAD^{commit} # timeout=10 Checking out Revision 4a6b3981d63b232863e5f26b33a9fb1bd2c97104 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 4a6b3981d63b232863e5f26b33a9fb1bd2c97104 # timeout=10 Commit message: "Fix robot file format issues" > git rev-parse FETCH_HEAD^{commit} # timeout=10 > git rev-list --no-walk 17cd10aba541b0653c6500c24f1258232e3e0478 # timeout=10 No emails were triggered. provisioning config files... copy managed file [npmrc] to file:/home/jenkins/.npmrc copy managed file [pipconf] to file:/home/jenkins/.config/pip/pip.conf copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml [controller-csit-3node-clustering-ask-all-calcium] $ /bin/bash /tmp/jenkins5938476067594940184.sh ---> python-tools-install.sh Setup pyenv: system * 3.8.13 (set by /opt/pyenv/version) * 3.9.13 (set by /opt/pyenv/version) * 3.10.13 (set by /opt/pyenv/version) * 3.11.7 (set by /opt/pyenv/version) lf-activate-venv(): INFO: Creating python3 venv at /tmp/venv-1v9O lf-activate-venv(): INFO: Save venv in file: /tmp/.os_lf_venv lf-activate-venv(): INFO: Installing: lftools lf-activate-venv(): INFO: Adding /tmp/venv-1v9O/bin to PATH Generating Requirements File Python 3.11.7 pip 24.1.1 from /tmp/venv-1v9O/lib/python3.11/site-packages/pip (python 3.11) appdirs==1.4.4 argcomplete==3.4.0 aspy.yaml==1.3.0 attrs==23.2.0 autopage==0.5.2 beautifulsoup4==4.12.3 boto3==1.34.140 botocore==1.34.140 bs4==0.0.2 cachetools==5.3.3 certifi==2024.7.4 cffi==1.16.0 cfgv==3.4.0 chardet==5.2.0 charset-normalizer==3.3.2 click==8.1.7 cliff==4.7.0 cmd2==2.4.3 cryptography==3.3.2 debtcollector==3.0.0 decorator==5.1.1 defusedxml==0.7.1 Deprecated==1.2.14 distlib==0.3.8 dnspython==2.6.1 docker==4.2.2 dogpile.cache==1.3.3 email_validator==2.2.0 filelock==3.15.4 future==1.0.0 gitdb==4.0.11 GitPython==3.1.43 google-auth==2.31.0 httplib2==0.22.0 identify==2.5.36 idna==3.7 importlib-resources==1.5.0 iso8601==2.1.0 Jinja2==3.1.4 jmespath==1.0.1 jsonpatch==1.33 jsonpointer==3.0.0 jsonschema==4.22.0 jsonschema-specifications==2023.12.1 keystoneauth1==5.7.0 kubernetes==30.1.0 lftools==0.37.10 lxml==5.2.2 MarkupSafe==2.1.5 msgpack==1.0.8 multi_key_dict==2.0.3 munch==4.0.0 netaddr==1.3.0 netifaces==0.11.0 niet==1.4.2 nodeenv==1.9.1 oauth2client==4.1.3 oauthlib==3.2.2 openstacksdk==3.2.0 os-client-config==2.1.0 os-service-types==1.7.0 osc-lib==3.1.0 oslo.config==9.5.0 oslo.context==5.5.0 oslo.i18n==6.3.0 oslo.log==6.1.0 oslo.serialization==5.4.0 oslo.utils==7.2.0 packaging==24.1 pbr==6.0.0 platformdirs==4.2.2 prettytable==3.10.0 pyasn1==0.6.0 pyasn1_modules==0.4.0 pycparser==2.22 pygerrit2==2.0.15 PyGithub==2.3.0 PyJWT==2.8.0 PyNaCl==1.5.0 pyparsing==2.4.7 pyperclip==1.9.0 pyrsistent==0.20.0 python-cinderclient==9.5.0 python-dateutil==2.9.0.post0 python-heatclient==3.5.0 python-jenkins==1.8.2 python-keystoneclient==5.4.0 python-magnumclient==4.6.0 python-novaclient==18.6.0 python-openstackclient==6.6.0 python-swiftclient==4.6.0 PyYAML==6.0.1 referencing==0.35.1 requests==2.32.3 requests-oauthlib==2.0.0 requestsexceptions==1.4.0 rfc3986==2.0.0 rpds-py==0.18.1 rsa==4.9 ruamel.yaml==0.18.6 ruamel.yaml.clib==0.2.8 s3transfer==0.10.2 simplejson==3.19.2 six==1.16.0 smmap==5.0.1 soupsieve==2.5 stevedore==5.2.0 tabulate==0.9.0 toml==0.10.2 tomlkit==0.12.5 tqdm==4.66.4 typing_extensions==4.12.2 tzdata==2024.1 urllib3==1.26.19 virtualenv==20.26.3 wcwidth==0.2.13 websocket-client==1.8.0 wrapt==1.16.0 xdg==6.0.0 xmltodict==0.13.0 yq==3.4.3 [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties content OS_STACK_TEMPLATE=csit-2-instance-type.yaml OS_CLOUD=vex OS_STACK_NAME=releng-controller-csit-3node-clustering-ask-all-calcium-43 OS_STACK_TEMPLATE_DIR=openstack-hot [EnvInject] - Variables injected successfully. provisioning config files... copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml [controller-csit-3node-clustering-ask-all-calcium] $ /bin/bash /tmp/jenkins16414021068381368327.sh ---> Create parameters file for OpenStack HOT OpenStack Heat parameters generated ----------------------------------- parameters: vm_0_count: '3' vm_0_flavor: 'v3-standard-4' vm_0_image: 'ZZCI - CentOS Stream 8 - builder - x86_64 - 20240601-160217.263' vm_1_count: '0' vm_1_flavor: 'v3-standard-2' vm_1_image: 'ZZCI - Ubuntu 18.04 - mininet-ovs-28 - x86_64 - 20230601-180106.003' job_name: '37733-43' silo: 'releng' [controller-csit-3node-clustering-ask-all-calcium] $ /bin/bash -l /tmp/jenkins1861938157943717601.sh ---> Create HEAT stack + source /home/jenkins/lf-env.sh + lf-activate-venv --python python3 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq ++ mktemp -d /tmp/venv-XXXX + lf_venv=/tmp/venv-vpHR + local venv_file=/tmp/.os_lf_venv + local python=python3 + local options + local set_path=true + local install_args= ++ getopt -o np:v: -l no-path,system-site-packages,python:,venv-file: -n lf-activate-venv -- --python python3 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq + options=' --python '\''python3'\'' -- '\''lftools[openstack]'\'' '\''kubernetes'\'' '\''niet'\'' '\''python-heatclient'\'' '\''python-openstackclient'\'' '\''python-magnumclient'\'' '\''yq'\''' + eval set -- ' --python '\''python3'\'' -- '\''lftools[openstack]'\'' '\''kubernetes'\'' '\''niet'\'' '\''python-heatclient'\'' '\''python-openstackclient'\'' '\''python-magnumclient'\'' '\''yq'\''' ++ set -- --python python3 -- 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq + true + case $1 in + python=python3 + shift 2 + true + case $1 in + shift + break + case $python in + local pkg_list= + [[ -d /opt/pyenv ]] + echo 'Setup pyenv:' Setup pyenv: + export PYENV_ROOT=/opt/pyenv + PYENV_ROOT=/opt/pyenv + export PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin + PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin + pyenv versions system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-calcium/.python-version) + command -v pyenv ++ pyenv init - --no-rehash + eval 'PATH="$(bash --norc -ec '\''IFS=:; paths=($PATH); for i in ${!paths[@]}; do if [[ ${paths[i]} == "'\'''\''/opt/pyenv/shims'\'''\''" ]]; then unset '\''\'\'''\''paths[i]'\''\'\'''\''; fi; done; echo "${paths[*]}"'\'')" export PATH="/opt/pyenv/shims:${PATH}" export PYENV_SHELL=bash source '\''/opt/pyenv/libexec/../completions/pyenv.bash'\'' pyenv() { local command command="${1:-}" if [ "$#" -gt 0 ]; then shift fi case "$command" in rehash|shell) eval "$(pyenv "sh-$command" "$@")" ;; *) command pyenv "$command" "$@" ;; esac }' +++ bash --norc -ec 'IFS=:; paths=($PATH); for i in ${!paths[@]}; do if [[ ${paths[i]} == "/opt/pyenv/shims" ]]; then unset '\''paths[i]'\''; fi; done; echo "${paths[*]}"' ++ PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin ++ export PATH=/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin ++ PATH=/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin ++ export PYENV_SHELL=bash ++ PYENV_SHELL=bash ++ source /opt/pyenv/libexec/../completions/pyenv.bash +++ complete -F _pyenv pyenv ++ lf-pyver python3 ++ local py_version_xy=python3 ++ local py_version_xyz= ++ awk '{ print $1 }' ++ pyenv versions ++ local command ++ command=versions ++ '[' 1 -gt 0 ']' ++ shift ++ case "$command" in ++ command pyenv versions ++ pyenv versions ++ sed 's/^[ *]* //' ++ grep -E '^[0-9.]*[0-9]$' ++ [[ ! -s /tmp/.pyenv_versions ]] +++ tail -n 1 +++ grep '^3' /tmp/.pyenv_versions +++ sort -V ++ py_version_xyz=3.11.7 ++ [[ -z 3.11.7 ]] ++ echo 3.11.7 ++ return 0 + pyenv local 3.11.7 + local command + command=local + '[' 2 -gt 0 ']' + shift + case "$command" in + command pyenv local 3.11.7 + pyenv local 3.11.7 + for arg in "$@" + case $arg in + pkg_list+='lftools[openstack] ' + for arg in "$@" + case $arg in + pkg_list+='kubernetes ' + for arg in "$@" + case $arg in + pkg_list+='niet ' + for arg in "$@" + case $arg in + pkg_list+='python-heatclient ' + for arg in "$@" + case $arg in + pkg_list+='python-openstackclient ' + for arg in "$@" + case $arg in + pkg_list+='python-magnumclient ' + for arg in "$@" + case $arg in + pkg_list+='yq ' + [[ -f /tmp/.os_lf_venv ]] ++ cat /tmp/.os_lf_venv + lf_venv=/tmp/venv-1v9O + echo 'lf-activate-venv(): INFO: Reuse venv:/tmp/venv-1v9O from' file:/tmp/.os_lf_venv lf-activate-venv(): INFO: Reuse venv:/tmp/venv-1v9O from file:/tmp/.os_lf_venv + /tmp/venv-1v9O/bin/python3 -m pip install --upgrade --quiet pip virtualenv + [[ -z lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq ]] + echo 'lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq ' lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient yq + /tmp/venv-1v9O/bin/python3 -m pip install --upgrade --quiet --upgrade-strategy eager 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient yq + type python3 + true + echo 'lf-activate-venv(): INFO: Adding /tmp/venv-1v9O/bin to PATH' lf-activate-venv(): INFO: Adding /tmp/venv-1v9O/bin to PATH + PATH=/tmp/venv-1v9O/bin:/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin + return 0 + openstack --os-cloud vex limits show --absolute +--------------------------+---------+ | Name | Value | +--------------------------+---------+ | maxTotalInstances | -1 | | maxTotalCores | 450 | | maxTotalRAMSize | 1000000 | | maxServerMeta | 128 | | maxImageMeta | 128 | | maxPersonality | 5 | | maxPersonalitySize | 10240 | | maxTotalKeypairs | 100 | | maxServerGroups | 10 | | maxServerGroupMembers | 10 | | maxTotalFloatingIps | -1 | | maxSecurityGroups | -1 | | maxSecurityGroupRules | -1 | | totalRAMUsed | 385024 | | totalCoresUsed | 94 | | totalInstancesUsed | 36 | | totalFloatingIpsUsed | 0 | | totalSecurityGroupsUsed | 0 | | totalServerGroupsUsed | 0 | | maxTotalVolumes | -1 | | maxTotalSnapshots | 10 | | maxTotalVolumeGigabytes | 4096 | | maxTotalBackups | 10 | | maxTotalBackupGigabytes | 1000 | | totalVolumesUsed | 1 | | totalGigabytesUsed | 20 | | totalSnapshotsUsed | 0 | | totalBackupsUsed | 0 | | totalBackupGigabytesUsed | 0 | +--------------------------+---------+ + pushd /opt/ciman/openstack-hot /opt/ciman/openstack-hot /w/workspace/controller-csit-3node-clustering-ask-all-calcium + lftools openstack --os-cloud vex stack create releng-controller-csit-3node-clustering-ask-all-calcium-43 csit-2-instance-type.yaml /w/workspace/controller-csit-3node-clustering-ask-all-calcium/stack-parameters.yaml Creating stack releng-controller-csit-3node-clustering-ask-all-calcium-43 Waiting to initialize infrastructure... Stack initialization successful. ------------------------------------ Stack Details ------------------------------------ {'added': None, 'capabilities': [], 'created_at': '2024-07-07T00:24:56Z', 'deleted': None, 'deleted_at': None, 'description': 'No description', 'environment': None, 'environment_files': None, 'files': None, 'files_container': None, 'id': '92ff0196-f229-4196-ad3f-255d1f30a4a0', 'is_rollback_disabled': True, 'links': [{'href': 'https://orchestration.public.mtl1.vexxhost.net/v1/12c36e260d8e4bb2913965203b1b491f/stacks/releng-controller-csit-3node-clustering-ask-all-calcium-43/92ff0196-f229-4196-ad3f-255d1f30a4a0', 'rel': 'self'}], 'location': Munch({'cloud': 'vex', 'region_name': 'ca-ymq-1', 'zone': None, 'project': Munch({'id': '12c36e260d8e4bb2913965203b1b491f', 'name': '61975f2c-7c17-4d69-82fa-c3ae420ad6fd', 'domain_id': None, 'domain_name': 'Default'})}), 'name': 'releng-controller-csit-3node-clustering-ask-all-calcium-43', 'notification_topics': [], 'outputs': [{'description': 'IP addresses of the 1st vm types', 'output_key': 'vm_0_ips', 'output_value': ['10.30.171.165', '10.30.171.170', '10.30.170.8']}, {'description': 'IP addresses of the 2nd vm types', 'output_key': 'vm_1_ips', 'output_value': []}], 'owner_id': ****, 'parameters': {'OS::project_id': '12c36e260d8e4bb2913965203b1b491f', 'OS::stack_id': '92ff0196-f229-4196-ad3f-255d1f30a4a0', 'OS::stack_name': 'releng-controller-csit-3node-clustering-ask-all-calcium-43', 'job_name': '37733-43', 'silo': 'releng', 'vm_0_count': '3', 'vm_0_flavor': 'v3-standard-4', 'vm_0_image': 'ZZCI - CentOS Stream 8 - builder - x86_64 - ' '20240601-160217.263', 'vm_1_count': '0', 'vm_1_flavor': 'v3-standard-2', 'vm_1_image': 'ZZCI - Ubuntu 18.04 - mininet-ovs-28 - x86_64 - ' '20230601-180106.003'}, 'parent_id': None, 'replaced': None, 'status': 'CREATE_COMPLETE', 'status_reason': 'Stack CREATE completed successfully', 'tags': [], 'template': None, 'template_description': 'No description', 'template_url': None, 'timeout_mins': 15, 'unchanged': None, 'updated': None, 'updated_at': None, 'user_project_id': 'cffe810bb6ed4a01afe18b3cde837482'} ------------------------------------ + popd /w/workspace/controller-csit-3node-clustering-ask-all-calcium [controller-csit-3node-clustering-ask-all-calcium] $ /bin/bash -l /tmp/jenkins12361202052037343199.sh ---> Copy SSH public keys to CSIT lab Setup pyenv: system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-calcium/.python-version) lf-activate-venv(): INFO: Reuse venv:/tmp/venv-1v9O from file:/tmp/.os_lf_venv lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes python-heatclient python-openstackclient lf-activate-venv(): INFO: Adding /tmp/venv-1v9O/bin to PATH SSH not responding on 10.30.170.8. Retrying in 10 seconds... SSH not responding on 10.30.171.165. Retrying in 10 seconds... SSH not responding on 10.30.171.170. Retrying in 10 seconds... Ping to 10.30.171.165 successful. Ping to 10.30.170.8 successful. Ping to 10.30.171.170 successful. SSH not responding on 10.30.171.165. Retrying in 10 seconds... SSH not responding on 10.30.171.170. Retrying in 10 seconds... SSH not responding on 10.30.170.8. Retrying in 10 seconds... Ping to 10.30.171.165 successful. Ping to 10.30.171.170 successful. Ping to 10.30.170.8 successful. SSH not responding on 10.30.171.165. Retrying in 10 seconds... Warning: Permanently added '10.30.170.8' (ECDSA) to the list of known hosts. Warning: Permanently added '10.30.171.170' (ECDSA) to the list of known hosts. 10.30.170.8 releng-37733-43-0-builder-2.novalocal Successfully copied public keys to slave 10.30.170.8 10.30.171.170 releng-37733-43-0-builder-1.novalocal Successfully copied public keys to slave 10.30.171.170 Ping to 10.30.171.165 successful. SSH not responding on 10.30.171.165. Retrying in 10 seconds... Ping to 10.30.171.165 successful. Warning: Permanently added '10.30.171.165' (ECDSA) to the list of known hosts. 10.30.171.165 releng-37733-43-0-builder-0.novalocal Successfully copied public keys to slave 10.30.171.165 Process 6686 ready. Process 6687 ready. Process 6688 ready. SSH ready on all stack servers. [controller-csit-3node-clustering-ask-all-calcium] $ /bin/bash -l /tmp/jenkins14357490696898898476.sh Setup pyenv: system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-calcium/.python-version) lf-activate-venv(): INFO: Creating python3 venv at /tmp/venv-AAZr lf-activate-venv(): INFO: Save venv in file: /w/workspace/controller-csit-3node-clustering-ask-all-calcium/.robot_venv lf-activate-venv(): INFO: Installing: setuptools wheel lf-activate-venv(): INFO: Adding /tmp/venv-AAZr/bin to PATH + echo 'Installing Python Requirements' Installing Python Requirements + cat + python -m pip install -r requirements.txt Looking in indexes: https://nexus3.opendaylight.org/repository/PyPi/simple Collecting docker-py (from -r requirements.txt (line 1)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/docker-py/1.10.6/docker_py-1.10.6-py2.py3-none-any.whl (50 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 50.0/50.0 kB 8.3 MB/s eta 0:00:00 Collecting ipaddr (from -r requirements.txt (line 2)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/ipaddr/2.2.0/ipaddr-2.2.0.tar.gz (26 kB) Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'done' Collecting netaddr (from -r requirements.txt (line 3)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/netaddr/1.3.0/netaddr-1.3.0-py3-none-any.whl (2.3 MB) Collecting netifaces (from -r requirements.txt (line 4)) Using cached netifaces-0.11.0-cp311-cp311-linux_x86_64.whl Collecting pyhocon (from -r requirements.txt (line 5)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/pyhocon/0.3.61/pyhocon-0.3.61-py3-none-any.whl (25 kB) Collecting requests (from -r requirements.txt (line 6)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/requests/2.32.3/requests-2.32.3-py3-none-any.whl (64 kB) Collecting robotframework (from -r requirements.txt (line 7)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework/7.0.1/robotframework-7.0.1-py3-none-any.whl (727 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 727.6/727.6 kB 13.9 MB/s eta 0:00:00 Collecting robotframework-httplibrary (from -r requirements.txt (line 8)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-httplibrary/0.4.2/robotframework-httplibrary-0.4.2.tar.gz (9.1 kB) Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'done' Collecting robotframework-requests==0.9.7 (from -r requirements.txt (line 9)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-requests/0.9.7/robotframework_requests-0.9.7-py3-none-any.whl (21 kB) Collecting robotframework-selenium2library (from -r requirements.txt (line 10)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-selenium2library/3.0.0/robotframework_selenium2library-3.0.0-py2.py3-none-any.whl (6.2 kB) Collecting robotframework-sshlibrary==3.8.0 (from -r requirements.txt (line 11)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-sshlibrary/3.8.0/robotframework-sshlibrary-3.8.0.tar.gz (51 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 51.4/51.4 kB 7.6 MB/s eta 0:00:00 Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'done' Collecting scapy (from -r requirements.txt (line 12)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/scapy/2.5.0/scapy-2.5.0.tar.gz (1.3 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.3/1.3 MB 14.8 MB/s eta 0:00:00 Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'done' Collecting jsonpath-rw (from -r requirements.txt (line 15)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/jsonpath-rw/1.4.0/jsonpath-rw-1.4.0.tar.gz (13 kB) Preparing metadata (setup.py): started Preparing metadata (setup.py): finished with status 'done' Collecting elasticsearch (from -r requirements.txt (line 18)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elasticsearch/8.14.0/elasticsearch-8.14.0-py3-none-any.whl (480 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 480.2/480.2 kB 15.4 MB/s eta 0:00:00 Collecting elasticsearch-dsl (from -r requirements.txt (line 19)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elasticsearch-dsl/8.14.0/elasticsearch_dsl-8.14.0-py3-none-any.whl (92 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 92.3/92.3 kB 12.1 MB/s eta 0:00:00 Collecting pyangbind (from -r requirements.txt (line 22)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/pyangbind/0.8.5/pyangbind-0.8.5-py3-none-any.whl (52 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 52.9/52.9 kB 5.8 MB/s eta 0:00:00 Collecting isodate (from -r requirements.txt (line 25)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/isodate/0.6.1/isodate-0.6.1-py2.py3-none-any.whl (41 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.7/41.7 kB 5.6 MB/s eta 0:00:00 Collecting jmespath (from -r requirements.txt (line 28)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/jmespath/1.0.1/jmespath-1.0.1-py3-none-any.whl (20 kB) Collecting jsonpatch (from -r requirements.txt (line 31)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/jsonpatch/1.33/jsonpatch-1.33-py2.py3-none-any.whl (12 kB) Collecting paramiko>=1.15.3 (from robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/paramiko/3.4.0/paramiko-3.4.0-py3-none-any.whl (225 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 225.9/225.9 kB 21.2 MB/s eta 0:00:00 Collecting scp>=0.13.0 (from robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/scp/0.15.0/scp-0.15.0-py2.py3-none-any.whl (8.8 kB) Collecting docker-pycreds>=0.2.1 (from docker-py->-r requirements.txt (line 1)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/docker-pycreds/0.4.0/docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB) Collecting six>=1.4.0 (from docker-py->-r requirements.txt (line 1)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/six/1.16.0/six-1.16.0-py2.py3-none-any.whl (11 kB) Collecting websocket-client>=0.32.0 (from docker-py->-r requirements.txt (line 1)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/websocket-client/1.8.0/websocket_client-1.8.0-py3-none-any.whl (58 kB) Collecting pyparsing<4,>=2 (from pyhocon->-r requirements.txt (line 5)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/pyparsing/3.1.2/pyparsing-3.1.2-py3-none-any.whl (103 kB) Collecting charset-normalizer<4,>=2 (from requests->-r requirements.txt (line 6)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/charset-normalizer/3.3.2/charset_normalizer-3.3.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (140 kB) Collecting idna<4,>=2.5 (from requests->-r requirements.txt (line 6)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/idna/3.7/idna-3.7-py3-none-any.whl (66 kB) Collecting urllib3<3,>=1.21.1 (from requests->-r requirements.txt (line 6)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/urllib3/2.2.2/urllib3-2.2.2-py3-none-any.whl (121 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 121.4/121.4 kB 19.6 MB/s eta 0:00:00 Collecting certifi>=2017.4.17 (from requests->-r requirements.txt (line 6)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/certifi/2024.7.4/certifi-2024.7.4-py3-none-any.whl (162 kB) Collecting webtest>=2.0 (from robotframework-httplibrary->-r requirements.txt (line 8)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/webtest/3.0.0/WebTest-3.0.0-py3-none-any.whl (31 kB) Collecting jsonpointer (from robotframework-httplibrary->-r requirements.txt (line 8)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/jsonpointer/3.0.0/jsonpointer-3.0.0-py2.py3-none-any.whl (7.6 kB) Collecting robotframework-seleniumlibrary>=3.0.0 (from robotframework-selenium2library->-r requirements.txt (line 10)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-seleniumlibrary/6.5.0/robotframework_seleniumlibrary-6.5.0-py2.py3-none-any.whl (104 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 104.6/104.6 kB 2.5 MB/s eta 0:00:00 Collecting ply (from jsonpath-rw->-r requirements.txt (line 15)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/ply/3.11/ply-3.11-py2.py3-none-any.whl (49 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 49.6/49.6 kB 4.0 MB/s eta 0:00:00 Collecting decorator (from jsonpath-rw->-r requirements.txt (line 15)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/decorator/5.1.1/decorator-5.1.1-py3-none-any.whl (9.1 kB) Collecting elastic-transport<9,>=8.13 (from elasticsearch->-r requirements.txt (line 18)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elastic-transport/8.13.1/elastic_transport-8.13.1-py3-none-any.whl (64 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 64.5/64.5 kB 11.2 MB/s eta 0:00:00 Collecting python-dateutil (from elasticsearch-dsl->-r requirements.txt (line 19)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/python-dateutil/2.9.0.post0/python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) Collecting typing-extensions (from elasticsearch-dsl->-r requirements.txt (line 19)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/typing-extensions/4.12.2/typing_extensions-4.12.2-py3-none-any.whl (37 kB) Collecting pyang (from pyangbind->-r requirements.txt (line 22)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/pyang/2.6.1/pyang-2.6.1-py2.py3-none-any.whl (594 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 594.7/594.7 kB 869.6 kB/s eta 0:00:00 Collecting lxml (from pyangbind->-r requirements.txt (line 22)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/lxml/5.2.2/lxml-5.2.2-cp311-cp311-manylinux_2_28_x86_64.whl (5.0 MB) Collecting regex (from pyangbind->-r requirements.txt (line 22)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/regex/2024.5.15/regex-2024.5.15-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (785 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 785.0/785.0 kB 17.1 MB/s eta 0:00:00 Collecting enum34 (from pyangbind->-r requirements.txt (line 22)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/enum34/1.1.10/enum34-1.1.10-py3-none-any.whl (11 kB) Collecting bcrypt>=3.2 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/bcrypt/4.1.3/bcrypt-4.1.3-cp39-abi3-manylinux_2_28_x86_64.whl (283 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 283.7/283.7 kB 7.9 MB/s eta 0:00:00 Collecting cryptography>=3.3 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/cryptography/42.0.8/cryptography-42.0.8-cp39-abi3-manylinux_2_28_x86_64.whl (3.9 MB) Collecting pynacl>=1.5 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/pynacl/1.5.0/PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (856 kB) Collecting selenium>=4.3.0 (from robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/selenium/4.22.0/selenium-4.22.0-py3-none-any.whl (9.4 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.4/9.4 MB 22.7 MB/s eta 0:00:00 Collecting robotframework-pythonlibcore>=4.4.1 (from robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-pythonlibcore/4.4.1/robotframework_pythonlibcore-4.4.1-py2.py3-none-any.whl (12 kB) Collecting click>=8.1.7 (from robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/click/8.1.7/click-8.1.7-py3-none-any.whl (97 kB) Collecting WebOb>=1.2 (from webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/webob/1.8.7/WebOb-1.8.7-py2.py3-none-any.whl (114 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 115.0/115.0 kB 2.6 MB/s eta 0:00:00 Collecting waitress>=0.8.5 (from webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/waitress/3.0.0/waitress-3.0.0-py3-none-any.whl (56 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 56.7/56.7 kB 1.0 MB/s eta 0:00:00 Collecting beautifulsoup4 (from webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/beautifulsoup4/4.12.3/beautifulsoup4-4.12.3-py3-none-any.whl (147 kB) Collecting cffi>=1.12 (from cryptography>=3.3->paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/cffi/1.16.0/cffi-1.16.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (464 kB) Collecting trio~=0.17 (from selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/trio/0.26.0/trio-0.26.0-py3-none-any.whl (475 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 475.7/475.7 kB 5.2 MB/s eta 0:00:00 Collecting trio-websocket~=0.9 (from selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/trio-websocket/0.11.1/trio_websocket-0.11.1-py3-none-any.whl (17 kB) Collecting soupsieve>1.2 (from beautifulsoup4->webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/soupsieve/2.5/soupsieve-2.5-py3-none-any.whl (36 kB) Collecting pycparser (from cffi>=1.12->cryptography>=3.3->paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/pycparser/2.22/pycparser-2.22-py3-none-any.whl (117 kB) Collecting attrs>=23.2.0 (from trio~=0.17->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/attrs/23.2.0/attrs-23.2.0-py3-none-any.whl (60 kB) Collecting sortedcontainers (from trio~=0.17->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/sortedcontainers/2.4.0/sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB) Collecting outcome (from trio~=0.17->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/outcome/1.3.0.post0/outcome-1.3.0.post0-py2.py3-none-any.whl (10 kB) Collecting sniffio>=1.3.0 (from trio~=0.17->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/sniffio/1.3.1/sniffio-1.3.1-py3-none-any.whl (10 kB) Collecting wsproto>=0.14 (from trio-websocket~=0.9->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/wsproto/1.2.0/wsproto-1.2.0-py3-none-any.whl (24 kB) Collecting pysocks!=1.5.7,<2.0,>=1.5.6 (from urllib3[socks]<3,>=1.26->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/pysocks/1.7.1/PySocks-1.7.1-py3-none-any.whl (16 kB) Collecting h11<1,>=0.9.0 (from wsproto>=0.14->trio-websocket~=0.9->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/h11/0.14.0/h11-0.14.0-py3-none-any.whl (58 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 58.3/58.3 kB 295.4 kB/s eta 0:00:00 Building wheels for collected packages: robotframework-sshlibrary, ipaddr, robotframework-httplibrary, scapy, jsonpath-rw Building wheel for robotframework-sshlibrary (setup.py): started Building wheel for robotframework-sshlibrary (setup.py): finished with status 'done' Created wheel for robotframework-sshlibrary: filename=robotframework_sshlibrary-3.8.0-py3-none-any.whl size=55117 sha256=8e22565c05cac059d77c8ef4547442abe6a5f4ad3acbdf64e0db5699ff807507 Stored in directory: /home/jenkins/.cache/pip/wheels/f7/c9/b3/a977b7bcc410d45ae27d240df3d00a12585509180e373ecccc Building wheel for ipaddr (setup.py): started Building wheel for ipaddr (setup.py): finished with status 'done' Created wheel for ipaddr: filename=ipaddr-2.2.0-py3-none-any.whl size=18282 sha256=ab5de7c0aacc97d56c6b0e62e61e3bd10e24e7d98371cda13bbe5788f477ff64 Stored in directory: /home/jenkins/.cache/pip/wheels/dc/6c/04/da2d847fa8d45c59af3e1d83e2acc29cb8adcbaf04c0898dbf Building wheel for robotframework-httplibrary (setup.py): started Building wheel for robotframework-httplibrary (setup.py): finished with status 'done' Created wheel for robotframework-httplibrary: filename=robotframework_httplibrary-0.4.2-py3-none-any.whl size=9954 sha256=90b67ffab203d2d2490c2c022dc421c8cbcf25bade61029e1529e2e02e1af58b Stored in directory: /home/jenkins/.cache/pip/wheels/aa/bc/0d/9a20dd51effef392aae2733cb4c7b66c6fa29fca33d88b57ed Building wheel for scapy (setup.py): started Building wheel for scapy (setup.py): finished with status 'done' Created wheel for scapy: filename=scapy-2.5.0-py2.py3-none-any.whl size=1444330 sha256=ebe043c24a7400cafa4d10b7b8f720ee8d73575b0dce1558f969ff4787e9749d Stored in directory: /home/jenkins/.cache/pip/wheels/3b/a7/d6/87eeec31fbfcb2bd8ace7dad8172b6d6ebbe1ec6827dbb389a Building wheel for jsonpath-rw (setup.py): started Building wheel for jsonpath-rw (setup.py): finished with status 'done' Created wheel for jsonpath-rw: filename=jsonpath_rw-1.4.0-py3-none-any.whl size=15128 sha256=7e0787946b1088664a0d3e5dff6d4d35f3ea822d1d89d92e2829dc592c2d7646 Stored in directory: /home/jenkins/.cache/pip/wheels/f1/54/63/9a8da38cefae13755097b36cc852decc25d8ef69c37d58d4eb Successfully built robotframework-sshlibrary ipaddr robotframework-httplibrary scapy jsonpath-rw Installing collected packages: sortedcontainers, ply, netifaces, ipaddr, enum34, websocket-client, WebOb, waitress, urllib3, typing-extensions, soupsieve, sniffio, six, scapy, robotframework-pythonlibcore, robotframework, regex, pysocks, pyparsing, pycparser, netaddr, lxml, jsonpointer, jmespath, idna, h11, decorator, click, charset-normalizer, certifi, bcrypt, attrs, wsproto, requests, python-dateutil, pyhocon, pyang, outcome, jsonpath-rw, jsonpatch, isodate, elastic-transport, docker-pycreds, cffi, beautifulsoup4, webtest, trio, robotframework-requests, pynacl, pyangbind, elasticsearch, docker-py, cryptography, trio-websocket, robotframework-httplibrary, paramiko, elasticsearch-dsl, selenium, scp, robotframework-sshlibrary, robotframework-seleniumlibrary, robotframework-selenium2library Successfully installed WebOb-1.8.7 attrs-23.2.0 bcrypt-4.1.3 beautifulsoup4-4.12.3 certifi-2024.7.4 cffi-1.16.0 charset-normalizer-3.3.2 click-8.1.7 cryptography-42.0.8 decorator-5.1.1 docker-py-1.10.6 docker-pycreds-0.4.0 elastic-transport-8.13.1 elasticsearch-8.14.0 elasticsearch-dsl-8.14.0 enum34-1.1.10 h11-0.14.0 idna-3.7 ipaddr-2.2.0 isodate-0.6.1 jmespath-1.0.1 jsonpatch-1.33 jsonpath-rw-1.4.0 jsonpointer-3.0.0 lxml-5.2.2 netaddr-1.3.0 netifaces-0.11.0 outcome-1.3.0.post0 paramiko-3.4.0 ply-3.11 pyang-2.6.1 pyangbind-0.8.5 pycparser-2.22 pyhocon-0.3.61 pynacl-1.5.0 pyparsing-3.1.2 pysocks-1.7.1 python-dateutil-2.9.0.post0 regex-2024.5.15 requests-2.32.3 robotframework-7.0.1 robotframework-httplibrary-0.4.2 robotframework-pythonlibcore-4.4.1 robotframework-requests-0.9.7 robotframework-selenium2library-3.0.0 robotframework-seleniumlibrary-6.5.0 robotframework-sshlibrary-3.8.0 scapy-2.5.0 scp-0.15.0 selenium-4.22.0 six-1.16.0 sniffio-1.3.1 sortedcontainers-2.4.0 soupsieve-2.5 trio-0.26.0 trio-websocket-0.11.1 typing-extensions-4.12.2 urllib3-2.2.2 waitress-3.0.0 websocket-client-1.8.0 webtest-3.0.0 wsproto-1.2.0 + pip freeze attrs==23.2.0 bcrypt==4.1.3 beautifulsoup4==4.12.3 certifi==2024.7.4 cffi==1.16.0 charset-normalizer==3.3.2 click==8.1.7 cryptography==42.0.8 decorator==5.1.1 distlib==0.3.8 docker-py==1.10.6 docker-pycreds==0.4.0 elastic-transport==8.13.1 elasticsearch==8.14.0 elasticsearch-dsl==8.14.0 enum34==1.1.10 filelock==3.15.4 h11==0.14.0 idna==3.7 ipaddr==2.2.0 isodate==0.6.1 jmespath==1.0.1 jsonpatch==1.33 jsonpath-rw==1.4.0 jsonpointer==3.0.0 lxml==5.2.2 netaddr==1.3.0 netifaces==0.11.0 outcome==1.3.0.post0 paramiko==3.4.0 platformdirs==4.2.2 ply==3.11 pyang==2.6.1 pyangbind==0.8.5 pycparser==2.22 pyhocon==0.3.61 PyNaCl==1.5.0 pyparsing==3.1.2 PySocks==1.7.1 python-dateutil==2.9.0.post0 regex==2024.5.15 requests==2.32.3 robotframework==7.0.1 robotframework-httplibrary==0.4.2 robotframework-pythonlibcore==4.4.1 robotframework-requests==0.9.7 robotframework-selenium2library==3.0.0 robotframework-seleniumlibrary==6.5.0 robotframework-sshlibrary==3.8.0 scapy==2.5.0 scp==0.15.0 selenium==4.22.0 six==1.16.0 sniffio==1.3.1 sortedcontainers==2.4.0 soupsieve==2.5 trio==0.26.0 trio-websocket==0.11.1 typing_extensions==4.12.2 urllib3==2.2.2 virtualenv==20.26.3 waitress==3.0.0 WebOb==1.8.7 websocket-client==1.8.0 WebTest==3.0.0 wsproto==1.2.0 [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties file path 'env.properties' [EnvInject] - Variables injected successfully. [controller-csit-3node-clustering-ask-all-calcium] $ /bin/bash -l /tmp/jenkins13301390430036269822.sh Setup pyenv: system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-calcium/.python-version) lf-activate-venv(): INFO: Reuse venv:/tmp/venv-1v9O from file:/tmp/.os_lf_venv lf-activate-venv(): INFO: Installing: python-heatclient python-openstackclient yq ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. lftools 0.37.10 requires urllib3<2.1.0, but you have urllib3 2.2.2 which is incompatible. lf-activate-venv(): INFO: Adding /tmp/venv-1v9O/bin to PATH + ODL_SYSTEM=() + TOOLS_SYSTEM=() + OPENSTACK_SYSTEM=() + OPENSTACK_CONTROLLERS=() + mapfile -t ADDR ++ openstack stack show -f json -c outputs releng-controller-csit-3node-clustering-ask-all-calcium-43 ++ jq -r '.outputs[] | select(.output_key | match("^vm_[0-9]+_ips$")) | .output_value | .[]' + for i in "${ADDR[@]}" ++ ssh 10.30.171.165 hostname -s Warning: Permanently added '10.30.171.165' (ECDSA) to the list of known hosts. + REMHOST=releng-37733-43-0-builder-0 + case ${REMHOST} in + ODL_SYSTEM=("${ODL_SYSTEM[@]}" "${i}") + for i in "${ADDR[@]}" ++ ssh 10.30.171.170 hostname -s Warning: Permanently added '10.30.171.170' (ECDSA) to the list of known hosts. + REMHOST=releng-37733-43-0-builder-1 + case ${REMHOST} in + ODL_SYSTEM=("${ODL_SYSTEM[@]}" "${i}") + for i in "${ADDR[@]}" ++ ssh 10.30.170.8 hostname -s Warning: Permanently added '10.30.170.8' (ECDSA) to the list of known hosts. + REMHOST=releng-37733-43-0-builder-2 + case ${REMHOST} in + ODL_SYSTEM=("${ODL_SYSTEM[@]}" "${i}") + echo NUM_ODL_SYSTEM=3 + echo NUM_TOOLS_SYSTEM=0 + '[' '' == yes ']' + NUM_OPENSTACK_SYSTEM=0 + echo NUM_OPENSTACK_SYSTEM=0 + '[' 0 -eq 2 ']' + echo ODL_SYSTEM_IP=10.30.171.165 ++ seq 0 2 + for i in $(seq 0 $(( ${#ODL_SYSTEM[@]} - 1 ))) + echo ODL_SYSTEM_1_IP=10.30.171.165 + for i in $(seq 0 $(( ${#ODL_SYSTEM[@]} - 1 ))) + echo ODL_SYSTEM_2_IP=10.30.171.170 + for i in $(seq 0 $(( ${#ODL_SYSTEM[@]} - 1 ))) + echo ODL_SYSTEM_3_IP=10.30.170.8 + echo TOOLS_SYSTEM_IP= ++ seq 0 -1 + openstack_index=0 + NUM_OPENSTACK_CONTROL_NODES=1 + echo NUM_OPENSTACK_CONTROL_NODES=1 ++ seq 0 0 + for i in $(seq 0 $((NUM_OPENSTACK_CONTROL_NODES - 1))) + echo OPENSTACK_CONTROL_NODE_1_IP= + NUM_OPENSTACK_COMPUTE_NODES=-1 + echo NUM_OPENSTACK_COMPUTE_NODES=-1 + '[' -1 -ge 2 ']' ++ seq 0 -2 + NUM_OPENSTACK_HAPROXY_NODES=0 + echo NUM_OPENSTACK_HAPROXY_NODES=0 ++ seq 0 -1 + echo 'Contents of slave_addresses.txt:' Contents of slave_addresses.txt: + cat slave_addresses.txt NUM_ODL_SYSTEM=3 NUM_TOOLS_SYSTEM=0 NUM_OPENSTACK_SYSTEM=0 ODL_SYSTEM_IP=10.30.171.165 ODL_SYSTEM_1_IP=10.30.171.165 ODL_SYSTEM_2_IP=10.30.171.170 ODL_SYSTEM_3_IP=10.30.170.8 TOOLS_SYSTEM_IP= NUM_OPENSTACK_CONTROL_NODES=1 OPENSTACK_CONTROL_NODE_1_IP= NUM_OPENSTACK_COMPUTE_NODES=-1 NUM_OPENSTACK_HAPROXY_NODES=0 [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties file path 'slave_addresses.txt' [EnvInject] - Variables injected successfully. [controller-csit-3node-clustering-ask-all-calcium] $ /bin/sh /tmp/jenkins5964274710589056967.sh Preparing for JRE Version 17 Karaf artifact is karaf Karaf project is integration Java home is /usr/lib/jvm/java-17-openjdk [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties file path 'set_variables.env' [EnvInject] - Variables injected successfully. [controller-csit-3node-clustering-ask-all-calcium] $ /bin/bash /tmp/jenkins2931378849747796080.sh --2024-07-07 00:27:10-- http://git.opendaylight.org/gerrit/gitweb?p=integration/distribution.git;a=blob_plain;f=pom.xml;hb=refs/heads/stable/calcium Resolving git.opendaylight.org (git.opendaylight.org)... 35.181.223.58, 52.47.161.242, 13.39.230.180, ... Connecting to git.opendaylight.org (git.opendaylight.org)|35.181.223.58|:80... connected. HTTP request sent, awaiting response... 200 OK Length: 2619 (2.6K) [text/plain] Saving to: ‘pom.xml’ 0K .. 100% 327M=0s 2024-07-07 00:27:11 (327 MB/s) - ‘pom.xml’ saved [2619/2619] Bundle version is 0.20.2-SNAPSHOT --2024-07-07 00:27:11-- https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/maven-metadata.xml Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6 Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 1404 (1.4K) [application/xml] Saving to: ‘maven-metadata.xml’ 0K . 100% 73.7M=0s 2024-07-07 00:27:11 (73.7 MB/s) - ‘maven-metadata.xml’ saved [1404/1404] org.opendaylight.integration karaf 0.20.2-SNAPSHOT 20240706.204634 46 20240706204634 pom 0.20.2-20240706.204634-46 20240706204634 tar.gz 0.20.2-20240706.204634-46 20240706204634 zip 0.20.2-20240706.204634-46 20240706204634 cyclonedx xml 0.20.2-20240706.204634-46 20240706204634 cyclonedx json 0.20.2-20240706.204634-46 20240706204634 Nexus timestamp is 0.20.2-20240706.204634-46 Distribution bundle URL is https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240706.204634-46.zip Distribution bundle is karaf-0.20.2-20240706.204634-46.zip Distribution bundle version is 0.20.2-SNAPSHOT Distribution folder is karaf-0.20.2-SNAPSHOT Nexus prefix is https://nexus.opendaylight.org [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties file path 'detect_variables.env' [EnvInject] - Variables injected successfully. [controller-csit-3node-clustering-ask-all-calcium] $ /bin/bash -l /tmp/jenkins6765234527008644972.sh Setup pyenv: system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-calcium/.python-version) lf-activate-venv(): INFO: Reuse venv:/tmp/venv-1v9O from file:/tmp/.os_lf_venv lf-activate-venv(): INFO: Installing: python-heatclient python-openstackclient lf-activate-venv(): INFO: Adding /tmp/venv-1v9O/bin to PATH Copying common-functions.sh to /tmp Copying common-functions.sh to 10.30.171.165:/tmp Warning: Permanently added '10.30.171.165' (ECDSA) to the list of known hosts. Copying common-functions.sh to 10.30.171.170:/tmp Warning: Permanently added '10.30.171.170' (ECDSA) to the list of known hosts. Copying common-functions.sh to 10.30.170.8:/tmp Warning: Permanently added '10.30.170.8' (ECDSA) to the list of known hosts. [controller-csit-3node-clustering-ask-all-calcium] $ /bin/bash /tmp/jenkins1654060903777992231.sh common-functions.sh is being sourced common-functions environment: MAVENCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ACTUALFEATURES: FEATURESCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg CUSTOMPROP: /tmp/karaf-0.20.2-SNAPSHOT/etc/custom.properties LOGCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg MEMCONF: /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv CONTROLLERMEM: 2048m AKKACONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf MODULESCONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf MODULESHARDSCONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf SUITES: ################################################# ## Configure Cluster and Start ## ################################################# ACTUALFEATURES: odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app SPACE_SEPARATED_FEATURES: odl-integration-compatible-with-all odl-jolokia odl-restconf odl-clustering-test-app Locating script plan to use... script plan exists!!! Changing the script plan path... # Place the scripts in run order: /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/scripts/car_people_shard.sh Executing /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/scripts/car_people_shard.sh... Add car-people shards file Copy shard config to member-1 with IP address 10.30.171.165 Warning: Permanently added '10.30.171.165' (ECDSA) to the list of known hosts. Copy shard config to member-2 with IP address 10.30.171.170 Warning: Permanently added '10.30.171.170' (ECDSA) to the list of known hosts. Copy shard config to member-3 with IP address 10.30.170.8 Warning: Permanently added '10.30.170.8' (ECDSA) to the list of known hosts. Finished running script plans Configuring member-1 with IP address 10.30.171.165 Warning: Permanently added '10.30.171.165' (ECDSA) to the list of known hosts. Warning: Permanently added '10.30.171.165' (ECDSA) to the list of known hosts. + source /tmp/common-functions.sh karaf-0.20.2-SNAPSHOT ++ [[ /tmp/common-functions.sh == \/\t\m\p\/\c\o\n\f\i\g\u\r\a\t\i\o\n\-\s\c\r\i\p\t\.\s\h ]] common-functions.sh is being sourced ++ echo 'common-functions.sh is being sourced' ++ BUNDLEFOLDER=karaf-0.20.2-SNAPSHOT ++ export MAVENCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ++ MAVENCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ++ export FEATURESCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg ++ FEATURESCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg ++ export CUSTOMPROP=/tmp/karaf-0.20.2-SNAPSHOT/etc/custom.properties ++ CUSTOMPROP=/tmp/karaf-0.20.2-SNAPSHOT/etc/custom.properties ++ export LOGCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg ++ LOGCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg ++ export MEMCONF=/tmp/karaf-0.20.2-SNAPSHOT/bin/setenv ++ MEMCONF=/tmp/karaf-0.20.2-SNAPSHOT/bin/setenv ++ export CONTROLLERMEM= ++ CONTROLLERMEM= ++ export AKKACONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf ++ AKKACONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf ++ export MODULESCONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf ++ MODULESCONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf ++ export MODULESHARDSCONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf ++ MODULESHARDSCONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf ++ print_common_env ++ cat common-functions environment: MAVENCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ACTUALFEATURES: FEATURESCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg CUSTOMPROP: /tmp/karaf-0.20.2-SNAPSHOT/etc/custom.properties LOGCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg MEMCONF: /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv CONTROLLERMEM: AKKACONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf MODULESCONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf MODULESHARDSCONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf SUITES: ++ SSH='ssh -t -t' ++ extra_services_cntl=' dnsmasq.service httpd.service libvirtd.service openvswitch.service ovs-vswitchd.service ovsdb-server.service rabbitmq-server.service ' ++ extra_services_cmp=' libvirtd.service openvswitch.service ovs-vswitchd.service ovsdb-server.service ' Changing to /tmp + echo 'Changing to /tmp' + cd /tmp Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240706.204634-46.zip + echo 'Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240706.204634-46.zip' + wget --progress=dot:mega https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240706.204634-46.zip --2024-07-07 00:27:20-- https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240706.204634-46.zip Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6 Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 268186792 (256M) [application/zip] Saving to: ‘karaf-0.20.2-20240706.204634-46.zip’ 0K ........ ........ ........ ........ ........ ........ 1% 41.7M 6s 3072K ........ ........ ........ ........ ........ ........ 2% 84.3M 4s 6144K ........ ........ ........ ........ ........ ........ 3% 86.7M 4s 9216K ........ ........ ........ ........ ........ ........ 4% 97.5M 4s 12288K ........ ........ ........ ........ ........ ........ 5% 94.8M 3s 15360K ........ ........ ........ ........ ........ ........ 7% 154M 3s 18432K ........ ........ ........ ........ ........ ........ 8% 89.1M 3s 21504K ........ ........ ........ ........ ........ ........ 9% 116M 3s 24576K ........ ........ ........ ........ ........ ........ 10% 102M 3s 27648K ........ ........ ........ ........ ........ ........ 11% 124M 3s 30720K ........ ........ ........ ........ ........ ........ 12% 111M 2s 33792K ........ ........ ........ ........ ........ ........ 14% 118M 2s 36864K ........ ........ ........ ........ ........ ........ 15% 134M 2s 39936K ........ ........ ........ ........ ........ ........ 16% 156M 2s 43008K ........ ........ ........ ........ ........ ........ 17% 118M 2s 46080K ........ ........ ........ ........ ........ ........ 18% 125M 2s 49152K ........ ........ ........ ........ ........ ........ 19% 134M 2s 52224K ........ ........ ........ ........ ........ ........ 21% 115M 2s 55296K ........ ........ ........ ........ ........ ........ 22% 117M 2s 58368K ........ ........ ........ ........ ........ ........ 23% 129M 2s 61440K ........ ........ ........ ........ ........ ........ 24% 76.1M 2s 64512K ........ ........ ........ ........ ........ ........ 25% 115M 2s 67584K ........ ........ ........ ........ ........ ........ 26% 126M 2s 70656K ........ ........ ........ ........ ........ ........ 28% 132M 2s 73728K ........ ........ ........ ........ ........ ........ 29% 121M 2s 76800K ........ ........ ........ ........ ........ ........ 30% 94.5M 2s 79872K ........ ........ ........ ........ ........ ........ 31% 126M 2s 82944K ........ ........ ........ ........ ........ ........ 32% 214M 2s 86016K ........ ........ ........ ........ ........ ........ 34% 204M 2s 89088K ........ ........ ........ ........ ........ ........ 35% 218M 1s 92160K ........ ........ ........ ........ ........ ........ 36% 233M 1s 95232K ........ ........ ........ ........ ........ ........ 37% 238M 1s 98304K ........ ........ ........ ........ ........ ........ 38% 232M 1s 101376K ........ ........ ........ ........ ........ ........ 39% 246M 1s 104448K ........ ........ ........ ........ ........ ........ 41% 261M 1s 107520K ........ ........ ........ ........ ........ ........ 42% 222M 1s 110592K ........ ........ ........ ........ ........ ........ 43% 225M 1s 113664K ........ ........ ........ ........ ........ ........ 44% 272M 1s 116736K ........ ........ ........ ........ ........ ........ 45% 223M 1s 119808K ........ ........ ........ ........ ........ ........ 46% 299M 1s 122880K ........ ........ ........ ........ ........ ........ 48% 256M 1s 125952K ........ ........ ........ ........ ........ ........ 49% 284M 1s 129024K ........ ........ ........ ........ ........ ........ 50% 275M 1s 132096K ........ ........ ........ ........ ........ ........ 51% 296M 1s 135168K ........ ........ ........ ........ ........ ........ 52% 289M 1s 138240K ........ ........ ........ ........ ........ ........ 53% 293M 1s 141312K ........ ........ ........ ........ ........ ........ 55% 321M 1s 144384K ........ ........ ........ ........ ........ ........ 56% 299M 1s 147456K ........ ........ ........ ........ ........ ........ 57% 272M 1s 150528K ........ ........ ........ ........ ........ ........ 58% 280M 1s 153600K ........ ........ ........ ........ ........ ........ 59% 281M 1s 156672K ........ ........ ........ ........ ........ ........ 60% 208M 1s 159744K ........ ........ ........ ........ ........ ........ 62% 306M 1s 162816K ........ ........ ........ ........ ........ ........ 63% 308M 1s 165888K ........ ........ ........ ........ ........ ........ 64% 141M 1s 168960K ........ ........ ........ ........ ........ ........ 65% 225M 1s 172032K ........ ........ ........ ........ ........ ........ 66% 274M 1s 175104K ........ ........ ........ ........ ........ ........ 68% 285M 1s 178176K ........ ........ ........ ........ ........ ........ 69% 297M 1s 181248K ........ ........ ........ ........ ........ ........ 70% 351M 0s 184320K ........ ........ ........ ........ ........ ........ 71% 346M 0s 187392K ........ ........ ........ ........ ........ ........ 72% 357M 0s 190464K ........ ........ ........ ........ ........ ........ 73% 301M 0s 193536K ........ ........ ........ ........ ........ ........ 75% 340M 0s 196608K ........ ........ ........ ........ ........ ........ 76% 339M 0s 199680K ........ ........ ........ ........ ........ ........ 77% 312M 0s 202752K ........ ........ ........ ........ ........ ........ 78% 312M 0s 205824K ........ ........ ........ ........ ........ ........ 79% 335M 0s 208896K ........ ........ ........ ........ ........ ........ 80% 339M 0s 211968K ........ ........ ........ ........ ........ ........ 82% 325M 0s 215040K ........ ........ ........ ........ ........ ........ 83% 335M 0s 218112K ........ ........ ........ ........ ........ ........ 84% 354M 0s 221184K ........ ........ ........ ........ ........ ........ 85% 345M 0s 224256K ........ ........ ........ ........ ........ ........ 86% 370M 0s 227328K ........ ........ ........ ........ ........ ........ 87% 364M 0s 230400K ........ ........ ........ ........ ........ ........ 89% 366M 0s 233472K ........ ........ ........ ........ ........ ........ 90% 334M 0s 236544K ........ ........ ........ ........ ........ ........ 91% 348M 0s 239616K ........ ........ ........ ........ ........ ........ 92% 291M 0s 242688K ........ ........ ........ ........ ........ ........ 93% 309M 0s 245760K ........ ........ ........ ........ ........ ........ 95% 302M 0s 248832K ........ ........ ........ ........ ........ ........ 96% 297M 0s 251904K ........ ........ ........ ........ ........ ........ 97% 228M 0s 254976K ........ ........ ........ ........ ........ ........ 98% 330M 0s 258048K ........ ........ ........ ........ ........ ........ 99% 265M 0s 261120K ........ .... 100% 292M=1.4s 2024-07-07 00:27:21 (183 MB/s) - ‘karaf-0.20.2-20240706.204634-46.zip’ saved [268186792/268186792] Extracting the new controller... + echo 'Extracting the new controller...' + unzip -q karaf-0.20.2-20240706.204634-46.zip Adding external repositories... + echo 'Adding external repositories...' + sed -ie 's%org.ops4j.pax.url.mvn.repositories=%org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases%g' /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg + cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ################################################################################ # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # ################################################################################ # # If set to true, the following property will not allow any certificate to be used # when accessing Maven repositories through SSL # #org.ops4j.pax.url.mvn.certificateCheck= # # Path to the local Maven settings file. # The repositories defined in this file will be automatically added to the list # of default repositories if the 'org.ops4j.pax.url.mvn.repositories' property # below is not set. # The following locations are checked for the existence of the settings.xml file # * 1. looks for the specified url # * 2. if not found looks for ${user.home}/.m2/settings.xml # * 3. if not found looks for ${maven.home}/conf/settings.xml # * 4. if not found looks for ${M2_HOME}/conf/settings.xml # #org.ops4j.pax.url.mvn.settings= # # Path to the local Maven repository which is used to avoid downloading # artifacts when they already exist locally. # The value of this property will be extracted from the settings.xml file # above, or defaulted to: # System.getProperty( "user.home" ) + "/.m2/repository" # org.ops4j.pax.url.mvn.localRepository=${karaf.home}/${karaf.default.repository} # # Default this to false. It's just weird to use undocumented repos # org.ops4j.pax.url.mvn.useFallbackRepositories=false # # Uncomment if you don't wanna use the proxy settings # from the Maven conf/settings.xml file # # org.ops4j.pax.url.mvn.proxySupport=false # # Comma separated list of repositories scanned when resolving an artifact. # Those repositories will be checked before iterating through the # below list of repositories and even before the local repository # A repository url can be appended with zero or more of the following flags: # @snapshots : the repository contains snaphots # @noreleases : the repository does not contain any released artifacts # # The following property value will add the system folder as a repo. # org.ops4j.pax.url.mvn.defaultRepositories=\ file:${karaf.home}/${karaf.default.repository}@id=system.repository@snapshots,\ file:${karaf.data}/kar@id=kar.repository@multi@snapshots,\ file:${karaf.base}/${karaf.default.repository}@id=child.system.repository@snapshots # Use the default local repo (e.g.~/.m2/repository) as a "remote" repo #org.ops4j.pax.url.mvn.defaultLocalRepoAsRemote=false # # Comma separated list of repositories scanned when resolving an artifact. # The default list includes the following repositories: # http://repo1.maven.org/maven2@id=central # http://repository.springsource.com/maven/bundles/release@id=spring.ebr # http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external # http://zodiac.springsource.com/maven/bundles/release@id=gemini # http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases # https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases # https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases # To add repositories to the default ones, prepend '+' to the list of repositories # to add. # A repository url can be appended with zero or more of the following flags: # @snapshots : the repository contains snapshots # @noreleases : the repository does not contain any released artifacts # @id=repository.id : the id for the repository, just like in the settings.xml this is optional but recommended # org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases ### ^^^ No remote repositories. This is the only ODL change compared to Karaf defaults.Configuring the startup features... + [[ True == \T\r\u\e ]] + echo 'Configuring the startup features...' + sed -ie 's/\(featuresBoot=\|featuresBoot =\)/featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app,/g' /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg + FEATURE_TEST_STRING=features-test + FEATURE_TEST_VERSION=0.20.2-SNAPSHOT + KARAF_VERSION=karaf4 + [[ integration == \i\n\t\e\g\r\a\t\i\o\n ]] + sed -ie 's%\(featuresRepositories=\|featuresRepositories =\)%featuresRepositories = mvn:org.opendaylight.integration/features-test/0.20.2-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features,%g' /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg + [[ ! -z '' ]] + cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg ################################################################################ # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # ################################################################################ # # Comma separated list of features repositories to register by default # featuresRepositories = mvn:org.opendaylight.integration/features-test/0.20.2-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features, file:${karaf.etc}/97847919-d091-419f-b580-b699f0fd2fda.xml # # Comma separated list of features to install at startup # featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app, a1334869-d56b-49d8-9c5b-fed53f7d2c4c # # Resource repositories (OBR) that the features resolver can use # to resolve requirements/capabilities # # The format of the resourceRepositories is # resourceRepositories=[xml:url|json:url],... # for Instance: # #resourceRepositories=xml:http://host/path/to/index.xml # or #resourceRepositories=json:http://host/path/to/index.json # # # Defines if the boot features are started in asynchronous mode (in a dedicated thread) # featuresBootAsynchronous=false # # Service requirements enforcement # # By default, the feature resolver checks the service requirements/capabilities of # bundles for new features (xml schema >= 1.3.0) in order to automatically installs # the required bundles. # The following flag can have those values: # - disable: service requirements are completely ignored # - default: service requirements are ignored for old features # - enforce: service requirements are always verified # #serviceRequirements=default # # Store cfg file for config element in feature # #configCfgStore=true # # Define if the feature service automatically refresh bundles # autoRefresh=true # # Configuration of features processing mechanism (overrides, blacklisting, modification of features) # XML file defines instructions related to features processing # versions.properties may declare properties to resolve placeholders in XML file # both files are relative to ${karaf.etc} # #featureProcessing=org.apache.karaf.features.xml #featureProcessingVersions=versions.properties + configure_karaf_log karaf4 '' + local -r karaf_version=karaf4 + local -r controllerdebugmap= + local logapi=log4j + grep log4j2 /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n log4j2.rootLogger.level = INFO #log4j2.rootLogger.type = asyncRoot #log4j2.rootLogger.includeLocation = false log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi log4j2.rootLogger.appenderRef.Console.ref = Console log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF} log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL log4j2.logger.spifly.name = org.apache.aries.spifly log4j2.logger.spifly.level = WARN log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit log4j2.logger.audit.level = INFO log4j2.logger.audit.additivity = false log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile # Console appender not used by default (see log4j2.rootLogger.appenderRefs) log4j2.appender.console.type = Console log4j2.appender.console.name = Console log4j2.appender.console.layout.type = PatternLayout log4j2.appender.console.layout.pattern = ${log4j2.pattern} log4j2.appender.rolling.type = RollingRandomAccessFile log4j2.appender.rolling.name = RollingFile log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i #log4j2.appender.rolling.immediateFlush = false log4j2.appender.rolling.append = true log4j2.appender.rolling.layout.type = PatternLayout log4j2.appender.rolling.layout.pattern = ${log4j2.pattern} log4j2.appender.rolling.policies.type = Policies log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy log4j2.appender.rolling.policies.size.size = 64MB log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy log4j2.appender.rolling.strategy.max = 7 log4j2.appender.audit.type = RollingRandomAccessFile log4j2.appender.audit.name = AuditRollingFile log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i log4j2.appender.audit.append = true log4j2.appender.audit.layout.type = PatternLayout log4j2.appender.audit.layout.pattern = ${log4j2.pattern} log4j2.appender.audit.policies.type = Policies log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy log4j2.appender.audit.policies.size.size = 8MB log4j2.appender.audit.strategy.type = DefaultRolloverStrategy log4j2.appender.audit.strategy.max = 7 log4j2.appender.osgi.type = PaxOsgi log4j2.appender.osgi.name = PaxOsgi log4j2.appender.osgi.filter = * #log4j2.logger.aether.name = shaded.org.eclipse.aether #log4j2.logger.aether.level = TRACE #log4j2.logger.http-headers.name = shaded.org.apache.http.headers #log4j2.logger.http-headers.level = DEBUG #log4j2.logger.maven.name = org.ops4j.pax.url.mvn #log4j2.logger.maven.level = TRACE + logapi=log4j2 Configuring the karaf log... karaf_version: karaf4, logapi: log4j2 + echo 'Configuring the karaf log... karaf_version: karaf4, logapi: log4j2' + '[' log4j2 == log4j2 ']' + sed -ie 's/log4j2.appender.rolling.policies.size.size = 64MB/log4j2.appender.rolling.policies.size.size = 1GB/g' /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg + orgmodule=org.opendaylight.yangtools.yang.parser.repo.YangTextSchemaContextResolver + orgmodule_=org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver + echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN' + echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN' controllerdebugmap: + unset IFS + echo 'controllerdebugmap: ' cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg + '[' -n '' ']' + echo 'cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg' + cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg ################################################################################ # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # ################################################################################ # Common pattern layout for appenders log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n # Root logger log4j2.rootLogger.level = INFO # uncomment to use asynchronous loggers, which require mvn:com.lmax/disruptor/3.3.2 library #log4j2.rootLogger.type = asyncRoot #log4j2.rootLogger.includeLocation = false log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi log4j2.rootLogger.appenderRef.Console.ref = Console log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF} # Filters for logs marked by org.opendaylight.odlparent.Markers log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL # Loggers configuration # Spifly logger log4j2.logger.spifly.name = org.apache.aries.spifly log4j2.logger.spifly.level = WARN # Security audit logger log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit log4j2.logger.audit.level = INFO log4j2.logger.audit.additivity = false log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile # Appenders configuration # Console appender not used by default (see log4j2.rootLogger.appenderRefs) log4j2.appender.console.type = Console log4j2.appender.console.name = Console log4j2.appender.console.layout.type = PatternLayout log4j2.appender.console.layout.pattern = ${log4j2.pattern} # Rolling file appender log4j2.appender.rolling.type = RollingRandomAccessFile log4j2.appender.rolling.name = RollingFile log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i # uncomment to not force a disk flush #log4j2.appender.rolling.immediateFlush = false log4j2.appender.rolling.append = true log4j2.appender.rolling.layout.type = PatternLayout log4j2.appender.rolling.layout.pattern = ${log4j2.pattern} log4j2.appender.rolling.policies.type = Policies log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy log4j2.appender.rolling.policies.size.size = 1GB log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy log4j2.appender.rolling.strategy.max = 7 # Audit file appender log4j2.appender.audit.type = RollingRandomAccessFile log4j2.appender.audit.name = AuditRollingFile log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i log4j2.appender.audit.append = true log4j2.appender.audit.layout.type = PatternLayout log4j2.appender.audit.layout.pattern = ${log4j2.pattern} log4j2.appender.audit.policies.type = Policies log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy log4j2.appender.audit.policies.size.size = 8MB log4j2.appender.audit.strategy.type = DefaultRolloverStrategy log4j2.appender.audit.strategy.max = 7 # OSGi appender log4j2.appender.osgi.type = PaxOsgi log4j2.appender.osgi.name = PaxOsgi log4j2.appender.osgi.filter = * # help with identification of maven-related problems with pax-url-aether #log4j2.logger.aether.name = shaded.org.eclipse.aether #log4j2.logger.aether.level = TRACE #log4j2.logger.http-headers.name = shaded.org.apache.http.headers #log4j2.logger.http-headers.level = DEBUG #log4j2.logger.maven.name = org.ops4j.pax.url.mvn #log4j2.logger.maven.level = TRACE log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN Configure java home: /usr/lib/jvm/java-17-openjdk max memory: 3072m memconf: /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv + set_java_vars /usr/lib/jvm/java-17-openjdk 3072m /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv + local -r java_home=/usr/lib/jvm/java-17-openjdk + local -r controllermem=3072m + local -r memconf=/tmp/karaf-0.20.2-SNAPSHOT/bin/setenv + echo Configure + echo ' java home: /usr/lib/jvm/java-17-openjdk' + echo ' max memory: 3072m' + echo ' memconf: /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv' + sed -ie 's%^# export JAVA_HOME%export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-17-openjdk}%g' /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv + sed -ie 's/JAVA_MAX_MEM="2048m"/JAVA_MAX_MEM=3072m/g' /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv cat /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv + echo 'cat /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv' + cat /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv #!/bin/sh # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # # # handle specific scripts; the SCRIPT_NAME is exactly the name of the Karaf # script: client, instance, shell, start, status, stop, karaf # # if [ "${KARAF_SCRIPT}" == "SCRIPT_NAME" ]; then # Actions go here... # fi # # general settings which should be applied for all scripts go here; please keep # in mind that it is possible that scripts might be executed more than once, e.g. # in example of the start script where the start script is executed first and the # karaf script afterwards. # # # The following section shows the possible configuration options for the default # karaf scripts # export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-17-openjdk} # Location of Java installation # export JAVA_OPTS # Generic JVM options, for instance, where you can pass the memory configuration # export JAVA_NON_DEBUG_OPTS # Additional non-debug JVM options # export EXTRA_JAVA_OPTS # Additional JVM options # export KARAF_HOME # Karaf home folder # export KARAF_DATA # Karaf data folder # export KARAF_BASE # Karaf base folder # export KARAF_ETC # Karaf etc folder # export KARAF_LOG # Karaf log folder # export KARAF_SYSTEM_OPTS # First citizen Karaf options # export KARAF_OPTS # Additional available Karaf options # export KARAF_DEBUG # Enable debug mode # export KARAF_REDIRECT # Enable/set the std/err redirection when using bin/start # export KARAF_NOROOT # Prevent execution as root if set to true Set Java version + echo 'Set Java version' + sudo /usr/sbin/alternatives --install /usr/bin/java java /usr/lib/jvm/java-17-openjdk/bin/java 1 + sudo /usr/sbin/alternatives --set java /usr/lib/jvm/java-17-openjdk/bin/java JDK default version ... + echo 'JDK default version ...' + java -version openjdk version "17.0.6-ea" 2023-01-17 LTS OpenJDK Runtime Environment (Red_Hat-17.0.6.0.9-0.3.ea.el8) (build 17.0.6-ea+9-LTS) OpenJDK 64-Bit Server VM (Red_Hat-17.0.6.0.9-0.3.ea.el8) (build 17.0.6-ea+9-LTS, mixed mode, sharing) Set JAVA_HOME + echo 'Set JAVA_HOME' + export JAVA_HOME=/usr/lib/jvm/java-17-openjdk + JAVA_HOME=/usr/lib/jvm/java-17-openjdk ++ readlink -e /usr/lib/jvm/java-17-openjdk/bin/java Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin/java Listing all open ports on controller system... + JAVA_RESOLVED=/usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin/java + echo 'Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin/java' + echo 'Listing all open ports on controller system...' + netstat -pnatu (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN - tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN - tcp 0 0 10.30.171.165:22 10.30.171.184:60214 ESTABLISHED - tcp 0 0 10.30.171.165:58474 199.204.45.87:443 TIME_WAIT - tcp6 0 0 :::111 :::* LISTEN - tcp6 0 0 :::5555 :::* LISTEN - tcp6 0 0 :::22 :::* LISTEN - udp 0 0 10.30.171.165:68 10.30.170.2:67 ESTABLISHED - udp 0 0 0.0.0.0:111 0.0.0.0:* - udp 0 0 127.0.0.1:323 0.0.0.0:* - udp6 0 0 :::111 :::* - udp6 0 0 ::1:323 :::* - Custom shard config exists!!! Copying the shard config... + '[' -f /tmp/custom_shard_config.txt ']' + echo 'Custom shard config exists!!!' + echo 'Copying the shard config...' + cp /tmp/custom_shard_config.txt /tmp/karaf-0.20.2-SNAPSHOT/bin/ Configuring cluster + echo 'Configuring cluster' + /tmp/karaf-0.20.2-SNAPSHOT/bin/configure_cluster.sh 1 10.30.171.165 10.30.171.170 10.30.170.8 ################################################ ## Configure Cluster ## ################################################ NOTE: Cluster configuration files not found. Copying from /tmp/karaf-0.20.2-SNAPSHOT/system/org/opendaylight/controller/sal-clustering-config/9.0.4 Configuring unique name in akka.conf Configuring hostname in akka.conf Configuring data and rpc seed nodes in akka.conf modules = [ { name = "inventory" namespace = "urn:opendaylight:inventory" shard-strategy = "module" }, { name = "topology" namespace = "urn:TBD:params:xml:ns:yang:network-topology" shard-strategy = "module" }, { name = "toaster" namespace = "http://netconfcentral.org/ns/toaster" shard-strategy = "module" }, { name = "car" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car" shard-strategy = "module" }, { name = "people" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people" shard-strategy = "module" }, { name = "car-people" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people" shard-strategy = "module" } ] Configuring replication type in module-shards.conf ################################################ ## NOTE: Manually restart controller to ## ## apply configuration. ## ################################################ Dump akka.conf + echo 'Dump akka.conf' + cat /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf odl-cluster-data { akka { remote { artery { enabled = on transport = tcp canonical.hostname = "10.30.171.165" canonical.port = 2550 } } cluster { # Using artery. seed-nodes = ["akka://opendaylight-cluster-data@10.30.171.165:2550", "akka://opendaylight-cluster-data@10.30.171.170:2550", "akka://opendaylight-cluster-data@10.30.170.8:2550"] roles = ["member-1"] # when under load we might trip a false positive on the failure detector # failure-detector { # heartbeat-interval = 4 s # acceptable-heartbeat-pause = 16s # } } persistence { # By default the snapshots/journal directories live in KARAF_HOME. You can choose to put it somewhere else by # modifying the following two properties. The directory location specified may be a relative or absolute path. # The relative path is always relative to KARAF_HOME. # snapshot-store.local.dir = "target/snapshots" # Use lz4 compression for LocalSnapshotStore snapshots snapshot-store.local.use-lz4-compression = false # Size of blocks for lz4 compression: 64KB, 256KB, 1MB or 4MB snapshot-store.local.lz4-blocksize = 256KB } disable-default-actor-system-quarantined-event-handling = "false" } } Dump modules.conf + echo 'Dump modules.conf' + cat /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf modules = [ { name = "inventory" namespace = "urn:opendaylight:inventory" shard-strategy = "module" }, { name = "topology" namespace = "urn:TBD:params:xml:ns:yang:network-topology" shard-strategy = "module" }, { name = "toaster" namespace = "http://netconfcentral.org/ns/toaster" shard-strategy = "module" }, { name = "car" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car" shard-strategy = "module" }, { name = "people" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people" shard-strategy = "module" }, { name = "car-people" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people" shard-strategy = "module" } ] Dump module-shards.conf + echo 'Dump module-shards.conf' + cat /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf module-shards = [ { name = "default" shards = [ { name = "default" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "inventory" shards = [ { name="inventory" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "topology" shards = [ { name="topology" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "toaster" shards = [ { name="toaster" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "car" shards = [ { name="car" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "people" shards = [ { name="people" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "car-people" shards = [ { name="car-people" replicas = ["member-1", "member-2", "member-3"] } ] } ] Configuring member-2 with IP address 10.30.171.170 Warning: Permanently added '10.30.171.170' (ECDSA) to the list of known hosts. Warning: Permanently added '10.30.171.170' (ECDSA) to the list of known hosts. + source /tmp/common-functions.sh karaf-0.20.2-SNAPSHOT ++ [[ /tmp/common-functions.sh == \/\t\m\p\/\c\o\n\f\i\g\u\r\a\t\i\o\n\-\s\c\r\i\p\t\.\s\h ]] common-functions.sh is being sourced ++ echo 'common-functions.sh is being sourced' ++ BUNDLEFOLDER=karaf-0.20.2-SNAPSHOT ++ export MAVENCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ++ MAVENCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ++ export FEATURESCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg ++ FEATURESCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg ++ export CUSTOMPROP=/tmp/karaf-0.20.2-SNAPSHOT/etc/custom.properties ++ CUSTOMPROP=/tmp/karaf-0.20.2-SNAPSHOT/etc/custom.properties ++ export LOGCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg ++ LOGCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg ++ export MEMCONF=/tmp/karaf-0.20.2-SNAPSHOT/bin/setenv ++ MEMCONF=/tmp/karaf-0.20.2-SNAPSHOT/bin/setenv ++ export CONTROLLERMEM= ++ CONTROLLERMEM= ++ export AKKACONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf ++ AKKACONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf ++ export MODULESCONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf ++ MODULESCONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf ++ export MODULESHARDSCONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf ++ MODULESHARDSCONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf ++ print_common_env ++ cat common-functions environment: MAVENCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ACTUALFEATURES: FEATURESCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg CUSTOMPROP: /tmp/karaf-0.20.2-SNAPSHOT/etc/custom.properties LOGCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg MEMCONF: /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv CONTROLLERMEM: AKKACONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf MODULESCONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf MODULESHARDSCONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf SUITES: ++ SSH='ssh -t -t' ++ extra_services_cntl=' dnsmasq.service httpd.service libvirtd.service openvswitch.service ovs-vswitchd.service ovsdb-server.service rabbitmq-server.service ' ++ extra_services_cmp=' libvirtd.service openvswitch.service ovs-vswitchd.service ovsdb-server.service ' Changing to /tmp Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240706.204634-46.zip + echo 'Changing to /tmp' + cd /tmp + echo 'Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240706.204634-46.zip' + wget --progress=dot:mega https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240706.204634-46.zip --2024-07-07 00:27:25-- https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240706.204634-46.zip Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6 Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 268186792 (256M) [application/zip] Saving to: ‘karaf-0.20.2-20240706.204634-46.zip’ 0K ........ ........ ........ ........ ........ ........ 1% 31.6M 8s 3072K ........ ........ ........ ........ ........ ........ 2% 59.3M 6s 6144K ........ ........ ........ ........ ........ ........ 3% 66.7M 5s 9216K ........ ........ ........ ........ ........ ........ 4% 140M 4s 12288K ........ ........ ........ ........ ........ ........ 5% 135M 4s 15360K ........ ........ ........ ........ ........ ........ 7% 165M 3s 18432K ........ ........ ........ ........ ........ ........ 8% 155M 3s 21504K ........ ........ ........ ........ ........ ........ 9% 173M 3s 24576K ........ ........ ........ ........ ........ ........ 10% 129M 3s 27648K ........ ........ ........ ........ ........ ........ 11% 188M 2s 30720K ........ ........ ........ ........ ........ ........ 12% 166M 2s 33792K ........ ........ ........ ........ ........ ........ 14% 200M 2s 36864K ........ ........ ........ ........ ........ ........ 15% 235M 2s 39936K ........ ........ ........ ........ ........ ........ 16% 233M 2s 43008K ........ ........ ........ ........ ........ ........ 17% 235M 2s 46080K ........ ........ ........ ........ ........ ........ 18% 221M 2s 49152K ........ ........ ........ ........ ........ ........ 19% 238M 2s 52224K ........ ........ ........ ........ ........ ........ 21% 234M 2s 55296K ........ ........ ........ ........ ........ ........ 22% 248M 2s 58368K ........ ........ ........ ........ ........ ........ 23% 208M 2s 61440K ........ ........ ........ ........ ........ ........ 24% 265M 1s 64512K ........ ........ ........ ........ ........ ........ 25% 248M 1s 67584K ........ ........ ........ ........ ........ ........ 26% 235M 1s 70656K ........ ........ ........ ........ ........ ........ 28% 228M 1s 73728K ........ ........ ........ ........ ........ ........ 29% 252M 1s 76800K ........ ........ ........ ........ ........ ........ 30% 222M 1s 79872K ........ ........ ........ ........ ........ ........ 31% 287M 1s 82944K ........ ........ ........ ........ ........ ........ 32% 266M 1s 86016K ........ ........ ........ ........ ........ ........ 34% 225M 1s 89088K ........ ........ ........ ........ ........ ........ 35% 293M 1s 92160K ........ ........ ........ ........ ........ ........ 36% 275M 1s 95232K ........ ........ ........ ........ ........ ........ 37% 319M 1s 98304K ........ ........ ........ ........ ........ ........ 38% 269M 1s 101376K ........ ........ ........ ........ ........ ........ 39% 323M 1s 104448K ........ ........ ........ ........ ........ ........ 41% 324M 1s 107520K ........ ........ ........ ........ ........ ........ 42% 332M 1s 110592K ........ ........ ........ ........ ........ ........ 43% 267M 1s 113664K ........ ........ ........ ........ ........ ........ 44% 317M 1s 116736K ........ ........ ........ ........ ........ ........ 45% 267M 1s 119808K ........ ........ ........ ........ ........ ........ 46% 174M 1s 122880K ........ ........ ........ ........ ........ ........ 48% 158M 1s 125952K ........ ........ ........ ........ ........ ........ 49% 115M 1s 129024K ........ ........ ........ ........ ........ ........ 50% 282M 1s 132096K ........ ........ ........ ........ ........ ........ 51% 345M 1s 135168K ........ ........ ........ ........ ........ ........ 52% 275M 1s 138240K ........ ........ ........ ........ ........ ........ 53% 169M 1s 141312K ........ ........ ........ ........ ........ ........ 55% 239M 1s 144384K ........ ........ ........ ........ ........ ........ 56% 317M 1s 147456K ........ ........ ........ ........ ........ ........ 57% 282M 1s 150528K ........ ........ ........ ........ ........ ........ 58% 272M 1s 153600K ........ ........ ........ ........ ........ ........ 59% 285M 1s 156672K ........ ........ ........ ........ ........ ........ 60% 261M 1s 159744K ........ ........ ........ ........ ........ ........ 62% 270M 1s 162816K ........ ........ ........ ........ ........ ........ 63% 243M 1s 165888K ........ ........ ........ ........ ........ ........ 64% 270M 0s 168960K ........ ........ ........ ........ ........ ........ 65% 248M 0s 172032K ........ ........ ........ ........ ........ ........ 66% 261M 0s 175104K ........ ........ ........ ........ ........ ........ 68% 248M 0s 178176K ........ ........ ........ ........ ........ ........ 69% 257M 0s 181248K ........ ........ ........ ........ ........ ........ 70% 225M 0s 184320K ........ ........ ........ ........ ........ ........ 71% 273M 0s 187392K ........ ........ ........ ........ ........ ........ 72% 237M 0s 190464K ........ ........ ........ ........ ........ ........ 73% 250M 0s 193536K ........ ........ ........ ........ ........ ........ 75% 242M 0s 196608K ........ ........ ........ ........ ........ ........ 76% 199M 0s 199680K ........ ........ ........ ........ ........ ........ 77% 289M 0s 202752K ........ ........ ........ ........ ........ ........ 78% 218M 0s 205824K ........ ........ ........ ........ ........ ........ 79% 209M 0s 208896K ........ ........ ........ ........ ........ ........ 80% 237M 0s 211968K ........ ........ ........ ........ ........ ........ 82% 208M 0s 215040K ........ ........ ........ ........ ........ ........ 83% 219M 0s 218112K ........ ........ ........ ........ ........ ........ 84% 212M 0s 221184K ........ ........ ........ ........ ........ ........ 85% 222M 0s 224256K ........ ........ ........ ........ ........ ........ 86% 272M 0s 227328K ........ ........ ........ ........ ........ ........ 87% 221M 0s 230400K ........ ........ ........ ........ ........ ........ 89% 227M 0s 233472K ........ ........ ........ ........ ........ ........ 90% 217M 0s 236544K ........ ........ ........ ........ ........ ........ 91% 205M 0s 239616K ........ ........ ........ ........ ........ ........ 92% 259M 0s 242688K ........ ........ ........ ........ ........ ........ 93% 201M 0s 245760K ........ ........ ........ ........ ........ ........ 95% 226M 0s 248832K ........ ........ ........ ........ ........ ........ 96% 236M 0s 251904K ........ ........ ........ ........ ........ ........ 97% 238M 0s 254976K ........ ........ ........ ........ ........ ........ 98% 234M 0s 258048K ........ ........ ........ ........ ........ ........ 99% 222M 0s 261120K ........ .... 100% 206M=1.3s 2024-07-07 00:27:26 (200 MB/s) - ‘karaf-0.20.2-20240706.204634-46.zip’ saved [268186792/268186792] Extracting the new controller... + echo 'Extracting the new controller...' + unzip -q karaf-0.20.2-20240706.204634-46.zip Adding external repositories... + echo 'Adding external repositories...' + sed -ie 's%org.ops4j.pax.url.mvn.repositories=%org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases%g' /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg + cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ################################################################################ # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # ################################################################################ # # If set to true, the following property will not allow any certificate to be used # when accessing Maven repositories through SSL # #org.ops4j.pax.url.mvn.certificateCheck= # # Path to the local Maven settings file. # The repositories defined in this file will be automatically added to the list # of default repositories if the 'org.ops4j.pax.url.mvn.repositories' property # below is not set. # The following locations are checked for the existence of the settings.xml file # * 1. looks for the specified url # * 2. if not found looks for ${user.home}/.m2/settings.xml # * 3. if not found looks for ${maven.home}/conf/settings.xml # * 4. if not found looks for ${M2_HOME}/conf/settings.xml # #org.ops4j.pax.url.mvn.settings= # # Path to the local Maven repository which is used to avoid downloading # artifacts when they already exist locally. # The value of this property will be extracted from the settings.xml file # above, or defaulted to: # System.getProperty( "user.home" ) + "/.m2/repository" # org.ops4j.pax.url.mvn.localRepository=${karaf.home}/${karaf.default.repository} # # Default this to false. It's just weird to use undocumented repos # org.ops4j.pax.url.mvn.useFallbackRepositories=false # # Uncomment if you don't wanna use the proxy settings # from the Maven conf/settings.xml file # # org.ops4j.pax.url.mvn.proxySupport=false # # Comma separated list of repositories scanned when resolving an artifact. # Those repositories will be checked before iterating through the # below list of repositories and even before the local repository # A repository url can be appended with zero or more of the following flags: # @snapshots : the repository contains snaphots # @noreleases : the repository does not contain any released artifacts # # The following property value will add the system folder as a repo. # org.ops4j.pax.url.mvn.defaultRepositories=\ file:${karaf.home}/${karaf.default.repository}@id=system.repository@snapshots,\ file:${karaf.data}/kar@id=kar.repository@multi@snapshots,\ file:${karaf.base}/${karaf.default.repository}@id=child.system.repository@snapshots # Use the default local repo (e.g.~/.m2/repository) as a "remote" repo #org.ops4j.pax.url.mvn.defaultLocalRepoAsRemote=false # # Comma separated list of repositories scanned when resolving an artifact. # The default list includes the following repositories: # http://repo1.maven.org/maven2@id=central # http://repository.springsource.com/maven/bundles/release@id=spring.ebr # http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external # http://zodiac.springsource.com/maven/bundles/release@id=gemini # http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases # https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases # https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases # To add repositories to the default ones, prepend '+' to the list of repositories # to add. # A repository url can be appended with zero or more of the following flags: # @snapshots : the repository contains snapshots # @noreleases : the repository does not contain any released artifacts # @id=repository.id : the id for the repository, just like in the settings.xml this is optional but recommended # org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases ### ^^^ No remote repositories. This is the only ODL change compared to Karaf defaults.+ [[ True == \T\r\u\e ]] Configuring the startup features... + echo 'Configuring the startup features...' + sed -ie 's/\(featuresBoot=\|featuresBoot =\)/featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app,/g' /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg + FEATURE_TEST_STRING=features-test + FEATURE_TEST_VERSION=0.20.2-SNAPSHOT + KARAF_VERSION=karaf4 + [[ integration == \i\n\t\e\g\r\a\t\i\o\n ]] + sed -ie 's%\(featuresRepositories=\|featuresRepositories =\)%featuresRepositories = mvn:org.opendaylight.integration/features-test/0.20.2-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features,%g' /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg + [[ ! -z '' ]] + cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg ################################################################################ # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # ################################################################################ # # Comma separated list of features repositories to register by default # featuresRepositories = mvn:org.opendaylight.integration/features-test/0.20.2-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features, file:${karaf.etc}/97847919-d091-419f-b580-b699f0fd2fda.xml # # Comma separated list of features to install at startup # featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app, a1334869-d56b-49d8-9c5b-fed53f7d2c4c # # Resource repositories (OBR) that the features resolver can use # to resolve requirements/capabilities # # The format of the resourceRepositories is # resourceRepositories=[xml:url|json:url],... # for Instance: # #resourceRepositories=xml:http://host/path/to/index.xml # or #resourceRepositories=json:http://host/path/to/index.json # # # Defines if the boot features are started in asynchronous mode (in a dedicated thread) # featuresBootAsynchronous=false # # Service requirements enforcement # # By default, the feature resolver checks the service requirements/capabilities of # bundles for new features (xml schema >= 1.3.0) in order to automatically installs # the required bundles. # The following flag can have those values: # - disable: service requirements are completely ignored # - default: service requirements are ignored for old features # - enforce: service requirements are always verified # #serviceRequirements=default # # Store cfg file for config element in feature # #configCfgStore=true # # Define if the feature service automatically refresh bundles # autoRefresh=true # # Configuration of features processing mechanism (overrides, blacklisting, modification of features) # XML file defines instructions related to features processing # versions.properties may declare properties to resolve placeholders in XML file # both files are relative to ${karaf.etc} # #featureProcessing=org.apache.karaf.features.xml #featureProcessingVersions=versions.properties + configure_karaf_log karaf4 '' + local -r karaf_version=karaf4 + local -r controllerdebugmap= + local logapi=log4j + grep log4j2 /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n log4j2.rootLogger.level = INFO #log4j2.rootLogger.type = asyncRoot #log4j2.rootLogger.includeLocation = false log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi log4j2.rootLogger.appenderRef.Console.ref = Console log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF} log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL log4j2.logger.spifly.name = org.apache.aries.spifly log4j2.logger.spifly.level = WARN log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit log4j2.logger.audit.level = INFO log4j2.logger.audit.additivity = false log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile # Console appender not used by default (see log4j2.rootLogger.appenderRefs) log4j2.appender.console.type = Console log4j2.appender.console.name = Console log4j2.appender.console.layout.type = PatternLayout log4j2.appender.console.layout.pattern = ${log4j2.pattern} log4j2.appender.rolling.type = RollingRandomAccessFile log4j2.appender.rolling.name = RollingFile log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i #log4j2.appender.rolling.immediateFlush = false log4j2.appender.rolling.append = true log4j2.appender.rolling.layout.type = PatternLayout log4j2.appender.rolling.layout.pattern = ${log4j2.pattern} log4j2.appender.rolling.policies.type = Policies log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy log4j2.appender.rolling.policies.size.size = 64MB log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy log4j2.appender.rolling.strategy.max = 7 log4j2.appender.audit.type = RollingRandomAccessFile log4j2.appender.audit.name = AuditRollingFile log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i log4j2.appender.audit.append = true log4j2.appender.audit.layout.type = PatternLayout log4j2.appender.audit.layout.pattern = ${log4j2.pattern} log4j2.appender.audit.policies.type = Policies log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy log4j2.appender.audit.policies.size.size = 8MB log4j2.appender.audit.strategy.type = DefaultRolloverStrategy log4j2.appender.audit.strategy.max = 7 log4j2.appender.osgi.type = PaxOsgi log4j2.appender.osgi.name = PaxOsgi log4j2.appender.osgi.filter = * #log4j2.logger.aether.name = shaded.org.eclipse.aether #log4j2.logger.aether.level = TRACE #log4j2.logger.http-headers.name = shaded.org.apache.http.headers #log4j2.logger.http-headers.level = DEBUG #log4j2.logger.maven.name = org.ops4j.pax.url.mvn #log4j2.logger.maven.level = TRACE Configuring the karaf log... karaf_version: karaf4, logapi: log4j2 + logapi=log4j2 + echo 'Configuring the karaf log... karaf_version: karaf4, logapi: log4j2' + '[' log4j2 == log4j2 ']' + sed -ie 's/log4j2.appender.rolling.policies.size.size = 64MB/log4j2.appender.rolling.policies.size.size = 1GB/g' /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg controllerdebugmap: + orgmodule=org.opendaylight.yangtools.yang.parser.repo.YangTextSchemaContextResolver + orgmodule_=org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver + echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN' + echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN' + unset IFS + echo 'controllerdebugmap: ' + '[' -n '' ']' + echo 'cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg' cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg + cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg ################################################################################ # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # ################################################################################ # Common pattern layout for appenders log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n # Root logger log4j2.rootLogger.level = INFO # uncomment to use asynchronous loggers, which require mvn:com.lmax/disruptor/3.3.2 library #log4j2.rootLogger.type = asyncRoot #log4j2.rootLogger.includeLocation = false log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi log4j2.rootLogger.appenderRef.Console.ref = Console log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF} # Filters for logs marked by org.opendaylight.odlparent.Markers log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL # Loggers configuration # Spifly logger log4j2.logger.spifly.name = org.apache.aries.spifly log4j2.logger.spifly.level = WARN # Security audit logger log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit log4j2.logger.audit.level = INFO log4j2.logger.audit.additivity = false log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile # Appenders configuration # Console appender not used by default (see log4j2.rootLogger.appenderRefs) log4j2.appender.console.type = Console log4j2.appender.console.name = Console log4j2.appender.console.layout.type = PatternLayout log4j2.appender.console.layout.pattern = ${log4j2.pattern} # Rolling file appender log4j2.appender.rolling.type = RollingRandomAccessFile log4j2.appender.rolling.name = RollingFile log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i # uncomment to not force a disk flush #log4j2.appender.rolling.immediateFlush = false log4j2.appender.rolling.append = true log4j2.appender.rolling.layout.type = PatternLayout log4j2.appender.rolling.layout.pattern = ${log4j2.pattern} log4j2.appender.rolling.policies.type = Policies log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy log4j2.appender.rolling.policies.size.size = 1GB log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy log4j2.appender.rolling.strategy.max = 7 # Audit file appender log4j2.appender.audit.type = RollingRandomAccessFile log4j2.appender.audit.name = AuditRollingFile log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i log4j2.appender.audit.append = true log4j2.appender.audit.layout.type = PatternLayout log4j2.appender.audit.layout.pattern = ${log4j2.pattern} log4j2.appender.audit.policies.type = Policies log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy log4j2.appender.audit.policies.size.size = 8MB log4j2.appender.audit.strategy.type = DefaultRolloverStrategy log4j2.appender.audit.strategy.max = 7 # OSGi appender log4j2.appender.osgi.type = PaxOsgi log4j2.appender.osgi.name = PaxOsgi log4j2.appender.osgi.filter = * # help with identification of maven-related problems with pax-url-aether #log4j2.logger.aether.name = shaded.org.eclipse.aether #log4j2.logger.aether.level = TRACE #log4j2.logger.http-headers.name = shaded.org.apache.http.headers #log4j2.logger.http-headers.level = DEBUG #log4j2.logger.maven.name = org.ops4j.pax.url.mvn #log4j2.logger.maven.level = TRACE log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN + set_java_vars /usr/lib/jvm/java-17-openjdk 3072m /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv + local -r java_home=/usr/lib/jvm/java-17-openjdk + local -r controllermem=3072m + local -r memconf=/tmp/karaf-0.20.2-SNAPSHOT/bin/setenv Configure java home: /usr/lib/jvm/java-17-openjdk max memory: 3072m memconf: /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv + echo Configure + echo ' java home: /usr/lib/jvm/java-17-openjdk' + echo ' max memory: 3072m' + echo ' memconf: /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv' + sed -ie 's%^# export JAVA_HOME%export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-17-openjdk}%g' /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv + sed -ie 's/JAVA_MAX_MEM="2048m"/JAVA_MAX_MEM=3072m/g' /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv cat /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv + echo 'cat /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv' + cat /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv #!/bin/sh # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # # # handle specific scripts; the SCRIPT_NAME is exactly the name of the Karaf # script: client, instance, shell, start, status, stop, karaf # # if [ "${KARAF_SCRIPT}" == "SCRIPT_NAME" ]; then # Actions go here... # fi # # general settings which should be applied for all scripts go here; please keep # in mind that it is possible that scripts might be executed more than once, e.g. # in example of the start script where the start script is executed first and the # karaf script afterwards. # # # The following section shows the possible configuration options for the default # karaf scripts # export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-17-openjdk} # Location of Java installation # export JAVA_OPTS # Generic JVM options, for instance, where you can pass the memory configuration # export JAVA_NON_DEBUG_OPTS # Additional non-debug JVM options # export EXTRA_JAVA_OPTS # Additional JVM options # export KARAF_HOME # Karaf home folder # export KARAF_DATA # Karaf data folder # export KARAF_BASE # Karaf base folder # export KARAF_ETC # Karaf etc folder # export KARAF_LOG # Karaf log folder # export KARAF_SYSTEM_OPTS # First citizen Karaf options # export KARAF_OPTS # Additional available Karaf options # export KARAF_DEBUG # Enable debug mode # export KARAF_REDIRECT # Enable/set the std/err redirection when using bin/start # export KARAF_NOROOT # Prevent execution as root if set to true Set Java version + echo 'Set Java version' + sudo /usr/sbin/alternatives --install /usr/bin/java java /usr/lib/jvm/java-17-openjdk/bin/java 1 + sudo /usr/sbin/alternatives --set java /usr/lib/jvm/java-17-openjdk/bin/java JDK default version ... + echo 'JDK default version ...' + java -version openjdk version "17.0.6-ea" 2023-01-17 LTS OpenJDK Runtime Environment (Red_Hat-17.0.6.0.9-0.3.ea.el8) (build 17.0.6-ea+9-LTS) OpenJDK 64-Bit Server VM (Red_Hat-17.0.6.0.9-0.3.ea.el8) (build 17.0.6-ea+9-LTS, mixed mode, sharing) Set JAVA_HOME + echo 'Set JAVA_HOME' + export JAVA_HOME=/usr/lib/jvm/java-17-openjdk + JAVA_HOME=/usr/lib/jvm/java-17-openjdk ++ readlink -e /usr/lib/jvm/java-17-openjdk/bin/java Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin/java Listing all open ports on controller system... + JAVA_RESOLVED=/usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin/java + echo 'Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin/java' + echo 'Listing all open ports on controller system...' + netstat -pnatu (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN - tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN - tcp 0 164 10.30.171.170:22 10.30.171.184:56988 ESTABLISHED - tcp 0 0 10.30.171.170:33076 199.204.45.87:443 TIME_WAIT - tcp6 0 0 :::111 :::* LISTEN - tcp6 0 0 :::5555 :::* LISTEN - tcp6 0 0 :::22 :::* LISTEN - udp 0 0 10.30.171.170:68 10.30.170.2:67 ESTABLISHED - udp 0 0 0.0.0.0:111 0.0.0.0:* - udp 0 0 127.0.0.1:323 0.0.0.0:* - udp6 0 0 :::111 :::* - udp6 0 0 ::1:323 :::* - Custom shard config exists!!! Copying the shard config... + '[' -f /tmp/custom_shard_config.txt ']' + echo 'Custom shard config exists!!!' + echo 'Copying the shard config...' + cp /tmp/custom_shard_config.txt /tmp/karaf-0.20.2-SNAPSHOT/bin/ Configuring cluster + echo 'Configuring cluster' + /tmp/karaf-0.20.2-SNAPSHOT/bin/configure_cluster.sh 2 10.30.171.165 10.30.171.170 10.30.170.8 ################################################ ## Configure Cluster ## ################################################ NOTE: Cluster configuration files not found. Copying from /tmp/karaf-0.20.2-SNAPSHOT/system/org/opendaylight/controller/sal-clustering-config/9.0.4 Configuring unique name in akka.conf Configuring hostname in akka.conf Configuring data and rpc seed nodes in akka.conf modules = [ { name = "inventory" namespace = "urn:opendaylight:inventory" shard-strategy = "module" }, { name = "topology" namespace = "urn:TBD:params:xml:ns:yang:network-topology" shard-strategy = "module" }, { name = "toaster" namespace = "http://netconfcentral.org/ns/toaster" shard-strategy = "module" }, { name = "car" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car" shard-strategy = "module" }, { name = "people" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people" shard-strategy = "module" }, { name = "car-people" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people" shard-strategy = "module" } ] Configuring replication type in module-shards.conf ################################################ ## NOTE: Manually restart controller to ## ## apply configuration. ## ################################################ Dump akka.conf + echo 'Dump akka.conf' + cat /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf odl-cluster-data { akka { remote { artery { enabled = on transport = tcp canonical.hostname = "10.30.171.170" canonical.port = 2550 } } cluster { # Using artery. seed-nodes = ["akka://opendaylight-cluster-data@10.30.171.165:2550", "akka://opendaylight-cluster-data@10.30.171.170:2550", "akka://opendaylight-cluster-data@10.30.170.8:2550"] roles = ["member-2"] # when under load we might trip a false positive on the failure detector # failure-detector { # heartbeat-interval = 4 s # acceptable-heartbeat-pause = 16s # } } persistence { # By default the snapshots/journal directories live in KARAF_HOME. You can choose to put it somewhere else by # modifying the following two properties. The directory location specified may be a relative or absolute path. # The relative path is always relative to KARAF_HOME. # snapshot-store.local.dir = "target/snapshots" # Use lz4 compression for LocalSnapshotStore snapshots snapshot-store.local.use-lz4-compression = false # Size of blocks for lz4 compression: 64KB, 256KB, 1MB or 4MB snapshot-store.local.lz4-blocksize = 256KB } disable-default-actor-system-quarantined-event-handling = "false" } } Dump modules.conf + echo 'Dump modules.conf' + cat /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf modules = [ { name = "inventory" namespace = "urn:opendaylight:inventory" shard-strategy = "module" }, { name = "topology" namespace = "urn:TBD:params:xml:ns:yang:network-topology" shard-strategy = "module" }, { name = "toaster" namespace = "http://netconfcentral.org/ns/toaster" shard-strategy = "module" }, { name = "car" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car" shard-strategy = "module" }, { name = "people" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people" shard-strategy = "module" }, { name = "car-people" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people" shard-strategy = "module" } ] Dump module-shards.conf + echo 'Dump module-shards.conf' + cat /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf module-shards = [ { name = "default" shards = [ { name = "default" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "inventory" shards = [ { name="inventory" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "topology" shards = [ { name="topology" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "toaster" shards = [ { name="toaster" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "car" shards = [ { name="car" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "people" shards = [ { name="people" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "car-people" shards = [ { name="car-people" replicas = ["member-1", "member-2", "member-3"] } ] } ] Configuring member-3 with IP address 10.30.170.8 Warning: Permanently added '10.30.170.8' (ECDSA) to the list of known hosts. Warning: Permanently added '10.30.170.8' (ECDSA) to the list of known hosts. + source /tmp/common-functions.sh karaf-0.20.2-SNAPSHOT common-functions.sh is being sourced ++ [[ /tmp/common-functions.sh == \/\t\m\p\/\c\o\n\f\i\g\u\r\a\t\i\o\n\-\s\c\r\i\p\t\.\s\h ]] ++ echo 'common-functions.sh is being sourced' ++ BUNDLEFOLDER=karaf-0.20.2-SNAPSHOT ++ export MAVENCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ++ MAVENCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ++ export FEATURESCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg ++ FEATURESCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg ++ export CUSTOMPROP=/tmp/karaf-0.20.2-SNAPSHOT/etc/custom.properties ++ CUSTOMPROP=/tmp/karaf-0.20.2-SNAPSHOT/etc/custom.properties ++ export LOGCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg ++ LOGCONF=/tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg ++ export MEMCONF=/tmp/karaf-0.20.2-SNAPSHOT/bin/setenv ++ MEMCONF=/tmp/karaf-0.20.2-SNAPSHOT/bin/setenv ++ export CONTROLLERMEM= ++ CONTROLLERMEM= ++ export AKKACONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf ++ AKKACONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf ++ export MODULESCONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf ++ MODULESCONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf ++ export MODULESHARDSCONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf ++ MODULESHARDSCONF=/tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf ++ print_common_env ++ cat common-functions environment: MAVENCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ACTUALFEATURES: FEATURESCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg CUSTOMPROP: /tmp/karaf-0.20.2-SNAPSHOT/etc/custom.properties LOGCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg MEMCONF: /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv CONTROLLERMEM: AKKACONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf MODULESCONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf MODULESHARDSCONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf SUITES: ++ SSH='ssh -t -t' ++ extra_services_cntl=' dnsmasq.service httpd.service libvirtd.service openvswitch.service ovs-vswitchd.service ovsdb-server.service rabbitmq-server.service ' ++ extra_services_cmp=' libvirtd.service openvswitch.service ovs-vswitchd.service ovsdb-server.service ' Changing to /tmp Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240706.204634-46.zip + echo 'Changing to /tmp' + cd /tmp + echo 'Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240706.204634-46.zip' + wget --progress=dot:mega https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240706.204634-46.zip --2024-07-07 00:27:30-- https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240706.204634-46.zip Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6 Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 268186792 (256M) [application/zip] Saving to: ‘karaf-0.20.2-20240706.204634-46.zip’ 0K ........ ........ ........ ........ ........ ........ 1% 115M 2s 3072K ........ ........ ........ ........ ........ ........ 2% 137M 2s 6144K ........ ........ ........ ........ ........ ........ 3% 134M 2s 9216K ........ ........ ........ ........ ........ ........ 4% 145M 2s 12288K ........ ........ ........ ........ ........ ........ 5% 167M 2s 15360K ........ ........ ........ ........ ........ ........ 7% 153M 2s 18432K ........ ........ ........ ........ ........ ........ 8% 179M 2s 21504K ........ ........ ........ ........ ........ ........ 9% 190M 2s 24576K ........ ........ ........ ........ ........ ........ 10% 200M 1s 27648K ........ ........ ........ ........ ........ ........ 11% 241M 1s 30720K ........ ........ ........ ........ ........ ........ 12% 223M 1s 33792K ........ ........ ........ ........ ........ ........ 14% 240M 1s 36864K ........ ........ ........ ........ ........ ........ 15% 241M 1s 39936K ........ ........ ........ ........ ........ ........ 16% 280M 1s 43008K ........ ........ ........ ........ ........ ........ 17% 278M 1s 46080K ........ ........ ........ ........ ........ ........ 18% 311M 1s 49152K ........ ........ ........ ........ ........ ........ 19% 202M 1s 52224K ........ ........ ........ ........ ........ ........ 21% 350M 1s 55296K ........ ........ ........ ........ ........ ........ 22% 347M 1s 58368K ........ ........ ........ ........ ........ ........ 23% 368M 1s 61440K ........ ........ ........ ........ ........ ........ 24% 365M 1s 64512K ........ ........ ........ ........ ........ ........ 25% 348M 1s 67584K ........ ........ ........ ........ ........ ........ 26% 332M 1s 70656K ........ ........ ........ ........ ........ ........ 28% 351M 1s 73728K ........ ........ ........ ........ ........ ........ 29% 349M 1s 76800K ........ ........ ........ ........ ........ ........ 30% 345M 1s 79872K ........ ........ ........ ........ ........ ........ 31% 346M 1s 82944K ........ ........ ........ ........ ........ ........ 32% 348M 1s 86016K ........ ........ ........ ........ ........ ........ 34% 340M 1s 89088K ........ ........ ........ ........ ........ ........ 35% 186M 1s 92160K ........ ........ ........ ........ ........ ........ 36% 296M 1s 95232K ........ ........ ........ ........ ........ ........ 37% 329M 1s 98304K ........ ........ ........ ........ ........ ........ 38% 363M 1s 101376K ........ ........ ........ ........ ........ ........ 39% 352M 1s 104448K ........ ........ ........ ........ ........ ........ 41% 353M 1s 107520K ........ ........ ........ ........ ........ ........ 42% 345M 1s 110592K ........ ........ ........ ........ ........ ........ 43% 362M 1s 113664K ........ ........ ........ ........ ........ ........ 44% 350M 1s 116736K ........ ........ ........ ........ ........ ........ 45% 366M 1s 119808K ........ ........ ........ ........ ........ ........ 46% 341M 1s 122880K ........ ........ ........ ........ ........ ........ 48% 366M 1s 125952K ........ ........ ........ ........ ........ ........ 49% 357M 1s 129024K ........ ........ ........ ........ ........ ........ 50% 364M 0s 132096K ........ ........ ........ ........ ........ ........ 51% 351M 0s 135168K ........ ........ ........ ........ ........ ........ 52% 367M 0s 138240K ........ ........ ........ ........ ........ ........ 53% 378M 0s 141312K ........ ........ ........ ........ ........ ........ 55% 360M 0s 144384K ........ ........ ........ ........ ........ ........ 56% 372M 0s 147456K ........ ........ ........ ........ ........ ........ 57% 368M 0s 150528K ........ ........ ........ ........ ........ ........ 58% 204M 0s 153600K ........ ........ ........ ........ ........ ........ 59% 349M 0s 156672K ........ ........ ........ ........ ........ ........ 60% 389M 0s 159744K ........ ........ ........ ........ ........ ........ 62% 402M 0s 162816K ........ ........ ........ ........ ........ ........ 63% 394M 0s 165888K ........ ........ ........ ........ ........ ........ 64% 400M 0s 168960K ........ ........ ........ ........ ........ ........ 65% 360M 0s 172032K ........ ........ ........ ........ ........ ........ 66% 368M 0s 175104K ........ ........ ........ ........ ........ ........ 68% 345M 0s 178176K ........ ........ ........ ........ ........ ........ 69% 240M 0s 181248K ........ ........ ........ ........ ........ ........ 70% 304M 0s 184320K ........ ........ ........ ........ ........ ........ 71% 318M 0s 187392K ........ ........ ........ ........ ........ ........ 72% 319M 0s 190464K ........ ........ ........ ........ ........ ........ 73% 297M 0s 193536K ........ ........ ........ ........ ........ ........ 75% 267M 0s 196608K ........ ........ ........ ........ ........ ........ 76% 319M 0s 199680K ........ ........ ........ ........ ........ ........ 77% 307M 0s 202752K ........ ........ ........ ........ ........ ........ 78% 282M 0s 205824K ........ ........ ........ ........ ........ ........ 79% 291M 0s 208896K ........ ........ ........ ........ ........ ........ 80% 198M 0s 211968K ........ ........ ........ ........ ........ ........ 82% 336M 0s 215040K ........ ........ ........ ........ ........ ........ 83% 198M 0s 218112K ........ ........ ........ ........ ........ ........ 84% 305M 0s 221184K ........ ........ ........ ........ ........ ........ 85% 122M 0s 224256K ........ ........ ........ ........ ........ ........ 86% 199M 0s 227328K ........ ........ ........ ........ ........ ........ 87% 192M 0s 230400K ........ ........ ........ ........ ........ ........ 89% 286M 0s 233472K ........ ........ ........ ........ ........ ........ 90% 296M 0s 236544K ........ ........ ........ ........ ........ ........ 91% 235M 0s 239616K ........ ........ ........ ........ ........ ........ 92% 232M 0s 242688K ........ ........ ........ ........ ........ ........ 93% 168M 0s 245760K ........ ........ ........ ........ ........ ........ 95% 255M 0s 248832K ........ ........ ........ ........ ........ ........ 96% 244M 0s 251904K ........ ........ ........ ........ ........ ........ 97% 259M 0s 254976K ........ ........ ........ ........ ........ ........ 98% 243M 0s 258048K ........ ........ ........ ........ ........ ........ 99% 286M 0s 261120K ........ .... 100% 326M=1.0s 2024-07-07 00:27:31 (265 MB/s) - ‘karaf-0.20.2-20240706.204634-46.zip’ saved [268186792/268186792] + echo 'Extracting the new controller...' Extracting the new controller... + unzip -q karaf-0.20.2-20240706.204634-46.zip Adding external repositories... + echo 'Adding external repositories...' + sed -ie 's%org.ops4j.pax.url.mvn.repositories=%org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases%g' /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg + cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ################################################################################ # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # ################################################################################ # # If set to true, the following property will not allow any certificate to be used # when accessing Maven repositories through SSL # #org.ops4j.pax.url.mvn.certificateCheck= # # Path to the local Maven settings file. # The repositories defined in this file will be automatically added to the list # of default repositories if the 'org.ops4j.pax.url.mvn.repositories' property # below is not set. # The following locations are checked for the existence of the settings.xml file # * 1. looks for the specified url # * 2. if not found looks for ${user.home}/.m2/settings.xml # * 3. if not found looks for ${maven.home}/conf/settings.xml # * 4. if not found looks for ${M2_HOME}/conf/settings.xml # #org.ops4j.pax.url.mvn.settings= # # Path to the local Maven repository which is used to avoid downloading # artifacts when they already exist locally. # The value of this property will be extracted from the settings.xml file # above, or defaulted to: # System.getProperty( "user.home" ) + "/.m2/repository" # org.ops4j.pax.url.mvn.localRepository=${karaf.home}/${karaf.default.repository} # # Default this to false. It's just weird to use undocumented repos # org.ops4j.pax.url.mvn.useFallbackRepositories=false # # Uncomment if you don't wanna use the proxy settings # from the Maven conf/settings.xml file # # org.ops4j.pax.url.mvn.proxySupport=false # # Comma separated list of repositories scanned when resolving an artifact. # Those repositories will be checked before iterating through the # below list of repositories and even before the local repository # A repository url can be appended with zero or more of the following flags: # @snapshots : the repository contains snaphots # @noreleases : the repository does not contain any released artifacts # # The following property value will add the system folder as a repo. # org.ops4j.pax.url.mvn.defaultRepositories=\ file:${karaf.home}/${karaf.default.repository}@id=system.repository@snapshots,\ file:${karaf.data}/kar@id=kar.repository@multi@snapshots,\ file:${karaf.base}/${karaf.default.repository}@id=child.system.repository@snapshots # Use the default local repo (e.g.~/.m2/repository) as a "remote" repo #org.ops4j.pax.url.mvn.defaultLocalRepoAsRemote=false # # Comma separated list of repositories scanned when resolving an artifact. # The default list includes the following repositories: # http://repo1.maven.org/maven2@id=central # http://repository.springsource.com/maven/bundles/release@id=spring.ebr # http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external # http://zodiac.springsource.com/maven/bundles/release@id=gemini # http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases # https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases # https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases # To add repositories to the default ones, prepend '+' to the list of repositories # to add. # A repository url can be appended with zero or more of the following flags: # @snapshots : the repository contains snapshots # @noreleases : the repository does not contain any released artifacts # @id=repository.id : the id for the repository, just like in the settings.xml this is optional but recommended # org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases ### ^^^ No remote repositories. This is the only ODL change compared to Karaf defaults.+ [[ True == \T\r\u\e ]] Configuring the startup features... + echo 'Configuring the startup features...' + sed -ie 's/\(featuresBoot=\|featuresBoot =\)/featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app,/g' /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg + FEATURE_TEST_STRING=features-test + FEATURE_TEST_VERSION=0.20.2-SNAPSHOT + KARAF_VERSION=karaf4 + [[ integration == \i\n\t\e\g\r\a\t\i\o\n ]] + sed -ie 's%\(featuresRepositories=\|featuresRepositories =\)%featuresRepositories = mvn:org.opendaylight.integration/features-test/0.20.2-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features,%g' /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg + [[ ! -z '' ]] + cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg ################################################################################ # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # ################################################################################ # # Comma separated list of features repositories to register by default # featuresRepositories = mvn:org.opendaylight.integration/features-test/0.20.2-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features, file:${karaf.etc}/97847919-d091-419f-b580-b699f0fd2fda.xml # # Comma separated list of features to install at startup # featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app, a1334869-d56b-49d8-9c5b-fed53f7d2c4c # # Resource repositories (OBR) that the features resolver can use # to resolve requirements/capabilities # # The format of the resourceRepositories is # resourceRepositories=[xml:url|json:url],... # for Instance: # #resourceRepositories=xml:http://host/path/to/index.xml # or #resourceRepositories=json:http://host/path/to/index.json # # # Defines if the boot features are started in asynchronous mode (in a dedicated thread) # featuresBootAsynchronous=false # # Service requirements enforcement # # By default, the feature resolver checks the service requirements/capabilities of # bundles for new features (xml schema >= 1.3.0) in order to automatically installs # the required bundles. # The following flag can have those values: # - disable: service requirements are completely ignored # - default: service requirements are ignored for old features # - enforce: service requirements are always verified # #serviceRequirements=default # # Store cfg file for config element in feature # #configCfgStore=true # # Define if the feature service automatically refresh bundles # autoRefresh=true # # Configuration of features processing mechanism (overrides, blacklisting, modification of features) # XML file defines instructions related to features processing # versions.properties may declare properties to resolve placeholders in XML file # both files are relative to ${karaf.etc} # #featureProcessing=org.apache.karaf.features.xml #featureProcessingVersions=versions.properties + configure_karaf_log karaf4 '' + local -r karaf_version=karaf4 + local -r controllerdebugmap= + local logapi=log4j + grep log4j2 /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n log4j2.rootLogger.level = INFO #log4j2.rootLogger.type = asyncRoot #log4j2.rootLogger.includeLocation = false log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi log4j2.rootLogger.appenderRef.Console.ref = Console log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF} log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL log4j2.logger.spifly.name = org.apache.aries.spifly log4j2.logger.spifly.level = WARN log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit log4j2.logger.audit.level = INFO log4j2.logger.audit.additivity = false log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile # Console appender not used by default (see log4j2.rootLogger.appenderRefs) log4j2.appender.console.type = Console log4j2.appender.console.name = Console log4j2.appender.console.layout.type = PatternLayout log4j2.appender.console.layout.pattern = ${log4j2.pattern} log4j2.appender.rolling.type = RollingRandomAccessFile log4j2.appender.rolling.name = RollingFile log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i #log4j2.appender.rolling.immediateFlush = false log4j2.appender.rolling.append = true log4j2.appender.rolling.layout.type = PatternLayout log4j2.appender.rolling.layout.pattern = ${log4j2.pattern} log4j2.appender.rolling.policies.type = Policies log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy log4j2.appender.rolling.policies.size.size = 64MB log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy log4j2.appender.rolling.strategy.max = 7 log4j2.appender.audit.type = RollingRandomAccessFile log4j2.appender.audit.name = AuditRollingFile log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i log4j2.appender.audit.append = true log4j2.appender.audit.layout.type = PatternLayout log4j2.appender.audit.layout.pattern = ${log4j2.pattern} log4j2.appender.audit.policies.type = Policies log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy log4j2.appender.audit.policies.size.size = 8MB log4j2.appender.audit.strategy.type = DefaultRolloverStrategy log4j2.appender.audit.strategy.max = 7 log4j2.appender.osgi.type = PaxOsgi log4j2.appender.osgi.name = PaxOsgi log4j2.appender.osgi.filter = * #log4j2.logger.aether.name = shaded.org.eclipse.aether #log4j2.logger.aether.level = TRACE #log4j2.logger.http-headers.name = shaded.org.apache.http.headers #log4j2.logger.http-headers.level = DEBUG #log4j2.logger.maven.name = org.ops4j.pax.url.mvn #log4j2.logger.maven.level = TRACE Configuring the karaf log... karaf_version: karaf4, logapi: log4j2 + logapi=log4j2 + echo 'Configuring the karaf log... karaf_version: karaf4, logapi: log4j2' + '[' log4j2 == log4j2 ']' + sed -ie 's/log4j2.appender.rolling.policies.size.size = 64MB/log4j2.appender.rolling.policies.size.size = 1GB/g' /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg + orgmodule=org.opendaylight.yangtools.yang.parser.repo.YangTextSchemaContextResolver + orgmodule_=org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver + echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN' controllerdebugmap: + echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN' + unset IFS + echo 'controllerdebugmap: ' + '[' -n '' ']' cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg + echo 'cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg' + cat /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg ################################################################################ # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # ################################################################################ # Common pattern layout for appenders log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n # Root logger log4j2.rootLogger.level = INFO # uncomment to use asynchronous loggers, which require mvn:com.lmax/disruptor/3.3.2 library #log4j2.rootLogger.type = asyncRoot #log4j2.rootLogger.includeLocation = false log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi log4j2.rootLogger.appenderRef.Console.ref = Console log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF} # Filters for logs marked by org.opendaylight.odlparent.Markers log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL # Loggers configuration # Spifly logger log4j2.logger.spifly.name = org.apache.aries.spifly log4j2.logger.spifly.level = WARN # Security audit logger log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit log4j2.logger.audit.level = INFO log4j2.logger.audit.additivity = false log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile # Appenders configuration # Console appender not used by default (see log4j2.rootLogger.appenderRefs) log4j2.appender.console.type = Console log4j2.appender.console.name = Console log4j2.appender.console.layout.type = PatternLayout log4j2.appender.console.layout.pattern = ${log4j2.pattern} # Rolling file appender log4j2.appender.rolling.type = RollingRandomAccessFile log4j2.appender.rolling.name = RollingFile log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i # uncomment to not force a disk flush #log4j2.appender.rolling.immediateFlush = false log4j2.appender.rolling.append = true log4j2.appender.rolling.layout.type = PatternLayout log4j2.appender.rolling.layout.pattern = ${log4j2.pattern} log4j2.appender.rolling.policies.type = Policies log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy log4j2.appender.rolling.policies.size.size = 1GB log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy log4j2.appender.rolling.strategy.max = 7 # Audit file appender log4j2.appender.audit.type = RollingRandomAccessFile log4j2.appender.audit.name = AuditRollingFile log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i log4j2.appender.audit.append = true log4j2.appender.audit.layout.type = PatternLayout log4j2.appender.audit.layout.pattern = ${log4j2.pattern} log4j2.appender.audit.policies.type = Policies log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy log4j2.appender.audit.policies.size.size = 8MB log4j2.appender.audit.strategy.type = DefaultRolloverStrategy log4j2.appender.audit.strategy.max = 7 # OSGi appender log4j2.appender.osgi.type = PaxOsgi log4j2.appender.osgi.name = PaxOsgi log4j2.appender.osgi.filter = * # help with identification of maven-related problems with pax-url-aether #log4j2.logger.aether.name = shaded.org.eclipse.aether #log4j2.logger.aether.level = TRACE #log4j2.logger.http-headers.name = shaded.org.apache.http.headers #log4j2.logger.http-headers.level = DEBUG #log4j2.logger.maven.name = org.ops4j.pax.url.mvn #log4j2.logger.maven.level = TRACE log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN + set_java_vars /usr/lib/jvm/java-17-openjdk 3072m /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv Configure java home: /usr/lib/jvm/java-17-openjdk max memory: 3072m memconf: /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv + local -r java_home=/usr/lib/jvm/java-17-openjdk + local -r controllermem=3072m + local -r memconf=/tmp/karaf-0.20.2-SNAPSHOT/bin/setenv + echo Configure + echo ' java home: /usr/lib/jvm/java-17-openjdk' + echo ' max memory: 3072m' + echo ' memconf: /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv' + sed -ie 's%^# export JAVA_HOME%export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-17-openjdk}%g' /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv + sed -ie 's/JAVA_MAX_MEM="2048m"/JAVA_MAX_MEM=3072m/g' /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv cat /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv + echo 'cat /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv' + cat /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv #!/bin/sh # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # # # handle specific scripts; the SCRIPT_NAME is exactly the name of the Karaf # script: client, instance, shell, start, status, stop, karaf # # if [ "${KARAF_SCRIPT}" == "SCRIPT_NAME" ]; then # Actions go here... # fi # # general settings which should be applied for all scripts go here; please keep # in mind that it is possible that scripts might be executed more than once, e.g. # in example of the start script where the start script is executed first and the # karaf script afterwards. # # # The following section shows the possible configuration options for the default # karaf scripts # export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-17-openjdk} # Location of Java installation # export JAVA_OPTS # Generic JVM options, for instance, where you can pass the memory configuration # export JAVA_NON_DEBUG_OPTS # Additional non-debug JVM options # export EXTRA_JAVA_OPTS # Additional JVM options # export KARAF_HOME # Karaf home folder # export KARAF_DATA # Karaf data folder # export KARAF_BASE # Karaf base folder # export KARAF_ETC # Karaf etc folder # export KARAF_LOG # Karaf log folder # export KARAF_SYSTEM_OPTS # First citizen Karaf options # export KARAF_OPTS # Additional available Karaf options # export KARAF_DEBUG # Enable debug mode # export KARAF_REDIRECT # Enable/set the std/err redirection when using bin/start # export KARAF_NOROOT # Prevent execution as root if set to true Set Java version + echo 'Set Java version' + sudo /usr/sbin/alternatives --install /usr/bin/java java /usr/lib/jvm/java-17-openjdk/bin/java 1 + sudo /usr/sbin/alternatives --set java /usr/lib/jvm/java-17-openjdk/bin/java JDK default version ... + echo 'JDK default version ...' + java -version openjdk version "17.0.6-ea" 2023-01-17 LTS OpenJDK Runtime Environment (Red_Hat-17.0.6.0.9-0.3.ea.el8) (build 17.0.6-ea+9-LTS) OpenJDK 64-Bit Server VM (Red_Hat-17.0.6.0.9-0.3.ea.el8) (build 17.0.6-ea+9-LTS, mixed mode, sharing) Set JAVA_HOME + echo 'Set JAVA_HOME' + export JAVA_HOME=/usr/lib/jvm/java-17-openjdk + JAVA_HOME=/usr/lib/jvm/java-17-openjdk ++ readlink -e /usr/lib/jvm/java-17-openjdk/bin/java Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin/java + JAVA_RESOLVED=/usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin/java + echo 'Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-17-openjdk-17.0.6.0.9-0.3.ea.el8.x86_64/bin/java' Listing all open ports on controller system... + echo 'Listing all open ports on controller system...' + netstat -pnatu (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN - tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN - tcp 0 0 10.30.170.8:22 10.30.171.184:55040 ESTABLISHED - tcp 0 0 10.30.170.8:46348 199.204.45.87:443 TIME_WAIT - tcp6 0 0 :::111 :::* LISTEN - tcp6 0 0 :::5555 :::* LISTEN - tcp6 0 0 :::22 :::* LISTEN - udp 0 0 10.30.170.8:68 10.30.170.3:67 ESTABLISHED - udp 0 0 0.0.0.0:111 0.0.0.0:* - udp 0 0 127.0.0.1:323 0.0.0.0:* - udp6 0 0 :::111 :::* - udp6 0 0 ::1:323 :::* - Custom shard config exists!!! Copying the shard config... + '[' -f /tmp/custom_shard_config.txt ']' + echo 'Custom shard config exists!!!' + echo 'Copying the shard config...' + cp /tmp/custom_shard_config.txt /tmp/karaf-0.20.2-SNAPSHOT/bin/ Configuring cluster + echo 'Configuring cluster' + /tmp/karaf-0.20.2-SNAPSHOT/bin/configure_cluster.sh 3 10.30.171.165 10.30.171.170 10.30.170.8 ################################################ ## Configure Cluster ## ################################################ NOTE: Cluster configuration files not found. Copying from /tmp/karaf-0.20.2-SNAPSHOT/system/org/opendaylight/controller/sal-clustering-config/9.0.4 Configuring unique name in akka.conf Configuring hostname in akka.conf Configuring data and rpc seed nodes in akka.conf modules = [ { name = "inventory" namespace = "urn:opendaylight:inventory" shard-strategy = "module" }, { name = "topology" namespace = "urn:TBD:params:xml:ns:yang:network-topology" shard-strategy = "module" }, { name = "toaster" namespace = "http://netconfcentral.org/ns/toaster" shard-strategy = "module" }, { name = "car" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car" shard-strategy = "module" }, { name = "people" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people" shard-strategy = "module" }, { name = "car-people" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people" shard-strategy = "module" } ] Configuring replication type in module-shards.conf ################################################ ## NOTE: Manually restart controller to ## ## apply configuration. ## ################################################ Dump akka.conf + echo 'Dump akka.conf' + cat /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf odl-cluster-data { akka { remote { artery { enabled = on transport = tcp canonical.hostname = "10.30.170.8" canonical.port = 2550 } } cluster { # Using artery. seed-nodes = ["akka://opendaylight-cluster-data@10.30.171.165:2550", "akka://opendaylight-cluster-data@10.30.171.170:2550", "akka://opendaylight-cluster-data@10.30.170.8:2550"] roles = ["member-3"] # when under load we might trip a false positive on the failure detector # failure-detector { # heartbeat-interval = 4 s # acceptable-heartbeat-pause = 16s # } } persistence { # By default the snapshots/journal directories live in KARAF_HOME. You can choose to put it somewhere else by # modifying the following two properties. The directory location specified may be a relative or absolute path. # The relative path is always relative to KARAF_HOME. # snapshot-store.local.dir = "target/snapshots" # Use lz4 compression for LocalSnapshotStore snapshots snapshot-store.local.use-lz4-compression = false # Size of blocks for lz4 compression: 64KB, 256KB, 1MB or 4MB snapshot-store.local.lz4-blocksize = 256KB } disable-default-actor-system-quarantined-event-handling = "false" } } Dump modules.conf + echo 'Dump modules.conf' + cat /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf modules = [ { name = "inventory" namespace = "urn:opendaylight:inventory" shard-strategy = "module" }, { name = "topology" namespace = "urn:TBD:params:xml:ns:yang:network-topology" shard-strategy = "module" }, { name = "toaster" namespace = "http://netconfcentral.org/ns/toaster" shard-strategy = "module" }, { name = "car" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car" shard-strategy = "module" }, { name = "people" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people" shard-strategy = "module" }, { name = "car-people" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people" shard-strategy = "module" } ] Dump module-shards.conf + echo 'Dump module-shards.conf' + cat /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf module-shards = [ { name = "default" shards = [ { name = "default" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "inventory" shards = [ { name="inventory" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "topology" shards = [ { name="topology" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "toaster" shards = [ { name="toaster" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "car" shards = [ { name="car" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "people" shards = [ { name="people" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "car-people" shards = [ { name="car-people" replicas = ["member-1", "member-2", "member-3"] } ] } ] Locating config plan to use... Finished running config plans Starting member-1 with IP address 10.30.171.165 Warning: Permanently added '10.30.171.165' (ECDSA) to the list of known hosts. Warning: Permanently added '10.30.171.165' (ECDSA) to the list of known hosts. Redirecting karaf console output to karaf_console.log Starting controller... start: Redirecting Karaf output to /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf_console.log Starting member-2 with IP address 10.30.171.170 Warning: Permanently added '10.30.171.170' (ECDSA) to the list of known hosts. Warning: Permanently added '10.30.171.170' (ECDSA) to the list of known hosts. Redirecting karaf console output to karaf_console.log Starting controller... start: Redirecting Karaf output to /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf_console.log Starting member-3 with IP address 10.30.170.8 Warning: Permanently added '10.30.170.8' (ECDSA) to the list of known hosts. Warning: Permanently added '10.30.170.8' (ECDSA) to the list of known hosts. Redirecting karaf console output to karaf_console.log Starting controller... start: Redirecting Karaf output to /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf_console.log [controller-csit-3node-clustering-ask-all-calcium] $ /bin/bash /tmp/jenkins6641519981097055716.sh common-functions.sh is being sourced common-functions environment: MAVENCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ACTUALFEATURES: FEATURESCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.apache.karaf.features.cfg CUSTOMPROP: /tmp/karaf-0.20.2-SNAPSHOT/etc/custom.properties LOGCONF: /tmp/karaf-0.20.2-SNAPSHOT/etc/org.ops4j.pax.logging.cfg MEMCONF: /tmp/karaf-0.20.2-SNAPSHOT/bin/setenv CONTROLLERMEM: 2048m AKKACONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/akka.conf MODULESCONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/modules.conf MODULESHARDSCONF: /tmp/karaf-0.20.2-SNAPSHOT/configuration/initial/module-shards.conf SUITES: + echo '#################################################' ################################################# + echo '## Verify Cluster is UP ##' ## Verify Cluster is UP ## + echo '#################################################' ################################################# + create_post_startup_script + cat + copy_and_run_post_startup_script + seed_index=1 ++ seq 1 3 + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_1_IP + echo 'Execute the post startup script on controller 10.30.171.165' Execute the post startup script on controller 10.30.171.165 + scp /w/workspace/controller-csit-3node-clustering-ask-all-calcium/post-startup-script.sh 10.30.171.165:/tmp/ Warning: Permanently added '10.30.171.165' (ECDSA) to the list of known hosts. + ssh 10.30.171.165 'bash /tmp/post-startup-script.sh 1' Warning: Permanently added '10.30.171.165' (ECDSA) to the list of known hosts. tcp6 0 0 :::8101 :::* LISTEN Waiting up to 3 minutes for controller to come up, checking every 5 seconds... 2024-07-07T00:28:31,190 | INFO | SystemReadyService-0 | SimpleSystemReadyMonitor | 299 - org.opendaylight.infrautils.ready-api - 6.0.9 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.] Controller is UP 2024-07-07T00:28:31,190 | INFO | SystemReadyService-0 | SimpleSystemReadyMonitor | 299 - org.opendaylight.infrautils.ready-api - 6.0.9 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.] Listing all open ports on controller system... (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN - tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN - tcp 0 164 10.30.171.165:22 10.30.171.184:53664 ESTABLISHED - tcp6 0 0 127.0.0.1:1099 :::* LISTEN 5790/java tcp6 0 0 :::111 :::* LISTEN - tcp6 0 0 :::6640 :::* LISTEN 5790/java tcp6 0 0 :::5555 :::* LISTEN - tcp6 0 0 :::8181 :::* LISTEN 5790/java tcp6 0 0 10.30.171.165:2550 :::* LISTEN 5790/java tcp6 0 0 :::22 :::* LISTEN - tcp6 0 0 :::12345 :::* LISTEN 5790/java tcp6 0 0 127.0.0.1:44444 :::* LISTEN 5790/java tcp6 0 0 :::6653 :::* LISTEN 5790/java tcp6 0 0 :::1790 :::* LISTEN 5790/java tcp6 0 0 :::8101 :::* LISTEN 5790/java tcp6 0 0 127.0.0.1:36261 :::* LISTEN 5790/java tcp6 0 0 :::6633 :::* LISTEN 5790/java tcp6 0 0 127.0.0.1:44954 127.0.0.1:1099 TIME_WAIT - tcp6 0 0 10.30.171.165:47772 10.30.170.8:2550 ESTABLISHED 5790/java tcp6 0 0 10.30.171.165:2550 10.30.170.8:38454 ESTABLISHED 5790/java tcp6 0 0 10.30.171.165:58818 10.30.171.170:2550 ESTABLISHED 5790/java tcp6 0 0 10.30.171.165:2550 10.30.171.170:49382 ESTABLISHED 5790/java tcp6 32 0 10.30.171.165:44688 199.204.45.87:443 CLOSE_WAIT 5790/java tcp6 0 0 127.0.0.1:52520 127.0.0.1:44444 TIME_WAIT - tcp6 0 0 10.30.171.165:58806 10.30.171.170:2550 ESTABLISHED 5790/java tcp6 0 0 10.30.171.165:2550 10.30.171.170:49370 ESTABLISHED 5790/java tcp6 0 0 127.0.0.1:52504 127.0.0.1:44444 TIME_WAIT - tcp6 0 0 10.30.171.165:47766 10.30.170.8:2550 ESTABLISHED 5790/java tcp6 0 0 10.30.171.165:2550 10.30.170.8:38458 ESTABLISHED 5790/java udp 0 0 10.30.171.165:68 10.30.170.2:67 ESTABLISHED - udp 0 0 0.0.0.0:111 0.0.0.0:* - udp 0 0 127.0.0.1:323 0.0.0.0:* - udp6 0 0 :::111 :::* - udp6 0 0 ::1:323 :::* - udp6 0 0 :::4342 :::* 5790/java looking for "BindException: Address already in use" in log file looking for "server is unhealthy" in log file + '[' 1 == 0 ']' + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_2_IP + echo 'Execute the post startup script on controller 10.30.171.170' Execute the post startup script on controller 10.30.171.170 + scp /w/workspace/controller-csit-3node-clustering-ask-all-calcium/post-startup-script.sh 10.30.171.170:/tmp/ Warning: Permanently added '10.30.171.170' (ECDSA) to the list of known hosts. + ssh 10.30.171.170 'bash /tmp/post-startup-script.sh 2' Warning: Permanently added '10.30.171.170' (ECDSA) to the list of known hosts. tcp6 0 0 :::8101 :::* LISTEN Waiting up to 3 minutes for controller to come up, checking every 5 seconds... 2024-07-07T00:28:32,415 | INFO | SystemReadyService-0 | SimpleSystemReadyMonitor | 299 - org.opendaylight.infrautils.ready-api - 6.0.9 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.] Controller is UP 2024-07-07T00:28:32,415 | INFO | SystemReadyService-0 | SimpleSystemReadyMonitor | 299 - org.opendaylight.infrautils.ready-api - 6.0.9 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.] Listing all open ports on controller system... (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN - tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN - tcp 0 164 10.30.171.170:22 10.30.171.184:32856 ESTABLISHED - tcp6 0 0 :::8101 :::* LISTEN 5730/java tcp6 0 0 :::6633 :::* LISTEN 5730/java tcp6 0 0 127.0.0.1:1099 :::* LISTEN 5730/java tcp6 0 0 :::111 :::* LISTEN - tcp6 0 0 :::6640 :::* LISTEN 5730/java tcp6 0 0 127.0.0.1:39889 :::* LISTEN 5730/java tcp6 0 0 :::5555 :::* LISTEN - tcp6 0 0 :::8181 :::* LISTEN 5730/java tcp6 0 0 10.30.171.170:2550 :::* LISTEN 5730/java tcp6 0 0 :::22 :::* LISTEN - tcp6 0 0 127.0.0.1:44444 :::* LISTEN 5730/java tcp6 0 0 :::6653 :::* LISTEN 5730/java tcp6 0 0 :::4189 :::* LISTEN 5730/java tcp6 0 0 :::1790 :::* LISTEN 5730/java tcp6 0 0 127.0.0.1:48288 127.0.0.1:1099 TIME_WAIT - tcp6 0 0 127.0.0.1:49578 127.0.0.1:44444 TIME_WAIT - tcp6 0 0 127.0.0.1:49592 127.0.0.1:44444 TIME_WAIT - tcp6 0 0 10.30.171.170:49382 10.30.171.165:2550 ESTABLISHED 5730/java tcp6 0 0 10.30.171.170:49370 10.30.171.165:2550 ESTABLISHED 5730/java tcp6 32 0 10.30.171.170:50486 199.204.45.87:443 CLOSE_WAIT 5730/java tcp6 0 0 10.30.171.170:2550 10.30.171.165:58818 ESTABLISHED 5730/java tcp6 0 0 10.30.171.170:48776 10.30.170.8:2550 ESTABLISHED 5730/java tcp6 0 0 10.30.171.170:48792 10.30.170.8:2550 ESTABLISHED 5730/java tcp6 0 0 10.30.171.170:2550 10.30.170.8:49278 ESTABLISHED 5730/java tcp6 0 0 10.30.171.170:2550 10.30.170.8:49272 ESTABLISHED 5730/java tcp6 0 0 10.30.171.170:2550 10.30.171.165:58806 ESTABLISHED 5730/java udp 0 0 10.30.171.170:68 10.30.170.2:67 ESTABLISHED - udp 0 0 0.0.0.0:111 0.0.0.0:* - udp 0 0 127.0.0.1:323 0.0.0.0:* - udp6 0 0 :::111 :::* - udp6 0 0 ::1:323 :::* - udp6 0 0 :::4342 :::* 5730/java looking for "BindException: Address already in use" in log file looking for "server is unhealthy" in log file + '[' 2 == 0 ']' + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_3_IP + echo 'Execute the post startup script on controller 10.30.170.8' Execute the post startup script on controller 10.30.170.8 + scp /w/workspace/controller-csit-3node-clustering-ask-all-calcium/post-startup-script.sh 10.30.170.8:/tmp/ Warning: Permanently added '10.30.170.8' (ECDSA) to the list of known hosts. + ssh 10.30.170.8 'bash /tmp/post-startup-script.sh 3' Warning: Permanently added '10.30.170.8' (ECDSA) to the list of known hosts. tcp6 0 0 :::8101 :::* LISTEN Waiting up to 3 minutes for controller to come up, checking every 5 seconds... 2024-07-07T00:28:31,913 | INFO | SystemReadyService-0 | SimpleSystemReadyMonitor | 299 - org.opendaylight.infrautils.ready-api - 6.0.9 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.] Controller is UP 2024-07-07T00:28:31,913 | INFO | SystemReadyService-0 | SimpleSystemReadyMonitor | 299 - org.opendaylight.infrautils.ready-api - 6.0.9 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.] Listing all open ports on controller system... (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) Active Internet connections (servers and established) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN - tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN - tcp 0 164 10.30.170.8:22 10.30.171.184:46666 ESTABLISHED - tcp6 0 0 :::111 :::* LISTEN - tcp6 0 0 :::6640 :::* LISTEN 5756/java tcp6 0 0 :::5555 :::* LISTEN - tcp6 0 0 :::8181 :::* LISTEN 5756/java tcp6 0 0 10.30.170.8:2550 :::* LISTEN 5756/java tcp6 0 0 :::22 :::* LISTEN - tcp6 0 0 127.0.0.1:42811 :::* LISTEN 5756/java tcp6 0 0 127.0.0.1:44444 :::* LISTEN 5756/java tcp6 0 0 :::6653 :::* LISTEN 5756/java tcp6 0 0 :::1790 :::* LISTEN 5756/java tcp6 0 0 :::8101 :::* LISTEN 5756/java tcp6 0 0 :::6633 :::* LISTEN 5756/java tcp6 0 0 127.0.0.1:1099 :::* LISTEN 5756/java tcp6 0 0 127.0.0.1:36020 127.0.0.1:44444 TIME_WAIT - tcp6 0 0 127.0.0.1:36012 127.0.0.1:44444 TIME_WAIT - tcp6 0 0 127.0.0.1:45820 127.0.0.1:1099 TIME_WAIT - tcp6 0 0 10.30.170.8:38458 10.30.171.165:2550 ESTABLISHED 5756/java tcp6 0 0 10.30.170.8:49272 10.30.171.170:2550 ESTABLISHED 5756/java tcp6 0 0 10.30.170.8:2550 10.30.171.165:47772 ESTABLISHED 5756/java tcp6 0 0 10.30.170.8:2550 10.30.171.165:47766 ESTABLISHED 5756/java tcp6 0 0 10.30.170.8:49278 10.30.171.170:2550 ESTABLISHED 5756/java tcp6 32 0 10.30.170.8:38686 199.204.45.87:443 CLOSE_WAIT 5756/java tcp6 0 0 10.30.170.8:2550 10.30.171.170:48776 ESTABLISHED 5756/java tcp6 0 0 127.0.0.1:45818 127.0.0.1:1099 TIME_WAIT - tcp6 0 0 10.30.170.8:38454 10.30.171.165:2550 ESTABLISHED 5756/java tcp6 0 0 10.30.170.8:2550 10.30.171.170:48792 ESTABLISHED 5756/java udp 0 0 10.30.170.8:68 10.30.170.3:67 ESTABLISHED - udp 0 0 0.0.0.0:111 0.0.0.0:* - udp 0 0 127.0.0.1:323 0.0.0.0:* - udp6 0 0 :::111 :::* - udp6 0 0 ::1:323 :::* - udp6 0 0 :::4342 :::* 5756/java looking for "BindException: Address already in use" in log file looking for "server is unhealthy" in log file + '[' 0 == 0 ']' + seed_index=1 + dump_controller_threads ++ seq 1 3 + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_1_IP + echo 'Let'\''s take the karaf thread dump' Let's take the karaf thread dump + ssh 10.30.171.165 'sudo ps aux' Warning: Permanently added '10.30.171.165' (ECDSA) to the list of known hosts. ++ tr -s ' ' ++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-ask-all-calcium/ps_before.log ++ cut -f2 '-d ' ++ grep -v grep + pid=5790 + echo 'karaf main: org.apache.karaf.main.Main, pid:5790' karaf main: org.apache.karaf.main.Main, pid:5790 + ssh 10.30.171.165 '/usr/lib/jvm/java-17-openjdk/bin/jstack -l 5790' Warning: Permanently added '10.30.171.165' (ECDSA) to the list of known hosts. + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_2_IP + echo 'Let'\''s take the karaf thread dump' Let's take the karaf thread dump + ssh 10.30.171.170 'sudo ps aux' Warning: Permanently added '10.30.171.170' (ECDSA) to the list of known hosts. ++ cut -f2 '-d ' ++ tr -s ' ' ++ grep -v grep ++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-ask-all-calcium/ps_before.log + pid=5730 + echo 'karaf main: org.apache.karaf.main.Main, pid:5730' karaf main: org.apache.karaf.main.Main, pid:5730 + ssh 10.30.171.170 '/usr/lib/jvm/java-17-openjdk/bin/jstack -l 5730' Warning: Permanently added '10.30.171.170' (ECDSA) to the list of known hosts. + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_3_IP + echo 'Let'\''s take the karaf thread dump' Let's take the karaf thread dump + ssh 10.30.170.8 'sudo ps aux' Warning: Permanently added '10.30.170.8' (ECDSA) to the list of known hosts. ++ tr -s ' ' ++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-ask-all-calcium/ps_before.log ++ grep -v grep ++ cut -f2 '-d ' + pid=5756 + echo 'karaf main: org.apache.karaf.main.Main, pid:5756' karaf main: org.apache.karaf.main.Main, pid:5756 + ssh 10.30.170.8 '/usr/lib/jvm/java-17-openjdk/bin/jstack -l 5756' Warning: Permanently added '10.30.170.8' (ECDSA) to the list of known hosts. + '[' 0 -gt 0 ']' + echo 'Generating controller variables...' Generating controller variables... ++ seq 1 3 + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_1_IP + odl_variables=' -v ODL_SYSTEM_1_IP:10.30.171.165' + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_2_IP + odl_variables=' -v ODL_SYSTEM_1_IP:10.30.171.165 -v ODL_SYSTEM_2_IP:10.30.171.170' + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_3_IP + odl_variables=' -v ODL_SYSTEM_1_IP:10.30.171.165 -v ODL_SYSTEM_2_IP:10.30.171.170 -v ODL_SYSTEM_3_IP:10.30.170.8' + echo 'Generating mininet variables...' Generating mininet variables... ++ seq 1 0 + get_test_suites SUITES + local __suite_list=SUITES + echo 'Locating test plan to use...' Locating test plan to use... + testplan_filepath=/w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/testplans/controller-clustering-ask-calcium.txt + '[' '!' -f /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/testplans/controller-clustering-ask-calcium.txt ']' + testplan_filepath=/w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/testplans/controller-clustering-ask.txt + '[' disabled '!=' disabled ']' + echo 'Changing the testplan path...' Changing the testplan path... + sed s:integration:/w/workspace/controller-csit-3node-clustering-ask-all-calcium: /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/testplans/controller-clustering-ask.txt + cat testplan.txt # Place the suites in run order: /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/action_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/action_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/singleton_service/global_rpc_kill.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/singleton_service/global_rpc_freeze.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/singleton_service/global_rpc_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot + '[' -z '' ']' ++ grep -E -v '(^[[:space:]]*#|^[[:space:]]*$)' testplan.txt ++ tr '\012' ' ' + suite_list='/w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/action_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/action_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/singleton_service/global_rpc_kill.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/singleton_service/global_rpc_freeze.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/singleton_service/global_rpc_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot ' + eval 'SUITES='\''/w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/action_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/action_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/singleton_service/global_rpc_kill.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/singleton_service/global_rpc_freeze.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/singleton_service/global_rpc_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot '\''' ++ SUITES='/w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/action_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/action_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/singleton_service/global_rpc_kill.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/singleton_service/global_rpc_freeze.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/singleton_service/global_rpc_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot ' + echo 'Starting Robot test suites /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/action_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/action_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/singleton_service/global_rpc_kill.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/singleton_service/global_rpc_freeze.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/singleton_service/global_rpc_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot ...' Starting Robot test suites /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/action_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/action_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/singleton_service/global_rpc_kill.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/singleton_service/global_rpc_freeze.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/singleton_service/global_rpc_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot ... + robot -N controller-clustering-ask.txt --removekeywords wuks -e exclude -e skip_if_calcium -v BUNDLEFOLDER:karaf-0.20.2-SNAPSHOT -v BUNDLE_URL:https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.20.2-SNAPSHOT/karaf-0.20.2-20240706.204634-46.zip -v CONTROLLER:10.30.171.165 -v CONTROLLER1:10.30.171.170 -v CONTROLLER2:10.30.170.8 -v CONTROLLER_USER:jenkins -v JAVA_HOME:/usr/lib/jvm/java-17-openjdk -v JDKVERSION:openjdk17 -v JENKINS_WORKSPACE:/w/workspace/controller-csit-3node-clustering-ask-all-calcium -v MININET: -v MININET1: -v MININET2: -v MININET_USER:jenkins -v NEXUSURL_PREFIX:https://nexus.opendaylight.org -v NUM_ODL_SYSTEM:3 -v NUM_TOOLS_SYSTEM:0 -v ODL_STREAM:calcium -v ODL_SYSTEM_IP:10.30.171.165 -v ODL_SYSTEM_1_IP:10.30.171.165 -v ODL_SYSTEM_2_IP:10.30.171.170 -v ODL_SYSTEM_3_IP:10.30.170.8 -v ODL_SYSTEM_USER:jenkins -v TOOLS_SYSTEM_IP: -v TOOLS_SYSTEM_USER:jenkins -v USER_HOME:/home/jenkins -v IS_KARAF_APPL:True -v WORKSPACE:/tmp /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/action_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_rpc_broker/action_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/singleton_service/global_rpc_kill.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/singleton_service/global_rpc_freeze.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/singleton_service/global_rpc_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-calcium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot ============================================================================== controller-clustering-ask.txt ============================================================================== controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te... ============================================================================== Stop_All_Members :: Stop every odl node. | FAIL | Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1 ------------------------------------------------------------------------------ Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS | ------------------------------------------------------------------------------ Start_All_And_Sync :: Start each member and wait for sync. | FAIL | Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== controller-clustering-ask.txt.Rpc Provider Precedence :: DOMRpcBroker testi... ============================================================================== Register_Rpc_On_Each_Node :: Register global rpc on each node of t... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_rpc_provider_precedence_register_rpc_on_each_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_rpc_provider_precedence_register_rpc_on_each_node&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Invoke_Rpc_On_Each_Node :: Verify that the rpc response comes from... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_rpc_provider_precedence_invoke_rpc_on_each_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_rpc_provider_precedence_invoke_rpc_on_each_node&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Unregister_Rpc_On_Node :: Unregister the rpc on one of the cluster... | PASS | ------------------------------------------------------------------------------ Invoke_Rpc_On_Node_With_Unregistered_Rpc :: Invoke rcp on the node... | PASS | ------------------------------------------------------------------------------ Invoke_Rpc_On_Remaining_Nodes :: Verify that the rpc response come... | PASS | ------------------------------------------------------------------------------ Reregister_Rpc_On_Node :: Reregister the rpc. | PASS | ------------------------------------------------------------------------------ Invoke_Rpc_On_Each_Node_Again :: Verify that the rpc response come... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_rpc_provider_precedence_invoke_rpc_on_each_node_again" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_rpc_provider_precedence_invoke_rpc_on_each_node_again&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Unregister_Rpc_On_Each_Node :: Unregister rpc on every node. | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_rpc_provider_precedence_unregister_rpc_on_each_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_rpc_provider_precedence_unregister_rpc_on_each_node&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Rpc Provider Precedence :: DOMRpcBro... | FAIL | 8 tests, 4 passed, 4 failed ============================================================================== controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te... ============================================================================== Stop_All_Members :: Stop every odl node. | FAIL | Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1 ------------------------------------------------------------------------------ Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS | ------------------------------------------------------------------------------ Start_All_And_Sync :: Start each member and wait for sync. | FAIL | Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== controller-clustering-ask.txt.Rpc Provider Partition And Heal :: DOMRpcBrok... ============================================================================== Register_Rpc_On_Two_Nodes :: Register rpc on two nodes of the odl ... | PASS | ------------------------------------------------------------------------------ Invoke_Rpc_On_Each_Node :: Invoke get-constant rpc on every node o... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_rpc_provider_partition_and_heal_invoke_rpc_on_each_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_rpc_provider_partition_and_heal_invoke_rpc_on_each_node&order=bug_status" Keyword 'Verify_Constant_On_Active_Nodes' failed after retrying for 10 seconds. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Isolate_One_Node :: Isolate one node with registered rpc. From the... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_rpc_provider_partition_and_heal_isolate_one_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_rpc_provider_partition_and_heal_isolate_one_node&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Invoke_Rpc_On_Nonisolated_Nodes :: Invoke rpc on non-islolated nodes. | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_rpc_provider_partition_and_heal_invoke_rpc_on_nonisolated_nodes" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_rpc_provider_partition_and_heal_invoke_rpc_on_nonisolated_nodes&order=bug_status" Keyword 'DrbCommons.Verify_Constant_On_Active_Nodes' failed after retrying for 1 minute. The last error was: Keyword 'Verify_Constant_On_Active_Nodes' failed after retrying for 10 seconds. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Rejoin_Isolated_Member :: Rejoin isolated node | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_rpc_provider_partition_and_heal_rejoin_isolated_member" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_rpc_provider_partition_and_heal_rejoin_isolated_member&order=bug_status" Variable '${isolated_idx}' not found. ------------------------------------------------------------------------------ Invoke_Rpc_On_Each_Node_Again :: Invoke rpc get-constant on every ... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_rpc_provider_partition_and_heal_invoke_rpc_on_each_node_again" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_rpc_provider_partition_and_heal_invoke_rpc_on_each_node_again&order=bug_status" Keyword 'Verify_Constant_On_Active_Nodes' failed after retrying for 10 seconds. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Unregister_Rpc_On_Each_Node :: Inregister rpc on both nodes. | PASS | ------------------------------------------------------------------------------ controller-clustering-ask.txt.Rpc Provider Partition And Heal :: D... | FAIL | 7 tests, 2 passed, 5 failed ============================================================================== controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te... ============================================================================== Stop_All_Members :: Stop every odl node. | FAIL | Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1 ------------------------------------------------------------------------------ Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS | ------------------------------------------------------------------------------ Start_All_And_Sync :: Start each member and wait for sync. | FAIL | Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== controller-clustering-ask.txt.Action Provider Precedence :: DOMRpcBroker te... ============================================================================== Register_Rpc_On_Each_Node :: Register routed rpc on each node of t... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_action_provider_precedence_register_rpc_on_each_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_precedence_register_rpc_on_each_node&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Invoke_Rpc_On_Each_Node :: Verify that the rpc response comes from... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_action_provider_precedence_invoke_rpc_on_each_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_precedence_invoke_rpc_on_each_node&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Unregister_Rpc_On_Node :: Unregister the rpc on one of the cluster... | PASS | ------------------------------------------------------------------------------ Invoke_Rpc_On_Node_With_Unregistered_Rpc :: Invoke rcp on the node... | PASS | ------------------------------------------------------------------------------ Invoke_Rpc_On_Remaining_Nodes :: Verify that the rpc response come... | PASS | ------------------------------------------------------------------------------ Reregister_Rpc_On_Node :: Reregister the rpc. | PASS | ------------------------------------------------------------------------------ Invoke_Rpc_On_Each_Node_Again :: Verify that the rpc response come... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_action_provider_precedence_invoke_rpc_on_each_node_again" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_precedence_invoke_rpc_on_each_node_again&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Unregister_Rpc_On_Each_Node :: Unregister rpc on every node. | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_action_provider_precedence_unregister_rpc_on_each_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_precedence_unregister_rpc_on_each_node&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Action Provider Precedence :: DOMRpc... | FAIL | 8 tests, 4 passed, 4 failed ============================================================================== controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te... ============================================================================== Stop_All_Members :: Stop every odl node. | FAIL | Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1 ------------------------------------------------------------------------------ Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS | ------------------------------------------------------------------------------ Start_All_And_Sync :: Start each member and wait for sync. | FAIL | Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== controller-clustering-ask.txt.Action Provider Partition And Heal :: DOMRpcB... ============================================================================== Register_Rpc_On_Two_Nodes :: Register rpc on two nodes of the odl ... | PASS | ------------------------------------------------------------------------------ Invoke_Rpc_On_Each_Node :: Invoke get-contexted-constant rpc on ev... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_action_provider_partition_and_heal_invoke_rpc_on_each_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_partition_and_heal_invoke_rpc_on_each_node&order=bug_status" Keyword 'Verify_Contexted_Constant_On_Active_Nodes' failed after retrying for 10 seconds. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Isolate_One_Node :: Isolate one node with registered rpc. | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_action_provider_partition_and_heal_isolate_one_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_partition_and_heal_isolate_one_node&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Invoke_Rpc_On_Remaining_Nodes :: Invoke rpc on non-islolated nodes... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_action_provider_partition_and_heal_invoke_rpc_on_remaining_nodes" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_partition_and_heal_invoke_rpc_on_remaining_nodes&order=bug_status" Keyword 'DrbCommons.Verify_Contexted_Constant_On_Active_Nodes' failed after retrying for 1 minute. The last error was: Keyword 'Verify_Contexted_Constant_On_Active_Nodes' failed after retrying for 10 seconds. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Rejoin_Isolated_Member :: Rejoin isolated node | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_action_provider_partition_and_heal_rejoin_isolated_member" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_partition_and_heal_rejoin_isolated_member&order=bug_status" Variable '${isolated_idx}' not found. ------------------------------------------------------------------------------ Invoke_Rpc_On_Each_Node_Again :: Invoke rpc get-contexted-constant... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_action_provider_partition_and_heal_invoke_rpc_on_each_node_again" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_partition_and_heal_invoke_rpc_on_each_node_again&order=bug_status" Keyword 'Verify_Contexted_Constant_On_Active_Nodes' failed after retrying for 10 seconds. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Unregister_Rpc_On_Each_Node :: Inregister rpc on both nodes. | PASS | ------------------------------------------------------------------------------ controller-clustering-ask.txt.Action Provider Partition And Heal :... | FAIL | 7 tests, 2 passed, 5 failed ============================================================================== controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te... ============================================================================== Stop_All_Members :: Stop every odl node. | FAIL | Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1 ------------------------------------------------------------------------------ Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS | ------------------------------------------------------------------------------ Start_All_And_Sync :: Start each member and wait for sync. | FAIL | Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== controller-clustering-ask.txt.Master Stability :: Cluster Singleton testing... ============================================================================== Register_Singleton_Constant_On_Each_Node_And_Verify :: Register a ... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_master_stability_register_singleton_constant_on_each_node_and_verify" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_master_stability_register_singleton_constant_on_each_node_and_verify&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Unregister_Singleton_Constant_On_Non_Master_Node :: Unregister the... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_master_stability_unregister_singleton_constant_on_non_master_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_master_stability_unregister_singleton_constant_on_non_master_node&order=bug_status" Variable '@{cs_candidates}' not found. Did you mean: @{cs_exp_candidates} ------------------------------------------------------------------------------ Monitor_Stability_While_Unregistered :: Verify that the owner rema... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_master_stability_monitor_stability_while_unregistered" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_master_stability_monitor_stability_while_unregistered&order=bug_status" Variable '${cs_owner}' not found. ------------------------------------------------------------------------------ Reregister_Singleton_Constant :: Re-register the unregistered cand... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_master_stability_reregister_singleton_constant" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_master_stability_reregister_singleton_constant&order=bug_status" Variable '${unregistered_node}' not found. ------------------------------------------------------------------------------ Verify_Stability_After_Reregistration :: Verify that the owner rem... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_master_stability_verify_stability_after_reregistration" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_master_stability_verify_stability_after_reregistration&order=bug_status" Variable '${cs_owner}' not found. ------------------------------------------------------------------------------ Unregister_Singleton_Constant_On_Each_Node :: Unregister the appli... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_master_stability_unregister_singleton_constant_on_each_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_master_stability_unregister_singleton_constant_on_each_node&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Master Stability :: Cluster Singleto... | FAIL | 6 tests, 0 passed, 6 failed ============================================================================== controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te... ============================================================================== Stop_All_Members :: Stop every odl node. | FAIL | Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1 ------------------------------------------------------------------------------ Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS | ------------------------------------------------------------------------------ Start_All_And_Sync :: Start each member and wait for sync. | FAIL | Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== controller-clustering-ask.txt.Partition And Heal :: Cluster Singleton testi... ============================================================================== Register_Singleton_Constant_On_Each_Node :: Register a candidate a... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_partition_and_heal_register_singleton_constant_on_each_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_partition_and_heal_register_singleton_constant_on_each_node&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Verify_Singleton_Constant_On_Each_Node :: Store the owner and cand... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_partition_and_heal_verify_singleton_constant_on_each_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_partition_and_heal_verify_singleton_constant_on_each_node&order=bug_status" Could not parse owner and candidates for device get-singleton-constant-service'] ------------------------------------------------------------------------------ Isolate_Owner_Node :: Isolate the cluster node which is the owner.... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_partition_and_heal_isolate_owner_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_partition_and_heal_isolate_owner_node&order=bug_status" Variable '${cs_owner}' not found. ------------------------------------------------------------------------------ Monitor_Stability_While_Isolated :: Monitor the stability of the s... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_partition_and_heal_monitor_stability_while_isolated" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_partition_and_heal_monitor_stability_while_isolated&order=bug_status" Invalid IF condition: Evaluating expression '"${index}" == "${cs_isolated_index}"' failed: Variable '${cs_isolated_index}' not found. ------------------------------------------------------------------------------ Rejoin_Isolated_node :: Rejoin isolated node. | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_partition_and_heal_rejoin_isolated_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_partition_and_heal_rejoin_isolated_node&order=bug_status" Variable '${cs_isolated_index}' not found. ------------------------------------------------------------------------------ Unregister_Singleton_Constant_On_Each_Node :: Unregister the appli... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_partition_and_heal_unregister_singleton_constant_on_each_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_partition_and_heal_unregister_singleton_constant_on_each_node&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Partition And Heal :: Cluster Single... | FAIL | 6 tests, 0 passed, 6 failed ============================================================================== controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te... ============================================================================== Stop_All_Members :: Stop every odl node. | FAIL | Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1 ------------------------------------------------------------------------------ Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS | ------------------------------------------------------------------------------ Start_All_And_Sync :: Start each member and wait for sync. | FAIL | Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== controller-clustering-ask.txt.Chasing The Leader :: Cluster Singleton testi... ============================================================================== Register_Candidates :: Register a candidate application on each no... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_chasing_the_leader_register_candidates" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_chasing_the_leader_register_candidates&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Do_Nothing :: Do nothing for the time of the test duration, becaus... | PASS | ------------------------------------------------------------------------------ Unregister_Candidates_And_Validate_Criteria :: Unregister the test... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_chasing_the_leader_unregister_candidates_and_validate_criteria" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_chasing_the_leader_unregister_candidates_and_validate_criteria&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Chasing The Leader :: Cluster Single... | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te... ============================================================================== Stop_All_Members :: Stop every odl node. | FAIL | Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1 ------------------------------------------------------------------------------ Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS | ------------------------------------------------------------------------------ Start_All_And_Sync :: Start each member and wait for sync. | FAIL | Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== controller-clustering-ask.txt.Global Rpc Kill :: Controller functional HA t... ============================================================================== Get_Basic_Rpc_Test_Owner :: Find a service owner and successors. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_Before_Stopping_On_Owner :: Run rpc on the service owner. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_Before_Stop_On_Successors :: Run rpc on non owher cluster nodes. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Stop_Current_Owner_Member :: Stop cluster node which is the owner. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Verify_New_Basic_Rpc_Test_Owner_Elected :: Verify new owner of the... | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_On_Remained_Cluster_Nodes :: Run rpc on remained cluster nodes. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Restart_Stopped_Member :: Restart stopped node | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Verify_New_Owner_Remained_After_Rejoin :: Verify no owner change h... | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_After_Rejoin_On_New_Owner :: Run rpc on the new service owner ... | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_After_Rejoin_On_Old_Owner :: Run rpc on rejoined cluster node. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_After_Rejoin_On_All :: Run rpc again on all nodes. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ controller-clustering-ask.txt.Global Rpc Kill :: Controller functi... | FAIL | Suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . 11 tests, 0 passed, 11 failed ============================================================================== controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te... ============================================================================== Stop_All_Members :: Stop every odl node. | FAIL | Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1 ------------------------------------------------------------------------------ Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS | ------------------------------------------------------------------------------ Start_All_And_Sync :: Start each member and wait for sync. | FAIL | Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== controller-clustering-ask.txt.Global Rpc Freeze :: Controller functional HA... ============================================================================== Get_Basic_Rpc_Test_Owner :: Find a service owner and successors. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_Before_Freezing_On_Owner :: Run rpc on the service owner. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_Before_Freeze_On_Successors :: Run rpc on non owher cluster no... | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Freeze_Current_Owner_Member :: Stop cluster node which is the owner. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Verify_New_Basic_Rpc_Test_Owner_Elected :: Verify new owner of the... | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_On_Remained_Cluster_Nodes :: Run rpc on remained cluster nodes. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Unfreeze_Frozen_Member :: Restart frozen node | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_After_Rejoin_On_New_Owner :: Run rpc on the new service owner ... | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_After_Rejoin_On_Old_Owner :: Run rpc on rejoined cluster node. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_After_Rejoin_On_All :: Run rpc again on all nodes. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ controller-clustering-ask.txt.Global Rpc Freeze :: Controller func... | FAIL | Suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . 10 tests, 0 passed, 10 failed ============================================================================== controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te... ============================================================================== Stop_All_Members :: Stop every odl node. | FAIL | Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1 ------------------------------------------------------------------------------ Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS | ------------------------------------------------------------------------------ Start_All_And_Sync :: Start each member and wait for sync. | FAIL | Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== controller-clustering-ask.txt.Global Rpc Isolate :: Controller functional H... ============================================================================== Get_Basic_Rpc_Test_Owner :: Find a service owner and successors. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_Before_Isolation_On_Owner :: Run rpc on the service owner. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_Before_Isolation_On_Successors :: Run rpc on non owher cluster... | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Isolate_Current_Owner_Member :: Isolating cluster node which is th... | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Verify_New_Basic_Rpc_Test_Owner_Elected :: Verify new owner of the... | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_On_Isolated_Node :: Run rpc on isolated cluster node. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_On_Non_Isolated_Cluster_Nodes :: Run rpc on remained cluster n... | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rejoin_Isolated_Member :: Rejoin isolated node | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_After_Rejoin_On_New_Owner :: Run rpc on the new service owner ... | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_After_Rejoin_On_Old_Owner :: Run rpc on rejoined cluster node. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_After_Rejoin_On_All :: Run rpc again on all nodes. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ controller-clustering-ask.txt.Global Rpc Isolate :: Controller fun... | FAIL | Suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . 11 tests, 0 passed, 11 failed ============================================================================== controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te... ============================================================================== Stop_All_Members :: Stop every odl node. | FAIL | Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1 ------------------------------------------------------------------------------ Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS | ------------------------------------------------------------------------------ Start_All_And_Sync :: Start each member and wait for sync. | FAIL | Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.170.8', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== controller-clustering-ask.txt.Carpeople Crud :: Suite for performing basic ... ============================================================================== Add_Cars_To_Leader :: Add 30 cars to car Leader by one big PUT. | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Added_Cars_On_Leader :: GET response from Leader should match ... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Added_Cars_On_Followers :: The same check on other members. | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Add_People_To_First_Follower :: Add 30 people to people first Foll... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Added_People_On_Leader :: GET response from Leader should matc... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Added_People_On_Followers :: The same check on other members. | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Buy_Cars_On_Leader :: Buy some cars on car-people Leader, loop of ... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Buy_Cars_On_Followers :: On each Follower buy corresponding ID seg... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Added_CarPeople_On_Leader :: GET car-person mappings from Lead... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Added_CarPeople_On_Followers :: The same check on other members. | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_All_CarPeople_On_Leader :: DELETE car-people container. No ... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_All_People_On_Leader :: DELETE people container. No verific... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_All_Cars_On_Leader :: DELETE cars container. No verificatio... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ controller-clustering-ask.txt.Carpeople Crud :: Suite for performi... | FAIL | Suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig 13 tests, 0 passed, 13 failed ============================================================================== controller-clustering-ask.txt.Car Failover Crud :: Suite mixing basic opera... ============================================================================== Add_Original_Cars_On_Old_Leader_And_Verify :: Add initial cars on ... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Stop_Original_Car_Leader :: Stop the car Leader to cause a new lea... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Wait_For_New_Leader :: Wait until new car Leader is elected. | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Original_Cars_On_New_Leader :: GET cars from new Leader, shoul... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Original_Cars_On_New_Followers :: The same check on other exis... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_Original_Cars_On_New_Leader :: Delete cars on the new Leader. | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Add_Leader_Cars_On_New_Leader :: Add cars on the new Leader. | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Leader_Cars_On_New_Leader :: GET cars from new Leader, should ... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Leader_Cars_On_New_Followers :: The same check on other existi... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_Leader_Cars_On_New_First_Follower :: Delete cars in new fir... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Add_Follower_Cars_On_New_First_Follower :: Add cars on the new fir... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Folower_Cars_On_New_Leader :: Get cars from the new Leader, sh... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Follower_Cars_On_New_Followers :: The same check on other exis... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Start_Old_Car_Leader :: Start the stopped member without deleting ... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Folower_Cars_On_Old_Leader :: GET cars from the restarted memb... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_Follower_Cars_On_New_Leader :: Delete cars on the last Leader. | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ controller-clustering-ask.txt.Car Failover Crud :: Suite mixing ba... | FAIL | Suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig 16 tests, 0 passed, 16 failed ============================================================================== controller-clustering-ask.txt.Car Persistence Recovery :: This test restart... ============================================================================== Add_Cars_On_Leader_And_Verify :: Single big PUT to datastore to ad... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Stop_All_Members :: Stop all controllers. | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Start_All_Members :: Start all controllers (should restore the per... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Memorize_Leader_And_Followers :: Locate current Leader of car Shard. | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Cars_On_Leader :: GET cars from Leader, should match the PUT d... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Cars_On_Followers :: The same check on other members. | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_Cars_On_Leader :: Delete cars on the new Leader. | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ controller-clustering-ask.txt.Car Persistence Recovery :: This tes... | FAIL | Suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.165:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig 7 tests, 0 passed, 7 failed ============================================================================== controller-clustering-ask.txt.Buycar Failover :: This test focuses on testi... ============================================================================== Add_Cars_To_Leader_And_Verify :: Add all needed cars to car Leader... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Add_People_To_First_Follower_And_Verify :: Add all needed people t... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Buy_Cars_On_Leader_And_Verify :: Buy some cars on the leader member. | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Buy_Cars_On_Follower_And_Verify :: Buy some cars on the first foll... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Reboot_People_Leader :: Previous people Leader is rebooted. We sho... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Buy_Cars_On_Leader_After_Reboot_And_Verify :: Buy some cars on the... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Buy_Cars_On_Follower_After_Reboot_And_Verify :: Buy some cars on t... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_All_CarPeople :: DELETE car-people container. No verificati... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_All_People :: DELETE people container. No verification beyo... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_All_Cars :: DELETE cars container. No verification beyond h... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ controller-clustering-ask.txt.Buycar Failover :: This test focuses... | FAIL | Suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig 10 tests, 0 passed, 10 failed ============================================================================== controller-clustering-ask.txt.Entity Isolate :: Suite for performing member... ============================================================================== Check All Shards Before Isolate :: Check all shards in controller. | FAIL | Parent suite setup failed: Evaluating expression 'json.loads(\'\'\'{\n "error": "javax.management.InstanceNotFoundException : org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "error_type": "javax.management.InstanceNotFoundException",\n "request": {\n "mbean": "org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "type": "read"\n },\n "stacktrace": "javax.management.InstanceNotFoundException: org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1088)\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBeanInfo(DefaultMBeanServerInterceptor.java:1358)\\n\\tat java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.getMBeanInfo(JmxMBeanServer.java:921)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:46)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:41)\\n\\tat org.jolokia.backend.executor.AbstractMBeanServerExecutor.call(AbstractMBeanServerExecutor.java:90)\\n\\tat org.jolokia.handler.ReadHandler.getMBeanInfo(ReadHandler.java:233)\\n\\tat org.jolokia.handler.ReadHandler.getAllAttributesNames(ReadHandler.java:245)\\n\\tat org.jolokia.handler.ReadHandler.resolveAttributes(ReadHandler.java:221)\\n\\tat org.jolokia.handler.ReadHandler.f... [ Message content over the limit has been removed. ] ...clipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\\n\\tat org.eclipse.jetty.server.Server.handle(Server.java:516)\\n\\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)\\n\\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)\\n\\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)\\n\\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)\\n\\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)\\n\\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)\\n\\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)\\n\\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)\\n\\tat java.base/java.lang.Thread.run(Thread.java:833)\\n",\n "status": 404\n}\n\'\'\')' failed: JSONDecodeError: Invalid control character at: line 8 column 190 (char 619) ------------------------------------------------------------------------------ Isolate Entity Leader :: Isolate the entity-ownership Leader to ca... | FAIL | Parent suite setup failed: Evaluating expression 'json.loads(\'\'\'{\n "error": "javax.management.InstanceNotFoundException : org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "error_type": "javax.management.InstanceNotFoundException",\n "request": {\n "mbean": "org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "type": "read"\n },\n "stacktrace": "javax.management.InstanceNotFoundException: org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1088)\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBeanInfo(DefaultMBeanServerInterceptor.java:1358)\\n\\tat java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.getMBeanInfo(JmxMBeanServer.java:921)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:46)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:41)\\n\\tat org.jolokia.backend.executor.AbstractMBeanServerExecutor.call(AbstractMBeanServerExecutor.java:90)\\n\\tat org.jolokia.handler.ReadHandler.getMBeanInfo(ReadHandler.java:233)\\n\\tat org.jolokia.handler.ReadHandler.getAllAttributesNames(ReadHandler.java:245)\\n\\tat org.jolokia.handler.ReadHandler.resolveAttributes(ReadHandler.java:221)\\n\\tat org.jolokia.handler.ReadHandler.f... [ Message content over the limit has been removed. ] ...clipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\\n\\tat org.eclipse.jetty.server.Server.handle(Server.java:516)\\n\\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)\\n\\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)\\n\\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)\\n\\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)\\n\\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)\\n\\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)\\n\\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)\\n\\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)\\n\\tat java.base/java.lang.Thread.run(Thread.java:833)\\n",\n "status": 404\n}\n\'\'\')' failed: JSONDecodeError: Invalid control character at: line 8 column 190 (char 619) ------------------------------------------------------------------------------ Check All Shards After Isolate :: Check all shards in controller. | FAIL | Parent suite setup failed: Evaluating expression 'json.loads(\'\'\'{\n "error": "javax.management.InstanceNotFoundException : org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "error_type": "javax.management.InstanceNotFoundException",\n "request": {\n "mbean": "org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "type": "read"\n },\n "stacktrace": "javax.management.InstanceNotFoundException: org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1088)\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBeanInfo(DefaultMBeanServerInterceptor.java:1358)\\n\\tat java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.getMBeanInfo(JmxMBeanServer.java:921)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:46)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:41)\\n\\tat org.jolokia.backend.executor.AbstractMBeanServerExecutor.call(AbstractMBeanServerExecutor.java:90)\\n\\tat org.jolokia.handler.ReadHandler.getMBeanInfo(ReadHandler.java:233)\\n\\tat org.jolokia.handler.ReadHandler.getAllAttributesNames(ReadHandler.java:245)\\n\\tat org.jolokia.handler.ReadHandler.resolveAttributes(ReadHandler.java:221)\\n\\tat org.jolokia.handler.ReadHandler.f... [ Message content over the limit has been removed. ] ...clipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\\n\\tat org.eclipse.jetty.server.Server.handle(Server.java:516)\\n\\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)\\n\\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)\\n\\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)\\n\\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)\\n\\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)\\n\\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)\\n\\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)\\n\\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)\\n\\tat java.base/java.lang.Thread.run(Thread.java:833)\\n",\n "status": 404\n}\n\'\'\')' failed: JSONDecodeError: Invalid control character at: line 8 column 190 (char 619) ------------------------------------------------------------------------------ Rejoin Entity Leader :: Rejoin the entity-ownership Leader. | FAIL | Parent suite setup failed: Evaluating expression 'json.loads(\'\'\'{\n "error": "javax.management.InstanceNotFoundException : org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "error_type": "javax.management.InstanceNotFoundException",\n "request": {\n "mbean": "org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "type": "read"\n },\n "stacktrace": "javax.management.InstanceNotFoundException: org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1088)\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBeanInfo(DefaultMBeanServerInterceptor.java:1358)\\n\\tat java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.getMBeanInfo(JmxMBeanServer.java:921)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:46)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:41)\\n\\tat org.jolokia.backend.executor.AbstractMBeanServerExecutor.call(AbstractMBeanServerExecutor.java:90)\\n\\tat org.jolokia.handler.ReadHandler.getMBeanInfo(ReadHandler.java:233)\\n\\tat org.jolokia.handler.ReadHandler.getAllAttributesNames(ReadHandler.java:245)\\n\\tat org.jolokia.handler.ReadHandler.resolveAttributes(ReadHandler.java:221)\\n\\tat org.jolokia.handler.ReadHandler.f... [ Message content over the limit has been removed. ] ...clipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\\n\\tat org.eclipse.jetty.server.Server.handle(Server.java:516)\\n\\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)\\n\\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)\\n\\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)\\n\\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)\\n\\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)\\n\\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)\\n\\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)\\n\\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)\\n\\tat java.base/java.lang.Thread.run(Thread.java:833)\\n",\n "status": 404\n}\n\'\'\')' failed: JSONDecodeError: Invalid control character at: line 8 column 190 (char 619) ------------------------------------------------------------------------------ Check All Shards After Rejoin :: Check all shards in controller. | FAIL | Parent suite setup failed: Evaluating expression 'json.loads(\'\'\'{\n "error": "javax.management.InstanceNotFoundException : org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "error_type": "javax.management.InstanceNotFoundException",\n "request": {\n "mbean": "org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "type": "read"\n },\n "stacktrace": "javax.management.InstanceNotFoundException: org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1088)\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBeanInfo(DefaultMBeanServerInterceptor.java:1358)\\n\\tat java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.getMBeanInfo(JmxMBeanServer.java:921)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:46)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:41)\\n\\tat org.jolokia.backend.executor.AbstractMBeanServerExecutor.call(AbstractMBeanServerExecutor.java:90)\\n\\tat org.jolokia.handler.ReadHandler.getMBeanInfo(ReadHandler.java:233)\\n\\tat org.jolokia.handler.ReadHandler.getAllAttributesNames(ReadHandler.java:245)\\n\\tat org.jolokia.handler.ReadHandler.resolveAttributes(ReadHandler.java:221)\\n\\tat org.jolokia.handler.ReadHandler.f... [ Message content over the limit has been removed. ] ...clipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\\n\\tat org.eclipse.jetty.server.Server.handle(Server.java:516)\\n\\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)\\n\\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)\\n\\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)\\n\\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)\\n\\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)\\n\\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)\\n\\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)\\n\\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)\\n\\tat java.base/java.lang.Thread.run(Thread.java:833)\\n",\n "status": 404\n}\n\'\'\')' failed: JSONDecodeError: Invalid control character at: line 8 column 190 (char 619) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Entity Isolate :: Suite for performi... | FAIL | Suite setup failed: Evaluating expression 'json.loads(\'\'\'{\n "error": "javax.management.InstanceNotFoundException : org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "error_type": "javax.management.InstanceNotFoundException",\n "request": {\n "mbean": "org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "type": "read"\n },\n "stacktrace": "javax.management.InstanceNotFoundException: org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1088)\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBeanInfo(DefaultMBeanServerInterceptor.java:1358)\\n\\tat java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.getMBeanInfo(JmxMBeanServer.java:921)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:46)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:41)\\n\\tat org.jolokia.backend.executor.AbstractMBeanServerExecutor.call(AbstractMBeanServerExecutor.java:90)\\n\\tat org.jolokia.handler.ReadHandler.getMBeanInfo(ReadHandler.java:233)\\n\\tat org.jolokia.handler.ReadHandler.getAllAttributesNames(ReadHandler.java:245)\\n\\tat org.jolokia.handler.ReadHandler.resolveAttributes(ReadHandler.java:221)\\n\\tat org.jolokia.handler.ReadHandler.f... [ Message content over the limit has been removed. ] ...clipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\\n\\tat org.eclipse.jetty.server.Server.handle(Server.java:516)\\n\\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)\\n\\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)\\n\\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)\\n\\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)\\n\\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)\\n\\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)\\n\\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)\\n\\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)\\n\\tat java.base/java.lang.Thread.run(Thread.java:833)\\n",\n "status": 404\n}\n\'\'\')' failed: JSONDecodeError: Invalid control character at: line 8 column 190 (char 619) 5 tests, 0 passed, 5 failed ============================================================================== controller-clustering-ask.txt.Buycar Failover Isolation :: This test focuse... ============================================================================== Add_Cars_To_Leader_And_Verify :: Add all needed cars to car Leader... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Add_People_To_First_Follower_And_Verify :: Add all needed people t... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Buy_Cars_On_Leader_And_Verify :: Buy some cars on the leader member. | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Buy_Cars_On_Follower_And_Verify :: Buy some cars on the first foll... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Isolate_and_Rejoin_People_Leader :: Previous people Leader is isol... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Buy_Cars_On_Leader_After_Rejoin_And_Verify :: Buy some cars on the... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Buy_Cars_On_Follower_After_Rejoin_And_Verify :: Buy some cars on t... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_All_CarPeople :: DELETE car-people container. No verificati... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_All_People :: DELETE people container. No verification beyo... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_All_Cars :: DELETE cars container. No verification beyond h... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ controller-clustering-ask.txt.Buycar Failover Isolation :: This te... | FAIL | Suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig 10 tests, 0 passed, 10 failed ============================================================================== controller-clustering-ask.txt.Car Failover Crud Isolation :: Suite mixing b... ============================================================================== Add_Original_Cars_On_Old_Leader_And_Verify :: Add initial cars on ... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Isolate_Original_Car_Leader :: Isolate the car Leader to cause a n... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Wait_For_New_Leader :: Wait until new car Leader is elected. | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Original_Cars_On_New_Leader :: GET cars from new Leader, shoul... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Original_Cars_On_New_Followers :: The same check on other exis... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_Original_Cars_On_New_Leader :: Delete cars on the new Leader. | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Add_Leader_Cars_On_New_Leader :: Add cars on the new Leader. | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Leader_Cars_On_New_Leader :: GET cars from new Leader, should ... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Leader_Cars_On_New_Followers :: The same check on other existi... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_Leader_Cars_On_New_First_Follower :: Delete cars in new fir... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Add_Follower_Cars_On_New_First_Follower :: Add cars on the new fir... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Folower_Cars_On_New_Leader :: Get cars from the new Leader, sh... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Follower_Cars_On_New_Followers :: The same check on other exis... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Rejoin_Old_Car_Leader :: Rejoin the isolated member without deleti... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Folower_Cars_On_Old_Leader :: GET cars from the restarted memb... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_Follower_Cars_On_New_Leader :: Delete cars on the last Leader. | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ controller-clustering-ask.txt.Car Failover Crud Isolation :: Suite... | FAIL | Suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig 16 tests, 0 passed, 16 failed ============================================================================== controller-clustering-ask.txt.Car Outage Corners :: Cluster suite for testi... ============================================================================== Stop_Majority_Of_The_Followers :: Stop half plus one car Follower ... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Attempt_To_Add_Cars_To_Leader :: Adding cars should fail, as major... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Start_Tipping_Follower :: Start one Follower member without persis... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Add_Cars_On_Tipping_Follower :: Add cars on the tipping Follower. | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Cars_On_Existing_Members :: On each up member: GET cars, shoul... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Start_Other_Followers :: Start other followers without persisted d... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Cars_On_New_Follower_Leader :: GET cars from a new follower to... | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_Cars_On_Leader :: Delete cars on Leader. | FAIL | Parent suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ controller-clustering-ask.txt.Car Outage Corners :: Cluster suite ... | FAIL | Suite setup failed: HTTPError: 500 Server Error: Internal Server Error for url: http://10.30.171.170:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig 8 tests, 0 passed, 8 failed ============================================================================== controller-clustering-ask.txt | FAIL | 195 tests, 24 passed, 171 failed ============================================================================== Output: /w/workspace/controller-csit-3node-clustering-ask-all-calcium/output.xml Log: /w/workspace/controller-csit-3node-clustering-ask-all-calcium/log.html Report: /w/workspace/controller-csit-3node-clustering-ask-all-calcium/report.html + true + echo 'Examining the files in data/log and checking filesize' Examining the files in data/log and checking filesize + ssh 10.30.171.165 'ls -altr /tmp/karaf-0.20.2-SNAPSHOT/data/log/' Warning: Permanently added '10.30.171.165' (ECDSA) to the list of known hosts. total 108420 drwxrwxr-x. 2 jenkins jenkins 48 Jul 7 00:27 . -rw-rw-r--. 1 jenkins jenkins 1720 Jul 7 00:27 karaf_console.log drwxrwxr-x. 9 jenkins jenkins 149 Jul 7 04:29 .. -rw-rw-r--. 1 jenkins jenkins 85112066 Jul 7 04:43 karaf.log + ssh 10.30.171.165 'du -hs /tmp/karaf-0.20.2-SNAPSHOT/data/log/*' Warning: Permanently added '10.30.171.165' (ECDSA) to the list of known hosts. 4.0K /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf_console.log 106M /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log + ssh 10.30.171.170 'ls -altr /tmp/karaf-0.20.2-SNAPSHOT/data/log/' Warning: Permanently added '10.30.171.170' (ECDSA) to the list of known hosts. total 107204 drwxrwxr-x. 2 jenkins jenkins 48 Jul 7 00:27 . -rw-rw-r--. 1 jenkins jenkins 2042 Jul 7 00:28 karaf_console.log drwxrwxr-x. 9 jenkins jenkins 149 Jul 7 04:29 .. -rw-rw-r--. 1 jenkins jenkins 83671805 Jul 7 04:43 karaf.log + ssh 10.30.171.170 'du -hs /tmp/karaf-0.20.2-SNAPSHOT/data/log/*' Warning: Permanently added '10.30.171.170' (ECDSA) to the list of known hosts. 4.0K /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf_console.log 105M /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log + ssh 10.30.170.8 'ls -altr /tmp/karaf-0.20.2-SNAPSHOT/data/log/' Warning: Permanently added '10.30.170.8' (ECDSA) to the list of known hosts. total 128076 drwxrwxr-x. 2 jenkins jenkins 48 Jul 7 00:27 . -rw-rw-r--. 1 jenkins jenkins 139578 Jul 7 00:36 karaf_console.log drwxrwxr-x. 4 jenkins jenkins 28 Jul 7 04:29 .. -rw-rw-r--. 1 jenkins jenkins 90801558 Jul 7 04:43 karaf.log + ssh 10.30.170.8 'du -hs /tmp/karaf-0.20.2-SNAPSHOT/data/log/*' Warning: Permanently added '10.30.170.8' (ECDSA) to the list of known hosts. 140K /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf_console.log 125M /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log + set +e ++ seq 1 3 + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_1_IP + echo 'Let'\''s take the karaf thread dump again' Let's take the karaf thread dump again + ssh 10.30.171.165 'sudo ps aux' Warning: Permanently added '10.30.171.165' (ECDSA) to the list of known hosts. ++ tr -s ' ' ++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-ask-all-calcium/ps_after.log ++ cut -f2 '-d ' ++ grep -v grep + pid=31047 + echo 'karaf main: org.apache.karaf.main.Main, pid:31047' karaf main: org.apache.karaf.main.Main, pid:31047 + ssh 10.30.171.165 '/usr/lib/jvm/java-17-openjdk/bin/jstack -l 31047' Warning: Permanently added '10.30.171.165' (ECDSA) to the list of known hosts. + echo 'killing karaf process...' killing karaf process... + ssh 10.30.171.165 bash -c 'ps axf | grep karaf | grep -v grep | awk '\''{print "kill -9 " $1}'\'' | sh' Warning: Permanently added '10.30.171.165' (ECDSA) to the list of known hosts. + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_2_IP + echo 'Let'\''s take the karaf thread dump again' Let's take the karaf thread dump again + ssh 10.30.171.170 'sudo ps aux' Warning: Permanently added '10.30.171.170' (ECDSA) to the list of known hosts. ++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-ask-all-calcium/ps_after.log ++ grep -v grep ++ tr -s ' ' ++ cut -f2 '-d ' + pid=30203 + echo 'karaf main: org.apache.karaf.main.Main, pid:30203' karaf main: org.apache.karaf.main.Main, pid:30203 + ssh 10.30.171.170 '/usr/lib/jvm/java-17-openjdk/bin/jstack -l 30203' Warning: Permanently added '10.30.171.170' (ECDSA) to the list of known hosts. + echo 'killing karaf process...' killing karaf process... + ssh 10.30.171.170 bash -c 'ps axf | grep karaf | grep -v grep | awk '\''{print "kill -9 " $1}'\'' | sh' Warning: Permanently added '10.30.171.170' (ECDSA) to the list of known hosts. + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_3_IP + echo 'Let'\''s take the karaf thread dump again' Let's take the karaf thread dump again + ssh 10.30.170.8 'sudo ps aux' Warning: Permanently added '10.30.170.8' (ECDSA) to the list of known hosts. ++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-ask-all-calcium/ps_after.log ++ grep -v grep ++ cut -f2 '-d ' ++ tr -s ' ' + pid=5756 + echo 'karaf main: org.apache.karaf.main.Main, pid:5756' karaf main: org.apache.karaf.main.Main, pid:5756 + ssh 10.30.170.8 '/usr/lib/jvm/java-17-openjdk/bin/jstack -l 5756' Warning: Permanently added '10.30.170.8' (ECDSA) to the list of known hosts. + echo 'killing karaf process...' killing karaf process... + ssh 10.30.170.8 bash -c 'ps axf | grep karaf | grep -v grep | awk '\''{print "kill -9 " $1}'\'' | sh' Warning: Permanently added '10.30.170.8' (ECDSA) to the list of known hosts. + sleep 5 ++ seq 1 3 + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_1_IP + echo 'Compressing karaf.log 1' Compressing karaf.log 1 + ssh 10.30.171.165 gzip --best /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log Warning: Permanently added '10.30.171.165' (ECDSA) to the list of known hosts. gzip: /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log: file size changed while zipping + echo 'Fetching compressed karaf.log 1' Fetching compressed karaf.log 1 + scp 10.30.171.165:/tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log.gz odl1_karaf.log.gz Warning: Permanently added '10.30.171.165' (ECDSA) to the list of known hosts. + ssh 10.30.171.165 rm -f /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log.gz Warning: Permanently added '10.30.171.165' (ECDSA) to the list of known hosts. + scp 10.30.171.165:/tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf_console.log odl1_karaf_console.log Warning: Permanently added '10.30.171.165' (ECDSA) to the list of known hosts. + ssh 10.30.171.165 rm -f /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf_console.log Warning: Permanently added '10.30.171.165' (ECDSA) to the list of known hosts. + echo 'Fetch GC logs' Fetch GC logs + mkdir -p gclogs-1 + scp '10.30.171.165:/tmp/karaf-0.20.2-SNAPSHOT/data/log/*.log' gclogs-1/ Warning: Permanently added '10.30.171.165' (ECDSA) to the list of known hosts. scp: /tmp/karaf-0.20.2-SNAPSHOT/data/log/*.log: No such file or directory + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_2_IP + echo 'Compressing karaf.log 2' Compressing karaf.log 2 + ssh 10.30.171.170 gzip --best /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log Warning: Permanently added '10.30.171.170' (ECDSA) to the list of known hosts. gzip: /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log: file size changed while zipping + echo 'Fetching compressed karaf.log 2' Fetching compressed karaf.log 2 + scp 10.30.171.170:/tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log.gz odl2_karaf.log.gz Warning: Permanently added '10.30.171.170' (ECDSA) to the list of known hosts. + ssh 10.30.171.170 rm -f /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log.gz Warning: Permanently added '10.30.171.170' (ECDSA) to the list of known hosts. + scp 10.30.171.170:/tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf_console.log odl2_karaf_console.log Warning: Permanently added '10.30.171.170' (ECDSA) to the list of known hosts. + ssh 10.30.171.170 rm -f /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf_console.log Warning: Permanently added '10.30.171.170' (ECDSA) to the list of known hosts. + echo 'Fetch GC logs' Fetch GC logs + mkdir -p gclogs-2 + scp '10.30.171.170:/tmp/karaf-0.20.2-SNAPSHOT/data/log/*.log' gclogs-2/ Warning: Permanently added '10.30.171.170' (ECDSA) to the list of known hosts. scp: /tmp/karaf-0.20.2-SNAPSHOT/data/log/*.log: No such file or directory + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_3_IP + echo 'Compressing karaf.log 3' Compressing karaf.log 3 + ssh 10.30.170.8 gzip --best /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log Warning: Permanently added '10.30.170.8' (ECDSA) to the list of known hosts. gzip: /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log: file size changed while zipping + echo 'Fetching compressed karaf.log 3' Fetching compressed karaf.log 3 + scp 10.30.170.8:/tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log.gz odl3_karaf.log.gz Warning: Permanently added '10.30.170.8' (ECDSA) to the list of known hosts. + ssh 10.30.170.8 rm -f /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf.log.gz Warning: Permanently added '10.30.170.8' (ECDSA) to the list of known hosts. + scp 10.30.170.8:/tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf_console.log odl3_karaf_console.log Warning: Permanently added '10.30.170.8' (ECDSA) to the list of known hosts. + ssh 10.30.170.8 rm -f /tmp/karaf-0.20.2-SNAPSHOT/data/log/karaf_console.log Warning: Permanently added '10.30.170.8' (ECDSA) to the list of known hosts. + echo 'Fetch GC logs' Fetch GC logs + mkdir -p gclogs-3 + scp '10.30.170.8:/tmp/karaf-0.20.2-SNAPSHOT/data/log/*.log' gclogs-3/ Warning: Permanently added '10.30.170.8' (ECDSA) to the list of known hosts. scp: /tmp/karaf-0.20.2-SNAPSHOT/data/log/*.log: No such file or directory + echo 'Examine copied files' Examine copied files + ls -lt total 55800 drwxrwxr-x. 2 jenkins jenkins 6 Jul 7 04:44 gclogs-3 -rw-rw-r--. 1 jenkins jenkins 139578 Jul 7 04:44 odl3_karaf_console.log -rw-rw-r--. 1 jenkins jenkins 2040923 Jul 7 04:44 odl3_karaf.log.gz drwxrwxr-x. 2 jenkins jenkins 6 Jul 7 04:44 gclogs-2 -rw-rw-r--. 1 jenkins jenkins 2042 Jul 7 04:44 odl2_karaf_console.log -rw-rw-r--. 1 jenkins jenkins 2520410 Jul 7 04:44 odl2_karaf.log.gz drwxrwxr-x. 2 jenkins jenkins 6 Jul 7 04:44 gclogs-1 -rw-rw-r--. 1 jenkins jenkins 1720 Jul 7 04:44 odl1_karaf_console.log -rw-rw-r--. 1 jenkins jenkins 2483620 Jul 7 04:44 odl1_karaf.log.gz -rw-rw-r--. 1 jenkins jenkins 149460 Jul 7 04:43 karaf_3_5756_threads_after.log -rw-rw-r--. 1 jenkins jenkins 15766 Jul 7 04:43 ps_after.log -rw-rw-r--. 1 jenkins jenkins 135240 Jul 7 04:43 karaf_2_30203_threads_after.log -rw-rw-r--. 1 jenkins jenkins 137715 Jul 7 04:43 karaf_1_31047_threads_after.log -rw-rw-r--. 1 jenkins jenkins 301557 Jul 7 04:43 report.html -rw-rw-r--. 1 jenkins jenkins 3309621 Jul 7 04:43 log.html -rw-rw-r--. 1 jenkins jenkins 45296294 Jul 7 04:43 output.xml -rw-rw-r--. 1 jenkins jenkins 4091 Jul 7 00:28 testplan.txt -rw-rw-r--. 1 jenkins jenkins 158766 Jul 7 00:28 karaf_3_5756_threads_before.log -rw-rw-r--. 1 jenkins jenkins 16603 Jul 7 00:28 ps_before.log -rw-rw-r--. 1 jenkins jenkins 161742 Jul 7 00:28 karaf_2_5730_threads_before.log -rw-rw-r--. 1 jenkins jenkins 167350 Jul 7 00:28 karaf_1_5790_threads_before.log -rw-rw-r--. 1 jenkins jenkins 3106 Jul 7 00:27 post-startup-script.sh -rw-rw-r--. 1 jenkins jenkins 252 Jul 7 00:27 startup-script.sh -rw-rw-r--. 1 jenkins jenkins 3450 Jul 7 00:27 configuration-script.sh -rw-rw-r--. 1 jenkins jenkins 695 Jul 7 00:27 custom_shard_config.txt -rw-rw-r--. 1 jenkins jenkins 134 Jul 7 00:27 scriptplan.txt -rw-rw-r--. 1 jenkins jenkins 335 Jul 7 00:27 detect_variables.env -rw-rw-r--. 1 jenkins jenkins 2619 Jul 7 00:27 pom.xml -rw-rw-r--. 1 jenkins jenkins 86 Jul 7 00:27 set_variables.env -rw-rw-r--. 1 jenkins jenkins 312 Jul 7 00:27 slave_addresses.txt -rw-rw-r--. 1 jenkins jenkins 570 Jul 7 00:26 requirements.txt -rw-rw-r--. 1 jenkins jenkins 26 Jul 7 00:26 env.properties -rw-rw-r--. 1 jenkins jenkins 335 Jul 7 00:24 stack-parameters.yaml drwxrwxr-x. 7 jenkins jenkins 4096 Jul 7 00:23 test drwxrwxr-x. 2 jenkins jenkins 6 Jul 7 00:23 test@tmp -rw-rw-r--. 1 jenkins jenkins 1404 Jul 6 20:46 maven-metadata.xml + true [controller-csit-3node-clustering-ask-all-calcium] $ /bin/sh /tmp/jenkins1545497515350336718.sh Cleaning up Robot installation... $ ssh-agent -k unset SSH_AUTH_SOCK; unset SSH_AGENT_PID; echo Agent pid 5294 killed; [ssh-agent] Stopped. Recording plot data Robot results publisher started... INFO: Checking test criticality is deprecated and will be dropped in a future release! -Parsing output xml: Done! -Copying log files to build dir: Done! -Assigning results to build: Done! -Checking thresholds: Done! Done publishing Robot results. Build step 'Publish Robot Framework test results' changed build result to UNSTABLE [PostBuildScript] - [INFO] Executing post build scripts. [controller-csit-3node-clustering-ask-all-calcium] $ /bin/bash /tmp/jenkins3695399354251626187.sh Archiving csit artifacts mv: cannot stat '*_1.png': No such file or directory mv: cannot stat '/tmp/odl1_*': No such file or directory mv: cannot stat '*_2.png': No such file or directory mv: cannot stat '/tmp/odl2_*': No such file or directory mv: cannot stat '*_3.png': No such file or directory mv: cannot stat '/tmp/odl3_*': No such file or directory % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 616k 0 616k 0 0 4307k 0 --:--:-- --:--:-- --:--:-- 4307k 100 4093k 0 4093k 0 0 4998k 0 --:--:-- --:--:-- --:--:-- 4992k Archive: robot-plugin.zip inflating: ./archives/robot-plugin/log.html inflating: ./archives/robot-plugin/output.xml inflating: ./archives/robot-plugin/report.html mv: cannot stat '*.log.gz': No such file or directory mv: cannot stat '*.csv': No such file or directory mv: cannot stat '*.png': No such file or directory [PostBuildScript] - [INFO] Executing post build scripts. [controller-csit-3node-clustering-ask-all-calcium] $ /bin/bash /tmp/jenkins3579639739792293836.sh [PostBuildScript] - [INFO] Executing post build scripts. [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties content OS_CLOUD=vex OS_STACK_NAME=releng-controller-csit-3node-clustering-ask-all-calcium-43 [EnvInject] - Variables injected successfully. provisioning config files... copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml [controller-csit-3node-clustering-ask-all-calcium] $ /bin/bash /tmp/jenkins2998586289853162670.sh ---> openstack-stack-delete.sh Setup pyenv: system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-calcium/.python-version) lf-activate-venv(): INFO: Reuse venv:/tmp/venv-1v9O from file:/tmp/.os_lf_venv lf-activate-venv(): INFO: Installing: lftools[openstack] kubernetes python-heatclient python-openstackclient lf-activate-venv(): INFO: Adding /tmp/venv-1v9O/bin to PATH INFO: Retrieving stack cost for: releng-controller-csit-3node-clustering-ask-all-calcium-43 DEBUG: Successfully retrieved stack cost: total: 1.6500000000000001 INFO: Deleting stack releng-controller-csit-3node-clustering-ask-all-calcium-43 Successfully deleted stack releng-controller-csit-3node-clustering-ask-all-calcium-43 [PostBuildScript] - [INFO] Executing post build scripts. [controller-csit-3node-clustering-ask-all-calcium] $ /bin/bash /tmp/jenkins119313151804377557.sh ---> sysstat.sh [controller-csit-3node-clustering-ask-all-calcium] $ /bin/bash /tmp/jenkins13161928333785859834.sh ---> package-listing.sh ++ facter osfamily ++ tr '[:upper:]' '[:lower:]' + OS_FAMILY=redhat + workspace=/w/workspace/controller-csit-3node-clustering-ask-all-calcium + START_PACKAGES=/tmp/packages_start.txt + END_PACKAGES=/tmp/packages_end.txt + DIFF_PACKAGES=/tmp/packages_diff.txt + PACKAGES=/tmp/packages_start.txt + '[' /w/workspace/controller-csit-3node-clustering-ask-all-calcium ']' + PACKAGES=/tmp/packages_end.txt + case "${OS_FAMILY}" in + rpm -qa + sort + '[' -f /tmp/packages_start.txt ']' + '[' -f /tmp/packages_end.txt ']' + diff /tmp/packages_start.txt /tmp/packages_end.txt + '[' /w/workspace/controller-csit-3node-clustering-ask-all-calcium ']' + mkdir -p /w/workspace/controller-csit-3node-clustering-ask-all-calcium/archives/ + cp -f /tmp/packages_diff.txt /tmp/packages_end.txt /tmp/packages_start.txt /w/workspace/controller-csit-3node-clustering-ask-all-calcium/archives/ [controller-csit-3node-clustering-ask-all-calcium] $ /bin/bash /tmp/jenkins4954963124807850761.sh ---> capture-instance-metadata.sh Setup pyenv: system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-calcium/.python-version) lf-activate-venv(): INFO: Reuse venv:/tmp/venv-1v9O from file:/tmp/.os_lf_venv lf-activate-venv(): INFO: Installing: lftools lf-activate-venv(): INFO: Adding /tmp/venv-1v9O/bin to PATH INFO: Running in OpenStack, capturing instance metadata [controller-csit-3node-clustering-ask-all-calcium] $ /bin/bash /tmp/jenkins15287832062351979184.sh provisioning config files... Could not find credentials [logs] for controller-csit-3node-clustering-ask-all-calcium #43 copy managed file [jenkins-log-archives-settings] to file:/w/workspace/controller-csit-3node-clustering-ask-all-calcium@tmp/config226484011415585199tmp Regular expression run condition: Expression=[^.*logs-s3.*], Label=[odl-logs-s3-cloudfront-index] Run condition [Regular expression match] enabling perform for step [Provide Configuration files] provisioning config files... copy managed file [jenkins-s3-log-ship] to file:/home/jenkins/.aws/credentials [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties content SERVER_ID=logs [EnvInject] - Variables injected successfully. [controller-csit-3node-clustering-ask-all-calcium] $ /bin/bash /tmp/jenkins11323354655860097929.sh ---> create-netrc.sh WARN: Log server credential not found. [controller-csit-3node-clustering-ask-all-calcium] $ /bin/bash /tmp/jenkins1823862203091927777.sh ---> python-tools-install.sh Setup pyenv: system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-calcium/.python-version) lf-activate-venv(): INFO: Reuse venv:/tmp/venv-1v9O from file:/tmp/.os_lf_venv lf-activate-venv(): INFO: Installing: lftools lf-activate-venv(): INFO: Adding /tmp/venv-1v9O/bin to PATH [controller-csit-3node-clustering-ask-all-calcium] $ /bin/bash /tmp/jenkins1562553269421823361.sh ---> sudo-logs.sh Archiving 'sudo' log.. [controller-csit-3node-clustering-ask-all-calcium] $ /bin/bash /tmp/jenkins12435995826059201236.sh ---> job-cost.sh Setup pyenv: system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-calcium/.python-version) lf-activate-venv(): INFO: Reuse venv:/tmp/venv-1v9O from file:/tmp/.os_lf_venv lf-activate-venv(): INFO: Installing: zipp==1.1.0 python-openstackclient urllib3~=1.26.15 lf-activate-venv(): INFO: Adding /tmp/venv-1v9O/bin to PATH DEBUG: total: 1.6500000000000001 INFO: Retrieving Stack Cost... INFO: Retrieving Pricing Info for: v3-standard-2 INFO: Archiving Costs [controller-csit-3node-clustering-ask-all-calcium] $ /bin/bash -l /tmp/jenkins15344781670057285954.sh ---> logs-deploy.sh Setup pyenv: system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-calcium/.python-version) lf-activate-venv(): INFO: Reuse venv:/tmp/venv-1v9O from file:/tmp/.os_lf_venv lf-activate-venv(): INFO: Installing: lftools lf-activate-venv(): INFO: Adding /tmp/venv-1v9O/bin to PATH WARNING: Nexus logging server not set INFO: S3 path logs/releng/vex-yul-odl-jenkins-1/controller-csit-3node-clustering-ask-all-calcium/43/ INFO: archiving logs to S3 ---> uname -a: Linux prd-centos8-robot-2c-8g-18399.novalocal 4.18.0-553.5.1.el8.x86_64 #1 SMP Tue May 21 05:46:01 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux ---> lscpu: Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 2 On-line CPU(s) list: 0,1 Thread(s) per core: 1 Core(s) per socket: 1 Socket(s): 2 NUMA node(s): 1 Vendor ID: AuthenticAMD CPU family: 23 Model: 49 Model name: AMD EPYC-Rome Processor Stepping: 0 CPU MHz: 2800.000 BogoMIPS: 5600.00 Virtualization: AMD-V Hypervisor vendor: KVM Virtualization type: full L1d cache: 32K L1i cache: 32K L2 cache: 512K L3 cache: 16384K NUMA node0 CPU(s): 0,1 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr wbnoinvd arat npt nrip_save umip rdpid arch_capabilities ---> nproc: 2 ---> df -h: Filesystem Size Used Avail Use% Mounted on devtmpfs 3.8G 0 3.8G 0% /dev tmpfs 3.8G 0 3.8G 0% /dev/shm tmpfs 3.8G 17M 3.8G 1% /run tmpfs 3.8G 0 3.8G 0% /sys/fs/cgroup /dev/vda1 40G 8.3G 32G 21% / tmpfs 770M 0 770M 0% /run/user/1001 ---> free -m: total used free shared buff/cache available Mem: 7697 627 4822 19 2247 6773 Swap: 1023 0 1023 ---> ip addr: 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: mtu 1458 qdisc mq state UP group default qlen 1000 link/ether fa:16:3e:cd:5b:f6 brd ff:ff:ff:ff:ff:ff altname enp0s3 altname ens3 inet 10.30.171.184/23 brd 10.30.171.255 scope global dynamic noprefixroute eth0 valid_lft 70614sec preferred_lft 70614sec inet6 fe80::f816:3eff:fecd:5bf6/64 scope link valid_lft forever preferred_lft forever ---> sar -b -r -n DEV: Linux 4.18.0-553.5.1.el8.x86_64 (centos-stream-8-robot-7d7a37eb-bc14-4dd6-9530-dc22c5eae738.noval) 07/07/2024 _x86_64_ (2 CPU) 00:22:32 LINUX RESTART (2 CPU) 12:23:02 AM tps rtps wtps bread/s bwrtn/s 12:24:01 AM 96.22 45.59 50.63 8391.87 31118.89 12:25:01 AM 125.27 4.02 121.26 104.90 11510.41 12:26:01 AM 18.68 0.85 17.83 123.98 1383.44 12:27:01 AM 25.70 0.00 25.70 0.00 1809.50 12:28:01 AM 46.45 2.52 43.94 396.00 6685.92 12:29:01 AM 2.33 0.08 2.25 31.99 61.05 12:30:01 AM 2.03 0.00 2.03 0.00 103.17 12:31:01 AM 0.50 0.00 0.50 0.00 19.64 12:32:01 AM 0.32 0.00 0.32 0.00 16.88 12:33:01 AM 0.23 0.00 0.23 0.00 10.21 12:34:01 AM 0.28 0.00 0.28 0.00 16.48 12:35:01 AM 0.23 0.00 0.23 0.00 15.01 12:36:02 AM 0.22 0.03 0.18 1.87 22.53 12:37:01 AM 0.39 0.00 0.39 0.00 8.79 12:38:01 AM 0.27 0.03 0.23 0.27 3.35 12:39:01 AM 1.35 0.03 1.32 1.33 33.36 12:40:01 AM 0.17 0.00 0.17 0.00 2.57 12:41:01 AM 0.33 0.00 0.33 0.00 6.18 12:42:01 AM 0.23 0.00 0.23 0.00 4.05 12:43:01 AM 0.23 0.00 0.23 0.00 3.40 12:44:01 AM 0.27 0.00 0.27 0.00 11.76 12:45:01 AM 7.41 5.75 1.67 555.55 32.06 12:46:01 AM 1.58 0.00 1.58 0.00 47.94 12:47:01 AM 0.17 0.00 0.17 0.00 3.73 12:48:01 AM 0.35 0.02 0.33 0.13 10.15 12:49:01 AM 0.23 0.00 0.23 0.00 3.75 12:50:01 AM 0.27 0.00 0.27 0.00 4.15 12:51:01 AM 0.23 0.00 0.23 0.00 9.40 12:52:01 AM 0.23 0.00 0.23 0.00 9.35 12:53:01 AM 0.23 0.00 0.23 0.00 7.68 12:54:01 AM 0.23 0.00 0.23 0.00 3.88 12:55:01 AM 0.15 0.00 0.15 0.00 1.07 12:56:01 AM 0.17 0.00 0.17 0.00 7.46 12:57:01 AM 0.45 0.00 0.45 0.00 5.72 12:58:01 AM 0.35 0.00 0.35 0.00 8.62 12:59:01 AM 0.25 0.00 0.25 0.00 12.33 01:00:01 AM 0.18 0.03 0.15 0.27 15.48 01:01:01 AM 0.23 0.02 0.22 0.13 10.06 01:02:01 AM 0.42 0.07 0.35 1.73 18.37 01:03:01 AM 0.45 0.00 0.45 0.00 18.74 01:04:01 AM 0.27 0.00 0.27 0.00 15.36 01:05:01 AM 0.25 0.00 0.25 0.00 24.95 01:06:01 AM 0.20 0.00 0.20 0.00 4.03 01:07:01 AM 0.13 0.00 0.13 0.00 3.98 01:08:01 AM 0.22 0.00 0.22 0.00 3.02 01:09:01 AM 0.28 0.00 0.28 0.00 7.17 01:10:01 AM 0.12 0.00 0.12 0.00 0.77 01:11:01 AM 0.15 0.00 0.15 0.00 3.63 01:12:01 AM 0.25 0.00 0.25 0.00 3.83 01:13:01 AM 0.27 0.00 0.27 0.00 9.53 01:14:01 AM 0.27 0.00 0.27 0.00 5.23 01:15:01 AM 0.15 0.00 0.15 0.00 10.93 01:16:01 AM 0.12 0.00 0.12 0.00 1.30 01:17:01 AM 0.12 0.00 0.12 0.00 1.15 01:18:01 AM 0.18 0.00 0.18 0.00 4.37 01:19:01 AM 0.27 0.00 0.27 0.00 5.90 01:20:01 AM 0.23 0.00 0.23 0.00 2.95 01:21:01 AM 0.37 0.00 0.37 0.00 11.86 01:22:01 AM 0.38 0.00 0.38 0.00 5.00 01:23:01 AM 0.27 0.00 0.27 0.00 3.93 01:24:01 AM 0.32 0.00 0.32 0.00 11.88 01:25:01 AM 0.17 0.00 0.17 0.00 2.37 01:26:01 AM 0.15 0.00 0.15 0.00 1.62 01:27:01 AM 0.13 0.00 0.13 0.00 10.68 01:28:01 AM 0.32 0.00 0.32 0.00 13.41 01:29:01 AM 0.83 0.00 0.83 0.00 22.00 01:30:01 AM 0.55 0.00 0.55 0.00 16.92 01:31:01 AM 0.30 0.00 0.30 0.00 10.61 01:32:01 AM 0.13 0.00 0.13 0.00 15.33 01:33:01 AM 0.27 0.00 0.27 0.00 12.85 01:34:01 AM 0.25 0.00 0.25 0.00 16.69 01:35:01 AM 0.30 0.00 0.30 0.00 19.73 01:36:01 AM 0.40 0.00 0.40 0.00 4.78 01:37:01 AM 0.20 0.00 0.20 0.00 3.00 01:38:01 AM 0.28 0.00 0.28 0.00 6.35 01:39:01 AM 0.20 0.00 0.20 0.00 3.07 01:40:01 AM 0.14 0.00 0.14 0.00 3.52 01:41:01 AM 0.17 0.00 0.17 0.00 2.03 01:42:01 AM 0.18 0.00 0.18 0.00 8.55 01:43:01 AM 0.35 0.00 0.35 0.00 11.56 01:44:01 AM 0.28 0.00 0.28 0.00 4.18 01:45:01 AM 0.13 0.00 0.13 0.00 1.43 01:46:01 AM 1.10 0.32 0.78 6.93 20.88 01:47:01 AM 0.20 0.00 0.20 0.00 7.50 01:48:01 AM 0.30 0.00 0.30 0.00 4.40 01:49:01 AM 0.40 0.00 0.40 0.00 10.75 01:50:01 AM 0.17 0.00 0.17 0.00 5.48 01:51:01 AM 0.18 0.00 0.18 0.00 8.16 01:52:01 AM 0.15 0.00 0.15 0.00 6.67 01:53:01 AM 0.22 0.00 0.22 0.00 3.40 01:54:01 AM 0.27 0.00 0.27 0.00 5.40 01:55:01 AM 0.18 0.00 0.18 0.00 6.53 01:56:01 AM 0.20 0.00 0.20 0.00 2.38 01:57:01 AM 0.17 0.00 0.17 0.00 10.15 01:58:01 AM 0.42 0.00 0.42 0.00 16.10 01:59:01 AM 0.45 0.00 0.45 0.00 16.04 02:00:01 AM 0.15 0.00 0.15 0.00 8.26 02:01:01 AM 0.17 0.00 0.17 0.00 12.16 02:02:01 AM 0.30 0.00 0.30 0.00 17.59 02:03:01 AM 0.25 0.00 0.25 0.00 10.48 02:04:01 AM 0.28 0.00 0.28 0.00 23.93 02:05:01 AM 0.12 0.00 0.12 0.00 2.27 02:06:01 AM 0.28 0.00 0.28 0.00 4.63 02:07:01 AM 0.13 0.00 0.13 0.00 3.73 02:08:01 AM 0.33 0.00 0.33 0.00 5.23 02:09:01 AM 0.25 0.00 0.25 0.00 6.05 02:10:01 AM 0.17 0.00 0.17 0.00 2.02 02:11:01 AM 0.17 0.00 0.17 0.00 8.45 02:12:01 AM 0.35 0.00 0.35 0.00 6.57 02:13:01 AM 0.25 0.00 0.25 0.00 13.66 02:14:01 AM 0.23 0.00 0.23 0.00 3.75 02:15:01 AM 0.20 0.00 0.20 0.00 2.25 02:16:01 AM 0.17 0.00 0.17 0.00 3.48 02:17:01 AM 0.13 0.00 0.13 0.00 2.92 02:18:01 AM 0.23 0.00 0.23 0.00 3.10 02:19:01 AM 0.27 0.00 0.27 0.00 10.20 02:20:01 AM 0.32 0.00 0.32 0.00 3.18 02:21:01 AM 0.43 0.00 0.43 0.00 6.48 02:22:01 AM 0.48 0.00 0.48 0.00 11.88 02:23:01 AM 0.37 0.00 0.37 0.00 4.02 02:24:01 AM 0.47 0.00 0.47 0.00 7.63 02:25:01 AM 0.32 0.00 0.32 0.00 11.08 02:26:01 AM 0.23 0.00 0.23 0.00 9.06 02:27:01 AM 0.15 0.00 0.15 0.00 15.33 02:28:01 AM 0.25 0.00 0.25 0.00 16.03 02:29:01 AM 0.50 0.00 0.50 0.00 20.24 02:30:01 AM 0.22 0.00 0.22 0.00 9.73 02:31:01 AM 0.28 0.00 0.28 0.00 18.69 02:32:01 AM 0.20 0.00 0.20 0.00 11.91 02:33:01 AM 0.23 0.00 0.23 0.00 19.86 02:34:01 AM 0.27 0.00 0.27 0.00 4.48 02:35:01 AM 0.15 0.00 0.15 0.00 2.08 02:36:01 AM 0.27 0.00 0.27 0.00 6.23 02:37:01 AM 0.17 0.00 0.17 0.00 2.15 02:38:01 AM 0.15 0.00 0.15 0.00 3.98 02:39:01 AM 0.97 0.00 0.97 0.00 14.96 02:40:01 AM 0.42 0.00 0.42 0.00 10.85 02:41:02 AM 0.43 0.00 0.43 0.00 12.18 02:42:01 AM 0.14 0.00 0.14 0.00 1.76 02:43:01 AM 0.13 0.00 0.13 0.00 1.07 02:44:01 AM 0.35 0.00 0.35 0.00 8.17 02:45:01 AM 0.12 0.00 0.12 0.00 2.80 02:46:01 AM 0.13 0.00 0.13 0.00 2.93 02:47:01 AM 0.13 0.00 0.13 0.00 7.21 02:48:01 AM 0.23 0.00 0.23 0.00 4.17 02:49:01 AM 0.37 0.00 0.37 0.00 9.88 02:50:01 AM 0.25 0.00 0.25 0.00 13.06 02:51:01 AM 0.18 0.00 0.18 0.00 14.44 02:52:01 AM 0.20 0.00 0.20 0.00 14.55 02:53:01 AM 0.22 0.00 0.22 0.00 9.51 02:54:01 AM 0.28 0.00 0.28 0.00 17.13 02:55:01 AM 0.15 0.00 0.15 0.00 14.95 02:56:01 AM 0.30 0.00 0.30 0.00 24.95 02:57:01 AM 0.37 0.00 0.37 0.00 7.33 02:58:01 AM 0.37 0.00 0.37 0.00 4.53 02:59:01 AM 0.40 0.00 0.40 0.00 8.43 03:00:01 AM 0.13 0.00 0.13 0.00 1.45 03:01:01 AM 0.15 0.00 0.15 0.00 1.20 03:02:01 AM 0.25 0.00 0.25 0.00 5.73 03:03:01 AM 0.27 0.00 0.27 0.00 4.17 03:04:01 AM 0.23 0.00 0.23 0.00 9.23 03:05:01 AM 0.18 0.00 0.18 0.00 8.05 03:06:01 AM 0.15 0.00 0.15 0.00 1.63 03:07:01 AM 0.13 0.00 0.13 0.00 0.95 03:08:01 AM 0.28 0.00 0.28 0.00 8.08 03:09:01 AM 0.22 0.00 0.22 0.00 5.23 03:10:01 AM 0.23 0.00 0.23 0.00 6.53 03:11:01 AM 0.35 0.00 0.35 0.00 9.36 03:12:01 AM 0.10 0.00 0.10 0.00 0.63 03:13:01 AM 0.25 0.00 0.25 0.00 7.56 03:14:01 AM 0.73 0.00 0.73 0.00 20.90 03:15:01 AM 0.48 0.00 0.48 0.00 15.71 03:16:01 AM 0.27 0.00 0.27 0.00 12.25 03:17:01 AM 0.28 0.00 0.28 0.00 14.10 03:18:01 AM 0.27 0.00 0.27 0.00 18.58 03:19:01 AM 0.22 0.00 0.22 0.00 9.23 03:20:01 AM 0.15 0.00 0.15 0.00 25.46 03:21:01 AM 0.17 0.00 0.17 0.00 4.95 03:22:01 AM 0.12 0.00 0.12 0.00 1.57 03:23:01 AM 0.22 0.00 0.22 0.00 5.67 03:24:01 AM 0.35 0.00 0.35 0.00 5.23 03:25:01 AM 0.22 0.00 0.22 0.00 4.42 03:26:01 AM 0.12 0.00 0.12 0.00 1.35 03:27:01 AM 0.13 0.00 0.13 0.00 0.93 03:28:01 AM 0.22 0.00 0.22 0.00 10.20 03:29:01 AM 0.25 0.00 0.25 0.00 8.85 03:30:01 AM 0.12 0.00 0.12 0.00 1.17 03:31:01 AM 0.27 0.00 0.27 0.00 4.95 03:32:01 AM 0.20 0.00 0.20 0.00 3.20 03:33:01 AM 0.20 0.00 0.20 0.00 4.35 03:34:01 AM 0.27 0.00 0.27 0.00 8.45 03:35:01 AM 0.17 0.00 0.17 0.00 1.60 03:36:01 AM 0.17 0.00 0.17 0.00 5.65 03:37:01 AM 0.17 0.00 0.17 0.00 13.88 03:38:01 AM 0.42 0.00 0.42 0.00 20.30 03:39:01 AM 0.23 0.00 0.23 0.00 16.19 03:40:01 AM 0.22 0.00 0.22 0.00 12.65 03:41:01 AM 0.27 0.00 0.27 0.00 16.29 03:42:01 AM 0.20 0.00 0.20 0.00 9.62 03:43:01 AM 0.18 0.00 0.18 0.00 22.46 03:44:01 AM 0.32 0.00 0.32 0.00 6.95 03:45:02 AM 0.17 0.00 0.17 0.00 2.48 03:46:01 AM 0.15 0.00 0.15 0.00 4.62 03:47:01 AM 0.18 0.00 0.18 0.00 2.17 03:48:01 AM 0.25 0.00 0.25 0.00 5.06 03:49:01 AM 0.38 0.00 0.38 0.00 6.46 03:50:01 AM 0.17 0.00 0.17 0.00 2.37 03:51:01 AM 0.18 0.00 0.18 0.00 11.38 03:52:01 AM 0.20 0.00 0.20 0.00 8.41 03:53:01 AM 0.17 0.00 0.17 0.00 14.50 03:54:01 AM 0.28 0.00 0.28 0.00 16.38 03:55:01 AM 0.25 0.00 0.25 0.00 15.86 03:56:01 AM 0.17 0.00 0.17 0.00 8.71 03:57:01 AM 0.43 0.00 0.43 0.00 19.75 03:58:01 AM 0.47 0.00 0.47 0.00 18.03 03:59:01 AM 0.22 0.00 0.22 0.00 17.13 04:00:01 AM 0.15 0.00 0.15 0.00 1.87 04:01:01 AM 0.13 0.00 0.13 0.00 0.95 04:02:01 AM 0.25 0.00 0.25 0.00 6.13 04:03:01 AM 0.27 0.00 0.27 0.00 4.03 04:04:01 AM 0.25 0.00 0.25 0.00 5.63 04:05:01 AM 0.13 0.00 0.13 0.00 1.43 04:06:01 AM 0.12 0.00 0.12 0.00 6.67 04:07:01 AM 0.17 0.00 0.17 0.00 6.80 04:08:01 AM 0.23 0.00 0.23 0.00 13.65 04:09:01 AM 0.75 0.00 0.75 0.00 21.82 04:10:01 AM 0.55 0.00 0.55 0.00 17.33 04:11:01 AM 0.33 0.00 0.33 0.00 11.81 04:12:01 AM 0.77 0.00 0.77 0.00 24.93 04:13:01 AM 0.62 0.00 0.62 0.00 17.86 04:14:01 AM 0.27 0.00 0.27 0.00 28.30 04:15:01 AM 0.13 0.00 0.13 0.00 4.80 04:16:01 AM 0.12 0.00 0.12 0.00 1.32 04:17:01 AM 0.12 0.00 0.12 0.00 3.32 04:18:01 AM 0.33 0.00 0.33 0.00 5.11 04:19:01 AM 0.20 0.00 0.20 0.00 3.22 04:20:01 AM 0.13 0.00 0.13 0.00 3.35 04:21:01 AM 0.18 0.00 0.18 0.00 1.92 04:22:01 AM 0.18 0.00 0.18 0.00 8.03 04:23:01 AM 0.27 0.00 0.27 0.00 9.01 04:24:01 AM 0.27 0.00 0.27 0.00 17.78 04:25:01 AM 0.27 0.00 0.27 0.00 10.01 04:26:01 AM 0.22 0.00 0.22 0.00 18.28 04:27:01 AM 0.12 0.00 0.12 0.00 7.47 04:28:01 AM 0.23 0.00 0.23 0.00 16.31 04:29:01 AM 0.25 0.00 0.25 0.00 17.79 04:30:01 AM 0.13 0.00 0.13 0.00 18.66 04:31:01 AM 0.17 0.00 0.17 0.00 5.48 04:32:01 AM 0.23 0.00 0.23 0.00 3.48 04:33:01 AM 0.28 0.00 0.28 0.00 6.92 04:34:01 AM 0.28 0.00 0.28 0.00 3.92 04:35:01 AM 0.17 0.00 0.17 0.00 3.62 04:36:01 AM 0.15 0.00 0.15 0.00 1.47 04:37:01 AM 0.12 0.00 0.12 0.00 0.78 04:38:01 AM 0.25 0.00 0.25 0.00 15.80 04:39:01 AM 0.27 0.00 0.27 0.00 10.16 04:40:01 AM 0.38 0.00 0.38 0.00 10.01 04:41:01 AM 0.53 0.00 0.53 0.00 18.54 04:42:01 AM 0.18 0.00 0.18 0.00 16.38 04:43:01 AM 0.20 0.00 0.20 0.00 14.05 04:44:01 AM 0.70 0.00 0.70 0.00 31.99 04:45:01 AM 19.24 0.42 18.83 47.72 2768.24 Average: 1.57 0.23 1.34 36.37 218.82 12:23:02 AM kbmemfree kbavail kbmemused %memused kbbuffers kbcached kbcommit %commit kbactive kbinact kbdirty 12:24:01 AM 5415416 7030864 2467016 31.30 2688 1806664 771824 8.64 215428 1964016 110084 12:25:01 AM 5190248 7015684 2692184 34.15 2688 2002508 703880 7.88 221060 2122788 30116 12:26:01 AM 5265420 7095524 2617012 33.20 2688 2007496 647340 7.25 240024 2031332 24 12:27:01 AM 5021476 7041216 2860956 36.30 2688 2189592 677568 7.59 257744 2234764 164772 12:28:01 AM 5037004 7070392 2845428 36.10 2688 2203084 638704 7.15 263332 2207668 28 12:29:01 AM 4992340 7027432 2890092 36.66 2688 2204812 725604 8.12 263472 2251828 760 12:30:01 AM 4988832 7024296 2893600 36.71 2688 2205192 725536 8.12 263472 2255500 140 12:31:01 AM 4987540 7023396 2894892 36.73 2688 2205552 725536 8.12 263472 2256896 104 12:32:01 AM 4985916 7022128 2896516 36.75 2688 2205908 725536 8.12 263472 2258656 64 12:33:01 AM 4984412 7020976 2898020 36.77 2688 2206268 725536 8.12 263472 2260048 216 12:34:01 AM 4982840 7019748 2899592 36.79 2688 2206612 725536 8.12 263472 2261528 168 12:35:01 AM 4981672 7018940 2900760 36.80 2688 2206964 725536 8.12 263472 2262788 120 12:36:02 AM 4981188 7019056 2901244 36.81 2688 2207576 725536 8.12 264196 2263192 76 12:37:01 AM 4980920 7018824 2901512 36.81 2688 2207612 725536 8.12 264196 2263552 16 12:38:01 AM 4980704 7018616 2901728 36.81 2688 2207612 722404 8.09 264324 2263436 16 12:39:01 AM 4980584 7018608 2901848 36.81 2688 2207728 722404 8.09 264328 2263744 4 12:40:01 AM 4980044 7018072 2902388 36.82 2688 2207732 740696 8.29 264328 2263796 4 12:41:01 AM 4979760 7017852 2902672 36.82 2688 2207804 740696 8.29 264332 2264288 16 12:42:01 AM 4979236 7017332 2903196 36.83 2688 2207808 740696 8.29 264332 2264912 8 12:43:01 AM 4978400 7016672 2904032 36.84 2688 2207984 721652 8.08 264480 2265492 192 12:44:01 AM 4976280 7014616 2906152 36.87 2688 2208048 721652 8.08 264480 2267620 20 12:45:01 AM 4958828 7014016 2923604 37.09 2688 2224852 747572 8.37 266940 2282448 4 12:46:01 AM 4958804 7013992 2923628 37.09 2688 2224852 747572 8.37 266940 2282588 4 12:47:01 AM 4957868 7013108 2924564 37.10 2688 2224904 747572 8.37 266940 2283512 12 12:48:01 AM 4957488 7012876 2924944 37.11 2688 2225052 747572 8.37 266944 2283800 8 12:49:01 AM 4957804 7013192 2924628 37.10 2688 2225052 747572 8.37 266944 2283480 4 12:50:01 AM 4957492 7013020 2924940 37.11 2688 2225192 733988 8.22 266944 2283752 96 12:51:01 AM 4957624 7013272 2924808 37.11 2688 2225308 733988 8.22 266944 2283752 4 12:52:01 AM 4957416 7013244 2925016 37.11 2688 2225488 733988 8.22 266944 2283932 4 12:53:01 AM 4957300 7013280 2925132 37.11 2688 2225640 733988 8.22 266944 2284088 4 12:54:01 AM 4957432 7013416 2925000 37.11 2688 2225644 733988 8.22 266944 2284164 4 12:55:01 AM 4957396 7013428 2925036 37.11 2688 2225692 733988 8.22 266944 2284144 56 12:56:01 AM 4956848 7013016 2925584 37.12 2688 2225828 733988 8.22 266948 2284396 4 12:57:01 AM 4956892 7013064 2925540 37.11 2688 2225832 733988 8.22 266948 2284412 16 12:58:01 AM 4956960 7013272 2925472 37.11 2688 2225972 733988 8.22 266948 2284464 8 12:59:01 AM 4956540 7013204 2925892 37.12 2688 2226324 733988 8.22 266948 2285068 104 01:00:01 AM 4954924 7011928 2927508 37.14 2688 2226672 735488 8.24 266948 2285948 24 01:01:01 AM 4953796 7011168 2928636 37.15 2688 2227036 735488 8.24 266952 2287316 176 01:02:01 AM 4954160 7011968 2928272 37.15 2688 2227464 735488 8.24 267048 2287132 136 01:03:01 AM 4953692 7011848 2928740 37.16 2688 2227820 735488 8.24 267048 2287464 72 01:04:01 AM 4953272 7011788 2929160 37.16 2688 2228180 735488 8.24 267048 2287892 72 01:05:01 AM 4952952 7012080 2929480 37.16 2688 2228784 735488 8.24 267048 2288504 4 01:06:01 AM 4952732 7011992 2929700 37.17 2688 2228916 735488 8.24 267048 2288640 76 01:07:01 AM 4952668 7011928 2929764 37.17 2688 2228916 735488 8.24 267048 2288776 4 01:08:01 AM 4952612 7011884 2929820 37.17 2688 2228928 735488 8.24 267048 2288864 16 01:09:01 AM 4952300 7011644 2930132 37.17 2688 2229000 735488 8.24 267048 2288924 4 01:10:01 AM 4952212 7011556 2930220 37.17 2688 2229004 735488 8.24 267052 2288860 4 01:11:01 AM 4952488 7011908 2929944 37.17 2688 2229076 735488 8.24 267048 2288748 4 01:12:01 AM 4952488 7011908 2929944 37.17 2688 2229076 735488 8.24 267048 2288784 4 01:13:01 AM 4952164 7011768 2930268 37.17 2688 2229260 715684 8.01 267048 2289108 16 01:14:01 AM 4952204 7011872 2930228 37.17 2688 2229324 715684 8.01 267048 2289136 24 01:15:01 AM 4951552 7011500 2930880 37.18 2688 2229604 715684 8.01 267048 2289368 4 01:16:01 AM 4951636 7011588 2930796 37.18 2688 2229608 715684 8.01 267048 2289488 4 01:17:01 AM 4951920 7011920 2930512 37.18 2688 2229656 715684 8.01 267048 2289412 60 01:18:01 AM 4951772 7011820 2930660 37.18 2688 2229704 715684 8.01 267048 2289464 40 01:19:01 AM 4951500 7011552 2930932 37.18 2688 2229708 715684 8.01 267048 2289536 4 01:20:01 AM 4950968 7011076 2931464 37.19 2688 2229764 750092 8.40 267048 2289872 64 01:21:01 AM 4951428 7011684 2931004 37.18 2688 2229912 733728 8.22 267048 2289684 4 01:22:01 AM 4950960 7011216 2931472 37.19 2688 2229912 733728 8.22 267048 2290216 4 01:23:01 AM 4951344 7011692 2931088 37.19 2688 2230004 733728 8.22 267048 2289780 64 01:24:01 AM 4951124 7011628 2931308 37.19 2688 2230160 733728 8.22 267048 2290020 4 01:25:01 AM 4951300 7011804 2931132 37.19 2688 2230160 733728 8.22 267048 2289952 4 01:26:01 AM 4951088 7011648 2931344 37.19 2688 2230216 733728 8.22 267048 2289992 64 01:27:01 AM 4950908 7011712 2931524 37.19 2688 2230460 733728 8.22 267048 2290240 16 01:28:01 AM 4950352 7011464 2932080 37.20 2688 2230768 733728 8.22 267048 2290892 4 01:29:01 AM 4949912 7011384 2932520 37.20 2688 2231128 733728 8.22 267048 2291384 40 01:30:01 AM 4949588 7011400 2932844 37.21 2688 2231468 749616 8.39 267048 2291576 24 01:31:01 AM 4949696 7011868 2932736 37.21 2688 2231828 749616 8.39 267048 2291720 168 01:32:01 AM 4948884 7011408 2933548 37.22 2688 2232188 749616 8.39 267048 2292460 108 01:33:01 AM 4948280 7011160 2934152 37.22 2688 2232536 749616 8.39 267048 2292932 184 01:34:01 AM 4947992 7011512 2934440 37.23 2688 2233176 733856 8.22 267048 2293300 432 01:35:01 AM 4947968 7011588 2934464 37.23 2688 2233276 733856 8.22 267048 2293452 4 01:36:01 AM 4948060 7011680 2934372 37.23 2688 2233276 733856 8.22 267048 2293336 4 01:37:01 AM 4947816 7011524 2934616 37.23 2688 2233360 733856 8.22 267048 2293624 84 01:38:01 AM 4948124 7011832 2934308 37.23 2688 2233360 733856 8.22 267048 2293420 4 01:39:01 AM 4947772 7011552 2934660 37.23 2688 2233432 733856 8.22 267048 2293556 80 01:40:01 AM 4947676 7011460 2934756 37.23 2688 2233436 734784 8.23 267048 2293728 4 01:41:01 AM 4947372 7011160 2935060 37.24 2688 2233440 734784 8.23 267048 2293944 16 01:42:01 AM 4947372 7011396 2935060 37.24 2688 2233676 734784 8.23 267048 2293872 68 01:43:01 AM 4947148 7011328 2935284 37.24 2688 2233832 734784 8.23 267056 2294068 4 01:44:01 AM 4947120 7011300 2935312 37.24 2688 2233832 734784 8.23 267056 2294128 4 01:45:01 AM 4947052 7011276 2935380 37.24 2688 2233876 734784 8.23 267056 2294104 48 01:46:01 AM 4946728 7011308 2935704 37.24 2688 2234232 734784 8.23 267180 2294356 152 01:47:01 AM 4946604 7011184 2935828 37.25 2688 2234232 734784 8.23 267180 2294356 4 01:48:01 AM 4946160 7010792 2936272 37.25 2688 2234284 734784 8.23 267180 2294772 56 01:49:01 AM 4946096 7010916 2936336 37.25 2688 2234472 734784 8.23 267180 2294808 108 01:50:01 AM 4946232 7011164 2936200 37.25 2688 2234584 747904 8.37 267180 2294832 108 01:51:01 AM 4945968 7011140 2936464 37.25 2688 2234836 747904 8.37 267180 2294952 152 01:52:01 AM 4945868 7011056 2936564 37.25 2688 2234836 747904 8.37 267180 2294968 4 01:53:01 AM 4945868 7011096 2936564 37.25 2688 2234876 747904 8.37 267180 2295008 44 01:54:01 AM 4945584 7010972 2936848 37.26 2688 2235036 747904 8.37 267188 2295228 152 01:55:01 AM 4945608 7010996 2936824 37.26 2688 2235036 747904 8.37 267188 2295160 4 01:56:01 AM 4945672 7011180 2936760 37.26 2688 2235160 747904 8.37 267188 2295284 128 01:57:01 AM 4945416 7011076 2937016 37.26 2688 2235320 747904 8.37 267188 2295600 44 01:58:01 AM 4944848 7010832 2937584 37.27 2688 2235644 747904 8.37 267188 2296084 24 01:59:01 AM 4944768 7011060 2937664 37.27 2688 2235952 747904 8.37 267188 2296232 24 02:00:01 AM 4944392 7010988 2938040 37.27 2688 2236256 763548 8.55 267188 2296596 112 02:01:01 AM 4943592 7010508 2938840 37.28 2688 2236560 746944 8.36 267188 2297372 88 02:02:01 AM 4943880 7011124 2938552 37.28 2688 2236888 746944 8.36 267204 2297220 16 02:03:01 AM 4942564 7010344 2939868 37.30 2688 2237420 746944 8.36 267204 2298208 340 02:04:01 AM 4943004 7011104 2939428 37.29 2688 2237740 746944 8.36 267204 2298064 40 02:05:01 AM 4942796 7010896 2939636 37.29 2688 2237740 746944 8.36 267204 2298140 4 02:06:01 AM 4942776 7010876 2939656 37.29 2688 2237748 746944 8.36 267232 2298324 16 02:07:01 AM 4942816 7011000 2939616 37.29 2688 2237824 746944 8.36 267232 2298092 4 02:08:01 AM 4942492 7010676 2939940 37.30 2688 2237824 746944 8.36 267232 2298444 8 02:09:01 AM 4942540 7010792 2939892 37.30 2688 2237892 746944 8.36 267232 2298240 8 02:10:01 AM 4942100 7010368 2940332 37.30 2688 2237892 750828 8.41 267232 2298648 16 02:11:01 AM 4941816 7010340 2940616 37.31 2688 2238076 730776 8.18 267232 2299040 4 02:12:01 AM 4941652 7010252 2940780 37.31 2688 2238148 730776 8.18 267232 2299108 24 02:13:01 AM 4941568 7010468 2940864 37.31 2688 2238448 730776 8.18 267232 2299272 4 02:14:01 AM 4941232 7010140 2941200 37.31 2688 2238448 730776 8.18 267232 2299340 4 02:15:01 AM 4941128 7010084 2941304 37.31 2688 2238492 730776 8.18 267232 2299388 52 02:16:01 AM 4941156 7010172 2941276 37.31 2688 2238552 730776 8.18 267232 2299368 56 02:17:01 AM 4941300 7010316 2941132 37.31 2688 2238552 730776 8.18 267232 2299680 12 02:18:01 AM 4940996 7010060 2941436 37.32 2688 2238600 730776 8.18 267232 2299628 56 02:19:01 AM 4941092 7010312 2941340 37.32 2688 2238756 730776 8.18 267232 2299772 4 02:20:01 AM 4940888 7010108 2941544 37.32 2688 2238756 752976 8.43 267232 2299820 4 02:21:01 AM 4940748 7010036 2941684 37.32 2688 2238816 752976 8.43 267232 2299760 4 02:22:01 AM 4940900 7010384 2941532 37.32 2688 2239012 752976 8.43 267232 2299856 4 02:23:01 AM 4940872 7010356 2941560 37.32 2688 2239012 752976 8.43 267232 2299972 4 02:24:01 AM 4940696 7010240 2941736 37.32 2688 2239072 752976 8.43 267232 2300084 4 02:25:01 AM 4940720 7010504 2941712 37.32 2688 2239312 752976 8.43 267228 2299976 8 02:26:01 AM 4940660 7010744 2941772 37.32 2688 2239612 752976 8.43 267228 2300168 148 02:27:01 AM 4940152 7010600 2942280 37.33 2688 2239976 752976 8.43 267228 2300852 100 02:28:01 AM 4939668 7010460 2942764 37.33 2688 2240320 752976 8.43 267228 2301184 76 02:29:01 AM 4939436 7010580 2942996 37.34 2688 2240668 752976 8.43 267228 2301488 40 02:30:01 AM 4938964 7010464 2943468 37.34 2688 2241032 728040 8.15 267228 2301824 180 02:31:01 AM 4939012 7010856 2943420 37.34 2688 2241376 728040 8.15 267228 2301940 56 02:32:01 AM 4938216 7010712 2944216 37.35 2688 2242020 728040 8.15 267228 2302612 412 02:33:01 AM 4937908 7010508 2944524 37.36 2688 2242124 728040 8.15 267228 2302860 4 02:34:01 AM 4938176 7010776 2944256 37.35 2688 2242124 728040 8.15 267228 2302864 4 02:35:01 AM 4937936 7010616 2944496 37.36 2688 2242204 728040 8.15 267228 2302924 84 02:36:01 AM 4938148 7010828 2944284 37.35 2688 2242204 728040 8.15 267228 2302808 4 02:37:01 AM 4938044 7010796 2944388 37.35 2688 2242276 728040 8.15 267228 2302852 80 02:38:01 AM 4937272 7010028 2945160 37.36 2688 2242280 726828 8.14 267228 2303292 8 02:39:01 AM 4937624 7010388 2944808 37.36 2688 2242284 726824 8.14 276920 2293296 4 02:40:01 AM 4937212 7010200 2945220 37.36 2688 2242508 749968 8.40 276920 2293564 40 02:41:02 AM 4937424 7010572 2945008 37.36 2688 2242664 749968 8.40 276928 2293596 4 02:42:01 AM 4937448 7010596 2944984 37.36 2688 2242664 749968 8.40 276928 2293596 4 02:43:01 AM 4937504 7010696 2944928 37.36 2688 2242708 749968 8.40 276928 2293640 48 02:44:01 AM 4937048 7010312 2945384 37.37 2688 2242780 749968 8.40 276928 2293916 16 02:45:01 AM 4937204 7010556 2945228 37.36 2688 2242868 749968 8.40 276928 2293800 56 02:46:01 AM 4937120 7010536 2945312 37.37 2688 2242940 749968 8.40 276932 2293860 64 02:47:01 AM 4936876 7010436 2945556 37.37 2688 2243076 749968 8.40 276936 2294000 4 02:48:01 AM 4936764 7010324 2945668 37.37 2688 2243076 749968 8.40 276936 2294320 4 02:49:01 AM 4936888 7010588 2945544 37.37 2688 2243216 749968 8.40 276936 2294208 20 02:50:01 AM 4936276 7010356 2946156 37.38 2688 2243604 735004 8.23 276936 2294636 96 02:51:01 AM 4936248 7010704 2946184 37.38 2688 2243960 735004 8.23 276936 2294892 56 02:52:01 AM 4935664 7010464 2946768 37.38 2688 2244304 735004 8.23 276936 2295348 32 02:53:01 AM 4935392 7010552 2947040 37.39 2688 2244664 735004 8.23 276936 2295592 192 02:54:01 AM 4935076 7010584 2947356 37.39 2688 2245012 735004 8.23 276936 2296124 140 02:55:01 AM 4934728 7010584 2947704 37.40 2688 2245360 735004 8.23 276936 2296404 72 02:56:01 AM 4933996 7010444 2948436 37.41 2688 2245952 735004 8.23 276936 2297184 44 02:57:01 AM 4933952 7010500 2948480 37.41 2688 2246052 735004 8.23 276936 2297288 4 02:58:01 AM 4933568 7010120 2948864 37.41 2688 2246056 735004 8.23 276936 2297588 4 02:59:01 AM 4933764 7010396 2948668 37.41 2688 2246136 735004 8.23 276936 2297332 4 03:00:01 AM 4933492 7010124 2948940 37.41 2688 2246136 735316 8.23 276936 2297532 4 03:01:01 AM 4932600 7009308 2949832 37.42 2688 2246208 735316 8.23 276936 2298044 76 03:02:01 AM 4933448 7010156 2948984 37.41 2688 2246208 735316 8.23 276940 2297464 4 03:03:01 AM 4933556 7010268 2948876 37.41 2688 2246212 735316 8.23 276940 2297352 4 03:04:01 AM 4933120 7010056 2949312 37.42 2688 2246436 735316 8.23 276940 2297856 56 03:05:01 AM 4933136 7010224 2949296 37.42 2688 2246588 735316 8.23 276940 2297860 4 03:06:01 AM 4933288 7010380 2949144 37.41 2688 2246592 735316 8.23 276940 2297856 4 03:07:01 AM 4932708 7009840 2949724 37.42 2688 2246632 735316 8.23 276940 2298188 60 03:08:01 AM 4932676 7009896 2949756 37.42 2688 2246720 735316 8.23 276948 2298136 4 03:09:01 AM 4932836 7010160 2949596 37.42 2688 2246824 735316 8.23 276948 2298152 52 03:10:01 AM 4932464 7009852 2949968 37.42 2688 2246888 729420 8.17 276948 2298352 4 03:11:01 AM 4932544 7010084 2949888 37.42 2688 2247028 729420 8.17 276948 2298296 4 03:12:01 AM 4932652 7010192 2949780 37.42 2688 2247028 729420 8.17 276948 2298180 4 03:13:01 AM 4932732 7010416 2949700 37.42 2688 2247172 729420 8.17 276948 2298324 4 03:14:01 AM 4932120 7010204 2950312 37.43 2688 2247572 751632 8.42 276948 2298804 16 03:15:01 AM 4931928 7010364 2950504 37.43 2688 2247924 751632 8.42 276948 2299068 32 03:16:01 AM 4931480 7010264 2950952 37.44 2688 2248272 751632 8.42 276948 2299416 96 03:17:01 AM 4931032 7010164 2951400 37.44 2688 2248620 751632 8.42 276948 2300132 148 03:18:01 AM 4930716 7010200 2951716 37.45 2688 2248980 751632 8.42 276948 2300120 52 03:19:01 AM 4930392 7010224 2952040 37.45 2688 2249320 751632 8.42 276948 2300652 208 03:20:01 AM 4929592 7009988 2952840 37.46 2688 2249884 735268 8.23 276948 2301148 40 03:21:01 AM 4929928 7010396 2952504 37.46 2688 2249952 735268 8.23 276948 2301108 4 03:22:01 AM 4930036 7010504 2952396 37.46 2688 2249952 735268 8.23 276948 2301116 4 03:23:01 AM 4929652 7010204 2952780 37.46 2688 2250036 735268 8.23 276948 2301316 4 03:24:01 AM 4929672 7010224 2952760 37.46 2688 2250036 735268 8.23 276948 2301276 4 03:25:01 AM 4929504 7010128 2952928 37.46 2688 2250108 735268 8.23 276948 2301280 4 03:26:01 AM 4929596 7010224 2952836 37.46 2688 2250112 735268 8.23 276948 2301284 4 03:27:01 AM 4929068 7009868 2953364 37.47 2688 2250284 735268 8.23 276948 2301664 184 03:28:01 AM 4929224 7010072 2953208 37.47 2688 2250332 735268 8.23 276948 2301800 4 03:29:01 AM 4928980 7009980 2953452 37.47 2688 2250484 735268 8.23 276952 2301820 4 03:30:01 AM 4928600 7009600 2953832 37.47 2688 2250484 754872 8.45 276952 2301984 8 03:31:01 AM 4929088 7010152 2953344 37.47 2688 2250548 735400 8.23 276952 2301816 4 03:32:01 AM 4928980 7010044 2953452 37.47 2688 2250548 735400 8.23 276952 2301812 4 03:33:01 AM 4928636 7009888 2953796 37.47 2688 2250736 735400 8.23 276956 2302148 144 03:34:01 AM 4928644 7009900 2953788 37.47 2688 2250740 735400 8.23 276956 2302084 4 03:35:01 AM 4928844 7010204 2953588 37.47 2688 2250844 735400 8.23 276956 2302120 112 03:36:01 AM 4928572 7010120 2953860 37.47 2688 2251032 735400 8.23 276956 2302436 164 03:37:01 AM 4927848 7009756 2954584 37.48 2688 2251388 735400 8.23 276956 2302892 160 03:38:01 AM 4928008 7010260 2954424 37.48 2688 2251740 735400 8.23 276956 2303028 60 03:39:01 AM 4927284 7009876 2955148 37.49 2688 2252072 735400 8.23 276956 2303568 4 03:40:01 AM 4926788 7009744 2955644 37.50 2688 2252436 735332 8.23 276956 2303856 64 03:41:01 AM 4927044 7010348 2955388 37.49 2688 2252796 735332 8.23 276956 2304096 20 03:42:01 AM 4926320 7010064 2956112 37.50 2688 2253232 735332 8.23 276956 2304548 248 03:43:01 AM 4925724 7009924 2956708 37.51 2688 2253688 735332 8.23 276956 2305160 80 03:44:01 AM 4925856 7010056 2956576 37.51 2688 2253688 735332 8.23 276956 2305132 4 03:45:02 AM 4925936 7010140 2956496 37.51 2688 2253700 735332 8.23 276956 2305064 16 03:46:01 AM 4925748 7010032 2956684 37.51 2688 2253772 735332 8.23 276956 2305148 4 03:47:01 AM 4925844 7010128 2956588 37.51 2688 2253772 735332 8.23 276956 2305148 4 03:48:01 AM 4925900 7010256 2956532 37.51 2688 2253844 735332 8.23 276956 2305224 4 03:49:01 AM 4925672 7010028 2956760 37.51 2688 2253844 735332 8.23 276956 2305292 4 03:50:01 AM 4925056 7009588 2957376 37.52 2688 2254020 738792 8.27 276956 2305752 184 03:51:01 AM 4925328 7010032 2957104 37.52 2688 2254196 736152 8.24 276956 2305580 56 03:52:01 AM 4925152 7010112 2957280 37.52 2688 2254460 736164 8.24 276956 2305828 136 03:53:01 AM 4924728 7010056 2957704 37.52 2688 2254820 736164 8.24 276956 2306196 96 03:54:01 AM 4924552 7010236 2957880 37.52 2688 2255172 736164 8.24 276956 2306620 64 03:55:01 AM 4923840 7009872 2958592 37.53 2688 2255528 736164 8.24 276956 2307020 24 03:56:01 AM 4923612 7010000 2958820 37.54 2688 2255876 736164 8.24 276956 2307256 160 03:57:01 AM 4923468 7010208 2958964 37.54 2688 2256228 736164 8.24 276956 2307608 32 03:58:01 AM 4922716 7010116 2959716 37.55 2688 2256888 736164 8.24 276956 2308304 296 03:59:01 AM 4922404 7009924 2960028 37.55 2688 2257008 736164 8.24 276956 2308752 4 04:00:01 AM 4922400 7009920 2960032 37.55 2688 2257008 735348 8.23 276956 2308448 4 04:01:01 AM 4921752 7009348 2960680 37.56 2688 2257084 735348 8.23 276956 2309104 84 04:02:01 AM 4922640 7010240 2959792 37.55 2688 2257088 735348 8.23 276960 2308524 4 04:03:01 AM 4922672 7010280 2959760 37.55 2688 2257096 735348 8.23 276960 2308528 16 04:04:01 AM 4922036 7009708 2960396 37.56 2688 2257160 735348 8.23 276960 2308668 4 04:05:01 AM 4922152 7009828 2960280 37.56 2688 2257164 735348 8.23 276960 2308604 4 04:06:01 AM 4924252 7012120 2958180 37.53 2688 2257356 735348 8.23 276960 2306752 24 04:07:01 AM 4921984 7010012 2960448 37.56 2688 2257524 718580 8.05 276960 2308960 24 04:08:01 AM 4921756 7010136 2960676 37.56 2688 2257868 735104 8.23 276960 2309420 48 04:09:01 AM 4921352 7010084 2961080 37.57 2688 2258228 735104 8.23 276960 2309724 24 04:10:01 AM 4920724 7009816 2961708 37.57 2688 2258588 735332 8.23 276960 2310016 16 04:11:01 AM 4920476 7009936 2961956 37.58 2688 2258944 735332 8.23 276968 2310468 128 04:12:01 AM 4920124 7009952 2962308 37.58 2688 2259316 736068 8.24 277788 2309936 40 04:13:01 AM 4919680 7009860 2962752 37.59 2688 2259660 736068 8.24 277788 2310284 188 04:14:01 AM 4919232 7010012 2963200 37.59 2688 2260260 736068 8.24 277788 2310952 40 04:15:01 AM 4919012 7009876 2963420 37.60 2688 2260344 736068 8.24 277788 2310936 4 04:16:01 AM 4919108 7009976 2963324 37.59 2688 2260348 736068 8.24 277796 2311096 4 04:17:01 AM 4919256 7010200 2963176 37.59 2688 2260424 736068 8.24 277788 2311060 4 04:18:01 AM 4919116 7010060 2963316 37.59 2688 2260424 736068 8.24 277796 2311156 4 04:19:01 AM 4919068 7010088 2963364 37.59 2688 2260500 736068 8.24 277788 2311324 80 04:20:01 AM 4918832 7009852 2963600 37.60 2688 2260500 727156 8.14 277788 2311372 4 04:21:01 AM 4918268 7009420 2964164 37.60 2688 2260632 727156 8.14 277788 2311844 136 04:22:01 AM 4917688 7008928 2964744 37.61 2688 2260720 727156 8.14 277788 2312280 48 04:23:01 AM 4916568 7008008 2965864 37.63 2688 2260928 742928 8.32 277788 2313424 72 04:24:01 AM 4916116 7007940 2966316 37.63 2688 2261312 742928 8.32 277788 2313996 24 04:25:01 AM 4916864 7009040 2965568 37.62 2688 2261656 742928 8.32 277788 2313220 152 04:26:01 AM 4916540 7009068 2965892 37.63 2688 2262008 742928 8.32 277788 2313464 16 04:27:01 AM 4916276 7009160 2966156 37.63 2688 2262368 742928 8.32 277788 2313864 176 04:28:01 AM 4915664 7008908 2966768 37.64 2688 2262728 742928 8.32 277788 2314272 136 04:29:01 AM 4915404 7009004 2967028 37.64 2688 2263076 742928 8.32 277788 2314784 64 04:30:01 AM 4914536 7008656 2967896 37.65 2688 2263596 737400 8.26 277788 2315284 52 04:31:01 AM 4914736 7008928 2967696 37.65 2688 2263668 737400 8.26 277788 2315252 4 04:32:01 AM 4914804 7009000 2967628 37.65 2688 2263672 737400 8.26 277788 2315372 4 04:33:01 AM 4914640 7008916 2967792 37.65 2688 2263752 737400 8.26 277788 2315540 4 04:34:01 AM 4914556 7008832 2967876 37.65 2688 2263752 737400 8.26 277788 2315532 4 04:35:01 AM 4914656 7009004 2967776 37.65 2688 2263824 737400 8.26 277788 2315420 4 04:36:01 AM 4914712 7009060 2967720 37.65 2688 2263824 737400 8.26 277788 2315420 8 04:37:01 AM 4914176 7008700 2968256 37.66 2688 2264000 737400 8.26 277788 2315704 184 04:38:01 AM 4914108 7008824 2968324 37.66 2688 2264192 756440 8.47 277788 2315876 4 04:39:01 AM 4911304 7006216 2971128 37.69 2688 2264388 756440 8.47 277792 2318400 4 04:40:01 AM 4910188 7005288 2972244 37.71 2688 2264568 812736 9.10 277792 2318896 4 04:41:01 AM 4907284 7002752 2975148 37.74 2688 2264936 846764 9.48 277792 2321868 4 04:42:01 AM 4906148 7002332 2976284 37.76 2688 2265652 846764 9.48 277792 2322692 296 04:43:01 AM 4903228 6999812 2979204 37.80 2688 2266044 886428 9.93 277792 2325076 336 04:44:01 AM 4967720 7068560 2914712 36.98 2688 2270304 639148 7.16 278652 2261504 3976 04:45:01 AM 4983624 6976476 2898808 36.78 2688 2168372 686452 7.69 500124 2047508 14248 Average: 4944174 7011931 2938258 37.28 2688 2237486 737457 8.26 272121 2292383 1288 12:23:02 AM IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s %ifutil 12:24:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:24:01 AM eth0 285.72 185.24 1586.71 58.92 0.00 0.00 0.00 0.00 12:25:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:25:01 AM eth0 58.63 47.80 505.62 8.54 0.00 0.00 0.00 0.00 12:26:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:26:01 AM eth0 19.61 9.42 5.88 3.44 0.00 0.00 0.00 0.00 12:27:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:27:01 AM eth0 43.91 34.46 270.39 7.39 0.00 0.00 0.00 0.00 12:28:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:28:01 AM eth0 596.80 510.40 124.51 109.51 0.00 0.00 0.00 0.00 12:29:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:29:01 AM eth0 29.44 22.48 14.15 3.96 0.00 0.00 0.00 0.00 12:30:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:30:01 AM eth0 11.73 10.21 2.30 1.89 0.00 0.00 0.00 0.00 12:31:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:31:01 AM eth0 10.15 9.65 2.06 1.81 0.00 0.00 0.00 0.00 12:32:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:32:01 AM eth0 10.61 9.78 2.12 1.81 0.00 0.00 0.00 0.00 12:33:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:33:01 AM eth0 10.55 10.13 2.71 2.26 0.00 0.00 0.00 0.00 12:34:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:34:01 AM eth0 9.83 9.57 2.07 1.85 0.00 0.00 0.00 0.00 12:35:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:35:01 AM eth0 9.78 9.58 2.02 1.77 0.00 0.00 0.00 0.00 12:36:02 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:36:02 AM eth0 13.18 12.63 2.91 2.27 0.00 0.00 0.00 0.00 12:37:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:37:01 AM eth0 3.93 0.44 0.72 0.22 0.00 0.00 0.00 0.00 12:38:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:38:01 AM eth0 1.80 0.25 0.32 0.20 0.00 0.00 0.00 0.00 12:39:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:39:01 AM eth0 4.22 0.48 0.51 0.11 0.00 0.00 0.00 0.00 12:40:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:40:01 AM eth0 5.56 0.17 0.60 0.01 0.00 0.00 0.00 0.00 12:41:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:41:01 AM eth0 2.40 0.57 0.44 0.24 0.00 0.00 0.00 0.00 12:42:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:42:01 AM eth0 1.23 0.05 0.14 0.00 0.00 0.00 0.00 0.00 12:43:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:43:01 AM eth0 9.23 8.50 2.11 1.35 0.00 0.00 0.00 0.00 12:44:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:44:01 AM eth0 3.35 2.97 0.65 0.30 0.00 0.00 0.00 0.00 12:45:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:45:01 AM eth0 1.87 1.47 0.17 0.17 0.00 0.00 0.00 0.00 12:46:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:46:01 AM eth0 1.63 0.42 0.48 0.27 0.00 0.00 0.00 0.00 12:47:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:47:01 AM eth0 7.20 3.20 1.56 0.70 0.00 0.00 0.00 0.00 12:48:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:48:01 AM eth0 3.87 0.47 0.66 0.41 0.00 0.00 0.00 0.00 12:49:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:49:01 AM eth0 4.43 0.18 0.48 0.07 0.00 0.00 0.00 0.00 12:50:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:50:01 AM eth0 8.70 6.78 1.44 0.53 0.00 0.00 0.00 0.00 12:51:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:51:01 AM eth0 4.27 3.78 0.69 0.35 0.00 0.00 0.00 0.00 12:52:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:52:01 AM eth0 8.38 8.12 1.26 0.61 0.00 0.00 0.00 0.00 12:53:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:53:01 AM eth0 0.73 0.57 0.09 0.09 0.00 0.00 0.00 0.00 12:54:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:54:01 AM eth0 1.02 0.10 0.19 0.07 0.00 0.00 0.00 0.00 12:55:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:55:01 AM eth0 4.35 4.05 0.62 0.29 0.00 0.00 0.00 0.00 12:56:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:56:01 AM eth0 1.28 0.60 0.11 0.10 0.00 0.00 0.00 0.00 12:57:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:57:01 AM eth0 4.13 0.25 0.68 0.20 0.00 0.00 0.00 0.00 12:58:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:58:01 AM eth0 7.47 7.15 1.38 0.77 0.00 0.00 0.00 0.00 12:59:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:59:01 AM eth0 11.15 9.76 2.32 1.87 0.00 0.00 0.00 0.00 01:00:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:00:01 AM eth0 12.01 9.75 2.18 1.81 0.00 0.00 0.00 0.00 01:01:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:01:01 AM eth0 15.69 10.18 2.75 7.21 0.00 0.00 0.00 0.00 01:02:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:02:01 AM eth0 10.32 9.64 2.09 1.78 0.00 0.00 0.00 0.00 01:03:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:03:01 AM eth0 10.15 9.75 2.05 1.81 0.00 0.00 0.00 0.00 01:04:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:04:01 AM eth0 10.48 9.80 2.14 1.87 0.00 0.00 0.00 0.00 01:05:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:05:01 AM eth0 15.10 14.76 3.35 2.72 0.00 0.00 0.00 0.00 01:06:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:06:01 AM eth0 1.12 0.60 0.18 0.07 0.00 0.00 0.00 0.00 01:07:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:07:01 AM eth0 3.65 0.18 0.50 0.07 0.00 0.00 0.00 0.00 01:08:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:08:01 AM eth0 3.90 0.75 1.31 0.77 0.00 0.00 0.00 0.00 01:09:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:09:01 AM eth0 1.35 0.58 0.52 0.36 0.00 0.00 0.00 0.00 01:10:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:10:01 AM eth0 4.32 0.08 0.60 0.01 0.00 0.00 0.00 0.00 01:11:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:11:01 AM eth0 3.60 0.38 0.37 0.05 0.00 0.00 0.00 0.00 01:12:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:12:01 AM eth0 0.58 0.00 0.03 0.00 0.00 0.00 0.00 0.00 01:13:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:13:01 AM eth0 7.42 6.97 1.57 0.89 0.00 0.00 0.00 0.00 01:14:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:14:01 AM eth0 3.82 3.30 0.73 0.36 0.00 0.00 0.00 0.00 01:15:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:15:01 AM eth0 3.57 3.22 0.65 0.30 0.00 0.00 0.00 0.00 01:16:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:16:01 AM eth0 0.53 0.13 0.08 0.01 0.00 0.00 0.00 0.00 01:17:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:17:01 AM eth0 2.67 2.48 0.51 0.19 0.00 0.00 0.00 0.00 01:18:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:18:01 AM eth0 0.58 0.35 0.06 0.07 0.00 0.00 0.00 0.00 01:19:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:19:01 AM eth0 0.38 0.15 0.09 0.07 0.00 0.00 0.00 0.00 01:20:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:20:01 AM eth0 3.90 3.68 0.63 0.27 0.00 0.00 0.00 0.00 01:21:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:21:01 AM eth0 1.22 0.57 0.15 0.09 0.00 0.00 0.00 0.00 01:22:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:22:01 AM eth0 1.28 0.23 0.45 0.19 0.00 0.00 0.00 0.00 01:23:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:23:01 AM eth0 11.05 8.78 1.48 0.66 0.00 0.00 0.00 0.00 01:24:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:24:01 AM eth0 0.98 0.63 0.17 0.16 0.00 0.00 0.00 0.00 01:25:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:25:01 AM eth0 0.15 0.00 0.01 0.00 0.00 0.00 0.00 0.00 01:26:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:26:01 AM eth0 3.68 3.43 0.62 0.25 0.00 0.00 0.00 0.00 01:27:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:27:01 AM eth0 8.85 8.33 1.47 0.89 0.00 0.00 0.00 0.00 01:28:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:28:01 AM eth0 8.63 8.73 2.01 1.78 0.00 0.00 0.00 0.00 01:29:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:29:01 AM eth0 10.11 9.98 2.43 2.07 0.00 0.00 0.00 0.00 01:30:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:30:01 AM eth0 9.43 9.65 2.02 1.80 0.00 0.00 0.00 0.00 01:31:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:31:01 AM eth0 9.83 9.38 2.16 1.74 0.00 0.00 0.00 0.00 01:32:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:32:01 AM eth0 10.75 9.90 2.38 2.00 0.00 0.00 0.00 0.00 01:33:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:33:01 AM eth0 9.68 9.78 2.27 1.99 0.00 0.00 0.00 0.00 01:34:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:34:01 AM eth0 16.23 16.01 3.66 2.99 0.00 0.00 0.00 0.00 01:35:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:35:01 AM eth0 1.12 0.42 0.21 0.06 0.00 0.00 0.00 0.00 01:36:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:36:01 AM eth0 4.30 0.10 0.57 0.01 0.00 0.00 0.00 0.00 01:37:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:37:01 AM eth0 0.88 0.50 0.33 0.24 0.00 0.00 0.00 0.00 01:38:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:38:01 AM eth0 0.40 0.00 0.02 0.00 0.00 0.00 0.00 0.00 01:39:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:39:01 AM eth0 0.60 0.33 0.15 0.10 0.00 0.00 0.00 0.00 01:40:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:40:01 AM eth0 4.42 0.14 0.54 0.01 0.00 0.00 0.00 0.00 01:41:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:41:01 AM eth0 3.88 0.47 0.82 0.45 0.00 0.00 0.00 0.00 01:42:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:42:01 AM eth0 11.71 10.58 2.18 1.18 0.00 0.00 0.00 0.00 01:43:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:43:01 AM eth0 5.76 0.68 0.77 0.12 0.00 0.00 0.00 0.00 01:44:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:44:01 AM eth0 2.17 0.17 0.24 0.07 0.00 0.00 0.00 0.00 01:45:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:45:01 AM eth0 2.78 2.32 0.51 0.17 0.00 0.00 0.00 0.00 01:46:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:46:01 AM eth0 1.10 0.52 0.10 0.10 0.00 0.00 0.00 0.00 01:47:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:47:01 AM eth0 2.05 0.05 0.22 0.00 0.00 0.00 0.00 0.00 01:48:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:48:01 AM eth0 3.93 3.38 1.10 0.60 0.00 0.00 0.00 0.00 01:49:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:49:01 AM eth0 8.07 7.33 1.66 0.85 0.00 0.00 0.00 0.00 01:50:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:50:01 AM eth0 6.30 4.57 0.87 0.36 0.00 0.00 0.00 0.00 01:51:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:51:01 AM eth0 5.30 5.11 0.77 0.47 0.00 0.00 0.00 0.00 01:52:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:52:01 AM eth0 0.53 0.03 0.05 0.00 0.00 0.00 0.00 0.00 01:53:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:53:01 AM eth0 3.25 3.05 0.61 0.22 0.00 0.00 0.00 0.00 01:54:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:54:01 AM eth0 1.35 0.80 0.24 0.18 0.00 0.00 0.00 0.00 01:55:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:55:01 AM eth0 0.18 0.00 0.01 0.00 0.00 0.00 0.00 0.00 01:56:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:56:01 AM eth0 4.53 4.07 1.13 0.57 0.00 0.00 0.00 0.00 01:57:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:57:01 AM eth0 9.20 5.15 1.78 1.05 0.00 0.00 0.00 0.00 01:58:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:58:01 AM eth0 10.65 9.67 2.73 2.30 0.00 0.00 0.00 0.00 01:59:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:59:01 AM eth0 8.60 8.55 1.82 1.61 0.00 0.00 0.00 0.00 02:00:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:00:01 AM eth0 14.90 8.41 2.74 1.55 0.00 0.00 0.00 0.00 02:01:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:01:01 AM eth0 12.63 8.80 2.27 1.62 0.00 0.00 0.00 0.00 02:02:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:02:01 AM eth0 9.90 9.23 1.95 1.69 0.00 0.00 0.00 0.00 02:03:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:03:01 AM eth0 14.41 14.08 3.52 2.82 0.00 0.00 0.00 0.00 02:04:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:04:01 AM eth0 7.02 5.97 1.22 1.04 0.00 0.00 0.00 0.00 02:05:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:05:01 AM eth0 1.20 0.18 0.23 0.07 0.00 0.00 0.00 0.00 02:06:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:06:01 AM eth0 0.60 0.17 0.27 0.19 0.00 0.00 0.00 0.00 02:07:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:07:01 AM eth0 3.68 0.30 0.43 0.04 0.00 0.00 0.00 0.00 02:08:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:08:01 AM eth0 1.90 0.80 1.05 0.77 0.00 0.00 0.00 0.00 02:09:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:09:01 AM eth0 7.51 0.43 32.97 0.11 0.00 0.00 0.00 0.00 02:10:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:10:01 AM eth0 27.58 10.75 15.33 9.57 0.00 0.00 0.00 0.00 02:11:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:11:01 AM eth0 53.57 34.48 29.62 68.51 0.00 0.00 0.00 0.00 02:12:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:12:01 AM eth0 4.65 4.13 0.96 0.54 0.00 0.00 0.00 0.00 02:13:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:13:01 AM eth0 4.05 3.78 0.75 0.36 0.00 0.00 0.00 0.00 02:14:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:14:01 AM eth0 0.87 0.17 0.12 0.07 0.00 0.00 0.00 0.00 02:15:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:15:01 AM eth0 3.88 3.60 0.60 0.26 0.00 0.00 0.00 0.00 02:16:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:16:01 AM eth0 1.00 0.43 0.09 0.08 0.00 0.00 0.00 0.00 02:17:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:17:01 AM eth0 3.28 0.23 0.58 0.20 0.00 0.00 0.00 0.00 02:18:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:18:01 AM eth0 3.85 3.18 0.91 0.41 0.00 0.00 0.00 0.00 02:19:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:19:01 AM eth0 3.80 0.68 0.28 0.17 0.00 0.00 0.00 0.00 02:20:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:20:01 AM eth0 8.30 0.00 0.96 0.00 0.00 0.00 0.00 0.00 02:21:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:21:01 AM eth0 6.08 3.77 6.11 0.31 0.00 0.00 0.00 0.00 02:22:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:22:01 AM eth0 5.33 4.92 0.82 0.41 0.00 0.00 0.00 0.00 02:23:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:23:01 AM eth0 0.25 0.00 0.01 0.00 0.00 0.00 0.00 0.00 02:24:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:24:01 AM eth0 4.68 4.07 0.73 0.34 0.00 0.00 0.00 0.00 02:25:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:25:01 AM eth0 8.68 8.60 1.43 0.91 0.00 0.00 0.00 0.00 02:26:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:26:01 AM eth0 8.60 8.36 1.71 1.53 0.00 0.00 0.00 0.00 02:27:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:27:01 AM eth0 12.48 10.01 2.71 2.08 0.00 0.00 0.00 0.00 02:28:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:28:01 AM eth0 10.40 10.10 2.63 2.26 0.00 0.00 0.00 0.00 02:29:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:29:01 AM eth0 11.06 9.58 2.51 1.99 0.00 0.00 0.00 0.00 02:30:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:30:01 AM eth0 13.71 9.68 2.48 1.81 0.00 0.00 0.00 0.00 02:31:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:31:01 AM eth0 10.16 9.35 2.03 1.74 0.00 0.00 0.00 0.00 02:32:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:32:01 AM eth0 17.18 16.31 3.73 3.00 0.00 0.00 0.00 0.00 02:33:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:33:01 AM eth0 1.83 0.82 0.52 0.33 0.00 0.00 0.00 0.00 02:34:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:34:01 AM eth0 0.93 0.27 0.41 0.26 0.00 0.00 0.00 0.00 02:35:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:35:01 AM eth0 0.70 0.35 0.08 0.05 0.00 0.00 0.00 0.00 02:36:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:36:01 AM eth0 4.68 0.00 0.30 0.00 0.00 0.00 0.00 0.00 02:37:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:37:01 AM eth0 1.93 0.23 0.25 0.04 0.00 0.00 0.00 0.00 02:38:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:38:01 AM eth0 1.18 0.50 0.67 0.45 0.00 0.00 0.00 0.00 02:39:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:39:01 AM eth0 0.75 0.13 0.13 0.07 0.00 0.00 0.00 0.00 02:40:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:40:01 AM eth0 12.73 11.45 2.34 1.23 0.00 0.00 0.00 0.00 02:41:02 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:41:02 AM eth0 0.85 0.65 0.09 0.10 0.00 0.00 0.00 0.00 02:42:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:42:01 AM eth0 1.59 0.03 0.10 0.00 0.00 0.00 0.00 0.00 02:43:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:43:01 AM eth0 2.60 2.43 0.54 0.18 0.00 0.00 0.00 0.00 02:44:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:44:01 AM eth0 3.87 2.83 1.02 0.53 0.00 0.00 0.00 0.00 02:45:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:45:01 AM eth0 7.18 6.85 1.28 0.54 0.00 0.00 0.00 0.00 02:46:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:46:01 AM eth0 5.36 5.06 0.79 0.48 0.00 0.00 0.00 0.00 02:47:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:47:01 AM eth0 0.82 0.23 0.08 0.03 0.00 0.00 0.00 0.00 02:48:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:48:01 AM eth0 0.70 0.27 0.29 0.20 0.00 0.00 0.00 0.00 02:49:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:49:01 AM eth0 7.26 6.68 1.46 0.84 0.00 0.00 0.00 0.00 02:50:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:50:01 AM eth0 10.08 10.20 2.12 1.89 0.00 0.00 0.00 0.00 02:51:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:51:01 AM eth0 9.58 9.70 2.03 1.81 0.00 0.00 0.00 0.00 02:52:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:52:01 AM eth0 9.28 9.35 1.98 1.76 0.00 0.00 0.00 0.00 02:53:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:53:01 AM eth0 9.53 9.63 2.00 1.78 0.00 0.00 0.00 0.00 02:54:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:54:01 AM eth0 9.95 9.82 2.14 1.87 0.00 0.00 0.00 0.00 02:55:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:55:01 AM eth0 11.46 9.65 2.19 1.81 0.00 0.00 0.00 0.00 02:56:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:56:01 AM eth0 14.58 13.88 3.26 2.57 0.00 0.00 0.00 0.00 02:57:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:57:01 AM eth0 1.03 0.35 0.15 0.06 0.00 0.00 0.00 0.00 02:58:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:58:01 AM eth0 0.67 0.22 0.27 0.19 0.00 0.00 0.00 0.00 02:59:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:59:01 AM eth0 1.50 0.43 0.27 0.11 0.00 0.00 0.00 0.00 03:00:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:00:01 AM eth0 1.73 0.28 0.49 0.20 0.00 0.00 0.00 0.00 03:01:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:01:01 AM eth0 1.03 0.27 0.13 0.04 0.00 0.00 0.00 0.00 03:02:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:02:01 AM eth0 0.37 0.05 0.02 0.00 0.00 0.00 0.00 0.00 03:03:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:03:01 AM eth0 3.65 0.08 0.44 0.01 0.00 0.00 0.00 0.00 03:04:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:04:01 AM eth0 10.63 10.40 2.41 1.41 0.00 0.00 0.00 0.00 03:05:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:05:01 AM eth0 1.02 0.70 0.10 0.11 0.00 0.00 0.00 0.00 03:06:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:06:01 AM eth0 5.32 0.03 0.68 0.00 0.00 0.00 0.00 0.00 03:07:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:07:01 AM eth0 5.02 2.48 1.07 0.37 0.00 0.00 0.00 0.00 03:08:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:08:01 AM eth0 3.95 3.37 0.91 0.51 0.00 0.00 0.00 0.00 03:09:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:09:01 AM eth0 9.03 8.05 1.37 0.66 0.00 0.00 0.00 0.00 03:10:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:10:01 AM eth0 7.07 4.42 1.34 0.64 0.00 0.00 0.00 0.00 03:11:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:11:01 AM eth0 1.52 0.35 0.17 0.04 0.00 0.00 0.00 0.00 03:12:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:12:01 AM eth0 1.52 0.00 0.11 0.00 0.00 0.00 0.00 0.00 03:13:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:13:01 AM eth0 9.93 8.25 1.62 0.88 0.00 0.00 0.00 0.00 03:14:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:14:01 AM eth0 11.25 11.06 2.38 2.11 0.00 0.00 0.00 0.00 03:15:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:15:01 AM eth0 9.13 9.27 1.94 1.74 0.00 0.00 0.00 0.00 03:16:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:16:01 AM eth0 9.76 9.70 2.03 1.81 0.00 0.00 0.00 0.00 03:17:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:17:01 AM eth0 12.50 9.62 2.66 1.94 0.00 0.00 0.00 0.00 03:18:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:18:01 AM eth0 9.70 9.80 2.04 1.82 0.00 0.00 0.00 0.00 03:19:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:19:01 AM eth0 11.40 9.38 2.34 1.83 0.00 0.00 0.00 0.00 03:20:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:20:01 AM eth0 16.61 13.43 3.39 2.45 0.00 0.00 0.00 0.00 03:21:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:21:01 AM eth0 0.93 0.33 0.10 0.05 0.00 0.00 0.00 0.00 03:22:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:22:01 AM eth0 0.58 0.10 0.08 0.01 0.00 0.00 0.00 0.00 03:23:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:23:01 AM eth0 0.68 0.33 0.08 0.04 0.00 0.00 0.00 0.00 03:24:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:24:01 AM eth0 0.55 0.10 0.10 0.06 0.00 0.00 0.00 0.00 03:25:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:25:01 AM eth0 0.75 0.30 0.09 0.04 0.00 0.00 0.00 0.00 03:26:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:26:01 AM eth0 0.88 0.00 0.13 0.00 0.00 0.00 0.00 0.00 03:27:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:27:01 AM eth0 9.45 6.58 1.83 0.87 0.00 0.00 0.00 0.00 03:28:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:28:01 AM eth0 3.52 3.02 0.84 0.43 0.00 0.00 0.00 0.00 03:29:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:29:01 AM eth0 1.78 0.85 0.53 0.35 0.00 0.00 0.00 0.00 03:30:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:30:01 AM eth0 5.83 0.10 0.77 0.00 0.00 0.00 0.00 0.00 03:31:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:31:01 AM eth0 2.80 2.10 0.53 0.20 0.00 0.00 0.00 0.00 03:32:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:32:01 AM eth0 1.75 0.05 0.24 0.00 0.00 0.00 0.00 0.00 03:33:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:33:01 AM eth0 4.18 3.63 1.02 0.56 0.00 0.00 0.00 0.00 03:34:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:34:01 AM eth0 0.85 0.13 0.12 0.10 0.00 0.00 0.00 0.00 03:35:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:35:01 AM eth0 2.58 2.50 0.49 0.45 0.00 0.00 0.00 0.00 03:36:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:36:01 AM eth0 11.23 10.21 1.91 1.19 0.00 0.00 0.00 0.00 03:37:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:37:01 AM eth0 10.13 9.68 2.34 1.96 0.00 0.00 0.00 0.00 03:38:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:38:01 AM eth0 10.03 9.52 2.06 1.78 0.00 0.00 0.00 0.00 03:39:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:39:01 AM eth0 9.38 9.40 2.02 1.80 0.00 0.00 0.00 0.00 03:40:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:40:01 AM eth0 10.56 9.73 2.14 1.81 0.00 0.00 0.00 0.00 03:41:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:41:01 AM eth0 9.70 9.76 2.03 1.81 0.00 0.00 0.00 0.00 03:42:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:42:01 AM eth0 12.03 11.75 2.55 2.24 0.00 0.00 0.00 0.00 03:43:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:43:01 AM eth0 9.48 8.97 2.15 1.55 0.00 0.00 0.00 0.00 03:44:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:44:01 AM eth0 0.55 0.12 0.10 0.06 0.00 0.00 0.00 0.00 03:45:02 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:45:02 AM eth0 0.30 0.02 0.01 0.00 0.00 0.00 0.00 0.00 03:46:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:46:01 AM eth0 0.75 0.37 0.19 0.05 0.00 0.00 0.00 0.00 03:47:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:47:01 AM eth0 0.22 0.00 0.01 0.00 0.00 0.00 0.00 0.00 03:48:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:48:01 AM eth0 4.05 0.28 0.33 0.04 0.00 0.00 0.00 0.00 03:49:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:49:01 AM eth0 0.42 0.10 0.09 0.06 0.00 0.00 0.00 0.00 03:50:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:50:01 AM eth0 7.68 7.53 1.51 0.92 0.00 0.00 0.00 0.00 03:51:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:51:01 AM eth0 7.58 7.50 1.74 1.09 0.00 0.00 0.00 0.00 03:52:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:52:01 AM eth0 7.08 7.21 1.47 1.34 0.00 0.00 0.00 0.00 03:53:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:53:01 AM eth0 9.26 9.53 1.98 1.78 0.00 0.00 0.00 0.00 03:54:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:54:01 AM eth0 9.58 9.70 2.09 1.87 0.00 0.00 0.00 0.00 03:55:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:55:01 AM eth0 9.60 9.80 2.03 1.82 0.00 0.00 0.00 0.00 03:56:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:56:01 AM eth0 9.31 9.45 1.98 1.77 0.00 0.00 0.00 0.00 03:57:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:57:01 AM eth0 9.43 9.60 1.99 1.78 0.00 0.00 0.00 0.00 03:58:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:58:01 AM eth0 18.14 16.99 3.99 3.14 0.00 0.00 0.00 0.00 03:59:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:59:01 AM eth0 1.02 0.67 0.43 0.32 0.00 0.00 0.00 0.00 04:00:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:00:01 AM eth0 0.33 0.00 0.01 0.00 0.00 0.00 0.00 0.00 04:01:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:01:01 AM eth0 4.05 0.42 5.90 0.05 0.00 0.00 0.00 0.00 04:02:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:02:01 AM eth0 0.25 0.00 0.01 0.00 0.00 0.00 0.00 0.00 04:03:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:03:01 AM eth0 0.35 0.05 0.02 0.00 0.00 0.00 0.00 0.00 04:04:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:04:01 AM eth0 0.62 0.37 0.15 0.10 0.00 0.00 0.00 0.00 04:05:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:05:01 AM eth0 0.15 0.00 0.01 0.00 0.00 0.00 0.00 0.00 04:06:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:06:01 AM eth0 7.38 7.48 1.53 0.96 0.00 0.00 0.00 0.00 04:07:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:07:01 AM eth0 6.47 6.47 1.56 0.94 0.00 0.00 0.00 0.00 04:08:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:08:01 AM eth0 9.85 9.95 2.05 1.83 0.00 0.00 0.00 0.00 04:09:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:09:01 AM eth0 9.26 9.33 2.02 1.80 0.00 0.00 0.00 0.00 04:10:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:10:01 AM eth0 9.53 9.68 2.02 1.81 0.00 0.00 0.00 0.00 04:11:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:11:01 AM eth0 9.50 9.65 2.13 1.81 0.00 0.00 0.00 0.00 04:12:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:12:01 AM eth0 10.38 10.41 2.09 1.87 0.00 0.00 0.00 0.00 04:13:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:13:01 AM eth0 9.35 9.46 2.01 1.79 0.00 0.00 0.00 0.00 04:14:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:14:01 AM eth0 14.85 14.53 3.28 2.72 0.00 0.00 0.00 0.00 04:15:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:15:01 AM eth0 0.57 0.35 0.09 0.05 0.00 0.00 0.00 0.00 04:16:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:16:01 AM eth0 0.57 0.10 0.15 0.01 0.00 0.00 0.00 0.00 04:17:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:17:01 AM eth0 0.73 0.33 0.13 0.04 0.00 0.00 0.00 0.00 04:18:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:18:01 AM eth0 0.20 0.00 0.01 0.00 0.00 0.00 0.00 0.00 04:19:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:19:01 AM eth0 0.57 0.42 0.14 0.11 0.00 0.00 0.00 0.00 04:20:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:20:01 AM eth0 0.28 0.00 0.01 0.00 0.00 0.00 0.00 0.00 04:21:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:21:01 AM eth0 2.73 2.82 0.62 0.51 0.00 0.00 0.00 0.00 04:22:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:22:01 AM eth0 6.30 6.32 1.26 0.68 0.00 0.00 0.00 0.00 04:23:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:23:01 AM eth0 7.66 7.81 1.62 1.10 0.00 0.00 0.00 0.00 04:24:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:24:01 AM eth0 10.50 10.63 2.27 2.02 0.00 0.00 0.00 0.00 04:25:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:25:01 AM eth0 9.15 9.21 1.97 1.75 0.00 0.00 0.00 0.00 04:26:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:26:01 AM eth0 9.85 9.43 2.10 1.77 0.00 0.00 0.00 0.00 04:27:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:27:01 AM eth0 9.50 9.72 2.02 1.81 0.00 0.00 0.00 0.00 04:28:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:28:01 AM eth0 9.38 9.52 2.01 1.80 0.00 0.00 0.00 0.00 04:29:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:29:01 AM eth0 10.06 10.05 2.49 2.07 0.00 0.00 0.00 0.00 04:30:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:30:01 AM eth0 12.36 11.91 2.71 2.21 0.00 0.00 0.00 0.00 04:31:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:31:01 AM eth0 0.37 0.32 0.07 0.04 0.00 0.00 0.00 0.00 04:32:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:32:01 AM eth0 0.73 0.00 0.07 0.00 0.00 0.00 0.00 0.00 04:33:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:33:01 AM eth0 0.73 0.55 0.32 0.24 0.00 0.00 0.00 0.00 04:34:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:34:01 AM eth0 0.42 0.10 0.09 0.06 0.00 0.00 0.00 0.00 04:35:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:35:01 AM eth0 0.45 0.30 0.07 0.04 0.00 0.00 0.00 0.00 04:36:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:36:01 AM eth0 0.38 0.10 0.13 0.01 0.00 0.00 0.00 0.00 04:37:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:37:01 AM eth0 7.18 7.17 1.48 0.90 0.00 0.00 0.00 0.00 04:38:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:38:01 AM eth0 6.38 6.23 1.38 0.89 0.00 0.00 0.00 0.00 04:39:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:39:01 AM eth0 6.72 6.62 1.39 0.93 0.00 0.00 0.00 0.00 04:40:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:40:01 AM eth0 5.48 5.52 1.20 0.73 0.00 0.00 0.00 0.00 04:41:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:41:01 AM eth0 6.46 6.43 3.32 0.93 0.00 0.00 0.00 0.00 04:42:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:42:01 AM eth0 10.25 10.28 5.97 1.80 0.00 0.00 0.00 0.00 04:43:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:43:01 AM eth0 7.22 6.75 3.50 0.93 0.00 0.00 0.00 0.00 04:44:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:44:01 AM eth0 24.34 19.04 13.56 2.59 0.00 0.00 0.00 0.00 04:45:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 04:45:01 AM eth0 85.72 65.64 261.77 219.87 0.00 0.00 0.00 0.00 Average: lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: eth0 10.15 8.08 11.98 2.67 0.00 0.00 0.00 0.00 ---> sar -P ALL: Linux 4.18.0-553.5.1.el8.x86_64 (centos-stream-8-robot-7d7a37eb-bc14-4dd6-9530-dc22c5eae738.noval) 07/07/2024 _x86_64_ (2 CPU) 00:22:32 LINUX RESTART (2 CPU) 12:23:02 AM CPU %user %nice %system %iowait %steal %idle 12:24:01 AM all 37.80 0.02 6.85 11.91 0.15 43.27 12:24:01 AM 0 31.93 0.03 7.42 8.33 0.20 52.08 12:24:01 AM 1 43.67 0.00 6.27 15.49 0.10 34.46 12:25:01 AM all 33.37 0.00 5.29 3.93 0.10 57.31 12:25:01 AM 0 19.24 0.00 3.98 3.90 0.12 72.76 12:25:01 AM 1 47.51 0.00 6.60 3.97 0.08 41.84 12:26:01 AM all 10.40 0.00 1.34 0.98 0.12 87.17 12:26:01 AM 0 1.75 0.00 1.07 0.55 0.08 96.54 12:26:01 AM 1 19.02 0.00 1.62 1.40 0.15 77.81 12:27:01 AM all 20.90 0.00 3.83 1.80 0.09 73.38 12:27:01 AM 0 22.12 0.00 4.04 1.96 0.08 71.80 12:27:01 AM 1 19.68 0.00 3.62 1.64 0.10 74.96 12:28:01 AM all 9.89 0.00 2.55 3.14 0.07 84.36 12:28:01 AM 0 12.41 0.00 3.08 3.40 0.08 81.03 12:28:01 AM 1 7.39 0.00 2.02 2.88 0.05 87.66 12:29:01 AM all 2.80 0.00 0.36 0.14 0.04 96.66 12:29:01 AM 0 2.57 0.00 0.32 0.23 0.05 96.83 12:29:01 AM 1 3.02 0.00 0.40 0.05 0.03 96.49 12:30:01 AM all 4.44 0.00 0.32 0.29 0.07 94.88 12:30:01 AM 0 2.39 0.00 0.25 0.59 0.07 96.71 12:30:01 AM 1 6.50 0.00 0.39 0.00 0.07 93.05 12:31:01 AM all 4.22 0.00 0.33 0.03 0.08 95.35 12:31:01 AM 0 2.31 0.00 0.22 0.00 0.08 97.39 12:31:01 AM 1 6.13 0.00 0.44 0.05 0.08 93.30 12:32:01 AM all 4.18 0.00 0.30 0.00 0.08 95.43 12:32:01 AM 0 3.70 0.00 0.28 0.00 0.08 95.94 12:32:01 AM 1 4.67 0.00 0.32 0.00 0.08 94.93 12:33:01 AM all 4.26 0.00 0.31 0.00 0.09 95.34 12:33:01 AM 0 3.39 0.00 0.28 0.00 0.10 96.22 12:33:01 AM 1 5.12 0.00 0.33 0.00 0.08 94.46 12:34:01 AM all 4.18 0.00 0.28 0.00 0.09 95.45 12:34:01 AM 0 2.26 0.00 0.18 0.00 0.08 97.47 12:34:01 AM 1 6.09 0.00 0.38 0.00 0.10 93.42 12:34:01 AM CPU %user %nice %system %iowait %steal %idle 12:35:01 AM all 4.10 0.00 0.28 0.00 0.08 95.54 12:35:01 AM 0 3.03 0.00 0.28 0.00 0.08 96.61 12:35:01 AM 1 5.17 0.00 0.28 0.00 0.07 94.47 12:36:02 AM all 4.84 0.00 0.28 0.00 0.08 94.80 12:36:02 AM 0 2.92 0.00 0.20 0.00 0.08 96.79 12:36:02 AM 1 6.76 0.00 0.35 0.00 0.08 92.80 12:37:01 AM all 0.33 0.00 0.08 0.00 0.05 99.53 12:37:01 AM 0 0.51 0.00 0.10 0.00 0.07 99.32 12:37:01 AM 1 0.15 0.00 0.07 0.00 0.03 99.75 12:38:01 AM all 0.38 0.00 0.18 0.00 0.04 99.41 12:38:01 AM 0 0.20 0.00 0.18 0.00 0.05 99.57 12:38:01 AM 1 0.55 0.00 0.17 0.00 0.03 99.25 12:39:01 AM all 0.49 0.00 0.07 0.01 0.05 99.38 12:39:01 AM 0 0.57 0.00 0.07 0.00 0.07 99.30 12:39:01 AM 1 0.42 0.00 0.07 0.02 0.03 99.47 12:40:01 AM all 0.33 0.00 0.08 0.00 0.05 99.55 12:40:01 AM 0 0.60 0.00 0.08 0.00 0.05 99.27 12:40:01 AM 1 0.05 0.00 0.07 0.00 0.05 99.83 12:41:01 AM all 0.50 0.00 0.07 0.00 0.06 99.37 12:41:01 AM 0 0.62 0.00 0.05 0.00 0.07 99.27 12:41:01 AM 1 0.38 0.00 0.08 0.00 0.05 99.48 12:42:01 AM all 0.33 0.00 0.06 0.00 0.03 99.58 12:42:01 AM 0 0.62 0.00 0.07 0.00 0.05 99.26 12:42:01 AM 1 0.03 0.00 0.05 0.00 0.02 99.90 12:43:01 AM all 2.87 0.00 1.95 0.00 0.06 95.12 12:43:01 AM 0 0.97 0.00 1.11 0.00 0.07 97.85 12:43:01 AM 1 4.80 0.00 2.82 0.00 0.05 92.34 12:44:01 AM all 5.34 0.00 4.24 0.00 0.06 90.35 12:44:01 AM 0 7.24 0.00 2.22 0.00 0.05 90.49 12:44:01 AM 1 3.56 0.00 6.15 0.00 0.07 90.22 12:45:01 AM all 1.74 0.23 0.46 0.19 0.06 97.31 12:45:01 AM 0 2.13 0.46 0.84 0.38 0.07 96.13 12:45:01 AM 1 1.37 0.00 0.10 0.02 0.05 98.47 12:45:01 AM CPU %user %nice %system %iowait %steal %idle 12:46:01 AM all 0.25 0.00 0.08 0.01 0.05 99.62 12:46:01 AM 0 0.10 0.00 0.07 0.00 0.05 99.78 12:46:01 AM 1 0.40 0.00 0.08 0.02 0.05 99.45 12:47:01 AM all 3.41 0.00 1.39 0.00 0.06 95.14 12:47:01 AM 0 2.33 0.00 1.05 0.00 0.05 96.57 12:47:01 AM 1 4.51 0.00 1.75 0.00 0.07 93.67 12:48:01 AM all 0.69 0.00 0.08 0.00 0.06 99.17 12:48:01 AM 0 0.67 0.00 0.10 0.00 0.08 99.15 12:48:01 AM 1 0.72 0.00 0.07 0.00 0.03 99.18 12:49:01 AM all 0.36 0.00 0.08 0.00 0.05 99.52 12:49:01 AM 0 0.58 0.00 0.07 0.00 0.05 99.30 12:49:01 AM 1 0.13 0.00 0.08 0.00 0.05 99.73 12:50:01 AM all 4.66 0.00 1.33 0.00 0.07 93.94 12:50:01 AM 0 0.62 0.00 0.10 0.00 0.07 99.22 12:50:01 AM 1 9.17 0.00 2.71 0.00 0.07 88.05 12:51:01 AM all 5.38 0.00 2.61 0.00 0.07 91.95 12:51:01 AM 0 3.86 0.00 0.36 0.00 0.07 95.71 12:51:01 AM 1 6.96 0.00 4.94 0.00 0.07 88.03 12:52:01 AM all 5.21 0.00 1.45 0.00 0.07 93.28 12:52:01 AM 0 2.44 0.00 0.27 0.00 0.07 97.21 12:52:01 AM 1 8.19 0.00 2.71 0.00 0.07 89.03 12:53:01 AM all 2.87 0.00 1.19 0.00 0.05 95.89 12:53:01 AM 0 5.11 0.00 2.35 0.00 0.05 92.48 12:53:01 AM 1 0.75 0.00 0.10 0.00 0.05 99.10 12:54:01 AM all 0.35 0.00 0.08 0.00 0.05 99.52 12:54:01 AM 0 0.58 0.00 0.08 0.00 0.05 99.28 12:54:01 AM 1 0.12 0.00 0.08 0.00 0.05 99.75 12:55:01 AM all 2.45 0.00 0.86 0.00 0.07 96.63 12:55:01 AM 0 4.75 0.00 1.65 0.00 0.07 93.53 12:55:01 AM 1 0.25 0.00 0.10 0.00 0.07 99.58 12:56:01 AM all 1.76 0.00 0.52 0.00 0.04 97.68 12:56:01 AM 0 2.71 0.00 0.94 0.00 0.03 96.31 12:56:01 AM 1 0.83 0.00 0.10 0.00 0.05 99.02 12:56:01 AM CPU %user %nice %system %iowait %steal %idle 12:57:01 AM all 0.36 0.00 0.09 0.00 0.04 99.51 12:57:01 AM 0 0.17 0.00 0.10 0.00 0.03 99.70 12:57:01 AM 1 0.55 0.00 0.08 0.00 0.05 99.32 12:58:01 AM all 4.03 0.00 1.88 0.03 0.06 94.00 12:58:01 AM 0 3.24 0.00 0.24 0.00 0.07 96.45 12:58:01 AM 1 4.84 0.00 3.59 0.05 0.05 91.46 12:59:01 AM all 4.95 0.00 0.73 0.00 0.08 94.24 12:59:01 AM 0 2.79 0.00 0.22 0.00 0.08 96.91 12:59:01 AM 1 7.16 0.00 1.25 0.00 0.09 91.50 01:00:01 AM all 4.21 0.00 0.32 0.00 0.08 95.40 01:00:01 AM 0 3.33 0.00 0.27 0.00 0.07 96.34 01:00:01 AM 1 5.09 0.00 0.37 0.00 0.08 94.46 01:01:01 AM all 4.13 0.00 0.31 0.00 0.09 95.47 01:01:01 AM 0 2.84 0.00 0.25 0.00 0.08 96.82 01:01:01 AM 1 5.42 0.00 0.37 0.00 0.10 94.11 01:02:01 AM all 4.27 0.00 0.33 0.00 0.08 95.32 01:02:01 AM 0 2.41 0.00 0.25 0.00 0.08 97.26 01:02:01 AM 1 6.14 0.00 0.40 0.00 0.08 93.38 01:03:01 AM all 4.23 0.00 0.28 0.01 0.10 95.38 01:03:01 AM 0 3.38 0.00 0.25 0.02 0.08 96.27 01:03:01 AM 1 5.09 0.00 0.30 0.00 0.12 94.50 01:04:01 AM all 4.24 0.00 0.32 0.00 0.08 95.37 01:04:01 AM 0 3.10 0.00 0.27 0.00 0.07 96.57 01:04:01 AM 1 5.39 0.00 0.37 0.00 0.08 94.16 01:05:01 AM all 5.54 0.00 0.26 0.00 0.09 94.11 01:05:01 AM 0 3.31 0.00 0.20 0.00 0.08 96.40 01:05:01 AM 1 7.77 0.00 0.32 0.00 0.10 91.81 01:06:01 AM all 0.65 0.00 0.09 0.00 0.06 99.20 01:06:01 AM 0 0.67 0.00 0.07 0.00 0.05 99.22 01:06:01 AM 1 0.63 0.00 0.12 0.00 0.07 99.18 01:07:01 AM all 0.34 0.00 0.06 0.00 0.06 99.54 01:07:01 AM 0 0.60 0.00 0.07 0.00 0.07 99.27 01:07:01 AM 1 0.08 0.00 0.05 0.00 0.05 99.82 01:07:01 AM CPU %user %nice %system %iowait %steal %idle 01:08:01 AM all 0.38 0.00 0.08 0.00 0.05 99.48 01:08:01 AM 0 0.58 0.00 0.07 0.00 0.03 99.32 01:08:01 AM 1 0.18 0.00 0.10 0.00 0.07 99.65 01:09:01 AM all 0.47 0.00 0.05 0.00 0.06 99.42 01:09:01 AM 0 0.60 0.00 0.05 0.00 0.07 99.28 01:09:01 AM 1 0.33 0.00 0.05 0.00 0.05 99.57 01:10:01 AM all 0.33 0.00 0.05 0.00 0.03 99.59 01:10:01 AM 0 0.52 0.00 0.05 0.00 0.03 99.40 01:10:01 AM 1 0.15 0.00 0.05 0.00 0.02 99.78 01:11:01 AM all 0.42 0.00 0.09 0.00 0.05 99.44 01:11:01 AM 0 0.37 0.00 0.07 0.00 0.03 99.53 01:11:01 AM 1 0.47 0.00 0.12 0.00 0.07 99.35 01:12:01 AM all 0.31 0.00 0.06 0.00 0.03 99.60 01:12:01 AM 0 0.13 0.00 0.07 0.00 0.05 99.75 01:12:01 AM 1 0.48 0.00 0.05 0.00 0.02 99.45 01:13:01 AM all 4.03 0.00 4.52 0.00 0.08 91.38 01:13:01 AM 0 6.46 0.00 8.85 0.00 0.09 84.61 01:13:01 AM 1 1.64 0.00 0.24 0.00 0.07 98.06 01:14:01 AM all 4.36 0.00 1.38 0.03 0.06 94.18 01:14:01 AM 0 0.95 0.00 0.19 0.05 0.05 98.77 01:14:01 AM 1 8.18 0.00 2.71 0.00 0.08 89.03 01:15:01 AM all 4.29 0.00 1.35 0.00 0.07 94.29 01:15:01 AM 0 5.64 0.00 2.00 0.00 0.07 92.29 01:15:01 AM 1 2.99 0.00 0.72 0.00 0.07 96.23 01:16:01 AM all 0.35 0.00 0.08 0.00 0.05 99.52 01:16:01 AM 0 0.17 0.00 0.10 0.00 0.08 99.65 01:16:01 AM 1 0.53 0.00 0.05 0.00 0.02 99.40 01:17:01 AM all 1.18 0.00 0.15 0.00 0.05 98.62 01:17:01 AM 0 0.30 0.00 0.13 0.00 0.07 99.50 01:17:01 AM 1 2.07 0.00 0.17 0.00 0.03 97.72 01:18:01 AM all 2.57 0.00 1.27 0.02 0.05 96.09 01:18:01 AM 0 0.63 0.00 0.12 0.00 0.07 99.18 01:18:01 AM 1 4.62 0.00 2.49 0.04 0.04 92.82 01:18:01 AM CPU %user %nice %system %iowait %steal %idle 01:19:01 AM all 0.38 0.00 0.06 0.00 0.04 99.52 01:19:01 AM 0 0.52 0.00 0.03 0.00 0.02 99.43 01:19:01 AM 1 0.25 0.00 0.08 0.00 0.07 99.60 01:20:01 AM all 2.30 0.00 0.81 0.00 0.06 96.83 01:20:01 AM 0 2.75 0.00 0.59 0.00 0.07 96.60 01:20:01 AM 1 1.86 0.00 1.04 0.00 0.05 97.05 01:21:01 AM all 1.69 0.00 0.48 0.00 0.05 97.78 01:21:01 AM 0 0.53 0.00 0.12 0.00 0.05 99.30 01:21:01 AM 1 2.87 0.00 0.86 0.00 0.05 96.21 01:22:01 AM all 0.35 0.00 0.08 0.00 0.04 99.53 01:22:01 AM 0 0.60 0.00 0.10 0.00 0.05 99.25 01:22:01 AM 1 0.10 0.00 0.05 0.00 0.03 99.82 01:23:01 AM all 3.71 0.00 1.30 0.00 0.08 94.91 01:23:01 AM 0 7.40 0.00 2.57 0.00 0.07 89.95 01:23:01 AM 1 0.30 0.00 0.13 0.00 0.08 99.48 01:24:01 AM all 3.88 0.00 1.34 0.00 0.05 94.74 01:24:01 AM 0 7.06 0.00 1.65 0.00 0.04 91.25 01:24:01 AM 1 0.85 0.00 1.03 0.00 0.07 98.05 01:25:01 AM all 0.23 0.00 0.08 0.00 0.07 99.63 01:25:01 AM 0 0.08 0.00 0.05 0.00 0.05 99.82 01:25:01 AM 1 0.37 0.00 0.10 0.00 0.08 99.45 01:26:01 AM all 1.34 0.00 0.15 0.00 0.07 98.44 01:26:01 AM 0 0.55 0.00 0.08 0.00 0.07 99.30 01:26:01 AM 1 2.15 0.00 0.22 0.00 0.07 97.56 01:27:01 AM all 6.00 0.00 1.60 0.00 0.07 92.33