Started by upstream project "integration-distribution-mri-test-scandium" build number 81 originally caused by: Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on prd-centos8-robot-2c-8g-11129 (centos8-robot-2c-8g) in workspace /w/workspace/controller-csit-3node-clustering-ask-all-scandium [ssh-agent] Looking for ssh-agent implementation... [ssh-agent] Exec ssh-agent (binary ssh-agent on a remote machine) $ ssh-agent SSH_AUTH_SOCK=/tmp/ssh-UHj0NPOK9d4L/agent.5279 SSH_AGENT_PID=5280 [ssh-agent] Started. Running ssh-add (command line suppressed) Identity added: /w/workspace/controller-csit-3node-clustering-ask-all-scandium@tmp/private_key_16925620187145567084.key (/w/workspace/controller-csit-3node-clustering-ask-all-scandium@tmp/private_key_16925620187145567084.key) [ssh-agent] Using credentials jenkins (Release Engineering Jenkins Key) The recommended git tool is: NONE using credential opendaylight-jenkins-ssh Wiping out workspace first. Cloning the remote Git repository Cloning repository git://devvexx.opendaylight.org/mirror/integration/test > git init /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test # timeout=10 Fetching upstream changes from git://devvexx.opendaylight.org/mirror/integration/test > git --version # timeout=10 > git --version # 'git version 2.43.0' using GIT_SSH to set credentials Release Engineering Jenkins Key [INFO] Currently running in a labeled security context [INFO] Currently SELinux is 'enforcing' on the host > /usr/bin/chcon --type=ssh_home_t /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test@tmp/jenkins-gitclient-ssh11053434870584261418.key Verifying host key using known hosts file You're using 'Known hosts file' strategy to verify ssh host keys, but your known_hosts file does not exist, please go to 'Manage Jenkins' -> 'Security' -> 'Git Host Key Verification Configuration' and configure host key verification. > git fetch --tags --force --progress -- git://devvexx.opendaylight.org/mirror/integration/test +refs/heads/*:refs/remotes/origin/* # timeout=10 > git config remote.origin.url git://devvexx.opendaylight.org/mirror/integration/test # timeout=10 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 > git config remote.origin.url git://devvexx.opendaylight.org/mirror/integration/test # timeout=10 Fetching upstream changes from git://devvexx.opendaylight.org/mirror/integration/test using GIT_SSH to set credentials Release Engineering Jenkins Key [INFO] Currently running in a labeled security context [INFO] Currently SELinux is 'enforcing' on the host > /usr/bin/chcon --type=ssh_home_t /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test@tmp/jenkins-gitclient-ssh18296835677329277865.key Verifying host key using known hosts file You're using 'Known hosts file' strategy to verify ssh host keys, but your known_hosts file does not exist, please go to 'Manage Jenkins' -> 'Security' -> 'Git Host Key Verification Configuration' and configure host key verification. > git fetch --tags --force --progress -- git://devvexx.opendaylight.org/mirror/integration/test master # timeout=10 > git rev-parse FETCH_HEAD^{commit} # timeout=10 Checking out Revision 6c60ddfc8acc87c45ab0767b2ba1d2c4e7d34388 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 6c60ddfc8acc87c45ab0767b2ba1d2c4e7d34388 # timeout=10 Commit message: "Adapt test for new pce-allocation field" > git rev-parse FETCH_HEAD^{commit} # timeout=10 > git rev-list --no-walk 62cb016f4f4171033927cf2ae7f4ac5095373e88 # timeout=10 No emails were triggered. provisioning config files... copy managed file [npmrc] to file:/home/jenkins/.npmrc copy managed file [pipconf] to file:/home/jenkins/.config/pip/pip.conf copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml [controller-csit-3node-clustering-ask-all-scandium] $ /bin/bash /tmp/jenkins11579522159611700130.sh ---> python-tools-install.sh Setup pyenv: system * 3.8.13 (set by /opt/pyenv/version) * 3.9.13 (set by /opt/pyenv/version) * 3.10.13 (set by /opt/pyenv/version) * 3.11.7 (set by /opt/pyenv/version) lf-activate-venv(): INFO: Creating python3 venv at /tmp/venv-CM3L lf-activate-venv(): INFO: Save venv in file: /tmp/.os_lf_venv lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv) lf-activate-venv(): INFO: Attempting to install with network-safe options... lf-activate-venv(): INFO: Base packages installed successfully lf-activate-venv(): INFO: Installing additional packages: lftools lf-activate-venv(): INFO: Adding /tmp/venv-CM3L/bin to PATH Generating Requirements File ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. httplib2 0.31.0 requires pyparsing<4,>=3.0.4, but you have pyparsing 2.4.7 which is incompatible. Python 3.11.7 pip 25.3 from /tmp/venv-CM3L/lib/python3.11/site-packages/pip (python 3.11) appdirs==1.4.4 argcomplete==3.6.3 aspy.yaml==1.3.0 attrs==25.4.0 autopage==0.5.2 beautifulsoup4==4.14.2 boto3==1.40.74 botocore==1.40.74 bs4==0.0.2 cachetools==6.2.2 certifi==2025.11.12 cffi==2.0.0 cfgv==3.4.0 chardet==5.2.0 charset-normalizer==3.4.4 click==8.3.1 cliff==4.11.0 cmd2==2.7.0 cryptography==3.3.2 debtcollector==3.0.0 decorator==5.2.1 defusedxml==0.7.1 Deprecated==1.3.1 distlib==0.4.0 dnspython==2.8.0 docker==7.1.0 dogpile.cache==1.5.0 durationpy==0.10 email-validator==2.3.0 filelock==3.20.0 future==1.0.0 gitdb==4.0.12 GitPython==3.1.45 google-auth==2.43.0 httplib2==0.31.0 identify==2.6.15 idna==3.11 importlib-resources==1.5.0 iso8601==2.1.0 Jinja2==3.1.6 jmespath==1.0.1 jsonpatch==1.33 jsonpointer==3.0.0 jsonschema==4.25.1 jsonschema-specifications==2025.9.1 keystoneauth1==5.12.0 kubernetes==34.1.0 lftools==0.37.15 lxml==6.0.2 markdown-it-py==4.0.0 MarkupSafe==3.0.3 mdurl==0.1.2 msgpack==1.1.2 multi_key_dict==2.0.3 munch==4.0.0 netaddr==1.3.0 niet==1.4.2 nodeenv==1.9.1 oauth2client==4.1.3 oauthlib==3.3.1 openstacksdk==4.8.0 os-service-types==1.8.1 osc-lib==4.2.0 oslo.config==10.1.0 oslo.context==6.1.0 oslo.i18n==6.7.0 oslo.log==7.2.1 oslo.serialization==5.8.0 oslo.utils==9.1.0 packaging==25.0 pbr==7.0.3 platformdirs==4.5.0 prettytable==3.17.0 psutil==7.1.3 pyasn1==0.6.1 pyasn1_modules==0.4.2 pycparser==2.23 pygerrit2==2.0.15 PyGithub==2.8.1 Pygments==2.19.2 PyJWT==2.10.1 PyNaCl==1.6.1 pyparsing==2.4.7 pyperclip==1.11.0 pyrsistent==0.20.0 python-cinderclient==9.8.0 python-dateutil==2.9.0.post0 python-heatclient==4.3.0 python-jenkins==1.8.3 python-keystoneclient==5.7.0 python-magnumclient==4.9.0 python-openstackclient==8.2.0 python-swiftclient==4.9.0 PyYAML==6.0.3 referencing==0.37.0 requests==2.32.5 requests-oauthlib==2.0.0 requestsexceptions==1.4.0 rfc3986==2.0.0 rich==14.2.0 rich-argparse==1.7.2 rpds-py==0.29.0 rsa==4.9.1 ruamel.yaml==0.18.16 ruamel.yaml.clib==0.2.15 s3transfer==0.14.0 simplejson==3.20.2 six==1.17.0 smmap==5.0.2 soupsieve==2.8 stevedore==5.5.0 tabulate==0.9.0 toml==0.10.2 tomlkit==0.13.3 tqdm==4.67.1 typing_extensions==4.15.0 tzdata==2025.2 urllib3==1.26.20 virtualenv==20.35.4 wcwidth==0.2.14 websocket-client==1.9.0 wrapt==2.0.1 xdg==6.0.0 xmltodict==1.0.2 yq==3.4.3 [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties content OS_STACK_TEMPLATE=csit-2-instance-type.yaml OS_CLOUD=vex OS_STACK_NAME=releng-controller-csit-3node-clustering-ask-all-scandium-81 OS_STACK_TEMPLATE_DIR=openstack-hot [EnvInject] - Variables injected successfully. provisioning config files... copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml [controller-csit-3node-clustering-ask-all-scandium] $ /bin/bash /tmp/jenkins8667212740295609084.sh ---> Create parameters file for OpenStack HOT OpenStack Heat parameters generated ----------------------------------- parameters: vm_0_count: '3' vm_0_flavor: 'v3-standard-4' vm_0_image: 'ZZCI - Ubuntu 22.04 - builder - x86_64 - 20250917-133034.447' vm_1_count: '0' vm_1_flavor: 'v3-standard-2' vm_1_image: 'ZZCI - Ubuntu 22.04 - mininet-ovs-217 - x86_64 - 20250917-133034.654' job_name: '47598-81' silo: 'releng' [controller-csit-3node-clustering-ask-all-scandium] $ /bin/bash -l /tmp/jenkins13290378386584559579.sh ---> Create HEAT stack + source /home/jenkins/lf-env.sh + lf-activate-venv --python python3 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient urllib3~=1.26.15 yq ++ mktemp -d /tmp/venv-XXXX + lf_venv=/tmp/venv-furz + local venv_file=/tmp/.os_lf_venv + local python=python3 + local options + local set_path=true + local install_args= ++ getopt -o np:v: -l no-path,system-site-packages,python:,venv-file: -n lf-activate-venv -- --python python3 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient urllib3~=1.26.15 yq + options=' --python '\''python3'\'' -- '\''lftools[openstack]'\'' '\''kubernetes'\'' '\''niet'\'' '\''python-heatclient'\'' '\''python-openstackclient'\'' '\''python-magnumclient'\'' '\''urllib3~=1.26.15'\'' '\''yq'\''' + eval set -- ' --python '\''python3'\'' -- '\''lftools[openstack]'\'' '\''kubernetes'\'' '\''niet'\'' '\''python-heatclient'\'' '\''python-openstackclient'\'' '\''python-magnumclient'\'' '\''urllib3~=1.26.15'\'' '\''yq'\''' ++ set -- --python python3 -- 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient urllib3~=1.26.15 yq + true + case $1 in + python=python3 + shift 2 + true + case $1 in + shift + break + case $python in + local pkg_list= + [[ -d /opt/pyenv ]] + echo 'Setup pyenv:' Setup pyenv: + export PYENV_ROOT=/opt/pyenv + PYENV_ROOT=/opt/pyenv + export PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin + PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin + pyenv versions system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-scandium/.python-version) + command -v pyenv ++ pyenv init - --no-rehash + eval 'PATH="$(bash --norc -ec '\''IFS=:; paths=($PATH); for i in ${!paths[@]}; do if [[ ${paths[i]} == "'\'''\''/opt/pyenv/shims'\'''\''" ]]; then unset '\''\'\'''\''paths[i]'\''\'\'''\''; fi; done; echo "${paths[*]}"'\'')" export PATH="/opt/pyenv/shims:${PATH}" export PYENV_SHELL=bash source '\''/opt/pyenv/libexec/../completions/pyenv.bash'\'' pyenv() { local command command="${1:-}" if [ "$#" -gt 0 ]; then shift fi case "$command" in rehash|shell) eval "$(pyenv "sh-$command" "$@")" ;; *) command pyenv "$command" "$@" ;; esac }' +++ bash --norc -ec 'IFS=:; paths=($PATH); for i in ${!paths[@]}; do if [[ ${paths[i]} == "/opt/pyenv/shims" ]]; then unset '\''paths[i]'\''; fi; done; echo "${paths[*]}"' ++ PATH=/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin ++ export PATH=/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin ++ PATH=/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin ++ export PYENV_SHELL=bash ++ PYENV_SHELL=bash ++ source /opt/pyenv/libexec/../completions/pyenv.bash +++ complete -F _pyenv pyenv ++ lf-pyver python3 ++ local py_version_xy=python3 ++ local py_version_xyz= ++ pyenv versions ++ local command ++ awk '{ print $1 }' ++ command=versions ++ '[' 1 -gt 0 ']' ++ shift ++ case "$command" in ++ command pyenv versions ++ pyenv versions ++ sed 's/^[ *]* //' ++ grep -E '^[0-9.]*[0-9]$' ++ [[ ! -s /tmp/.pyenv_versions ]] +++ grep '^3' /tmp/.pyenv_versions +++ tail -n 1 +++ sort -V ++ py_version_xyz=3.11.7 ++ [[ -z 3.11.7 ]] ++ echo 3.11.7 ++ return 0 + pyenv local 3.11.7 + local command + command=local + '[' 2 -gt 0 ']' + shift + case "$command" in + command pyenv local 3.11.7 + pyenv local 3.11.7 + for arg in "$@" + case $arg in + pkg_list+='lftools[openstack] ' + for arg in "$@" + case $arg in + pkg_list+='kubernetes ' + for arg in "$@" + case $arg in + pkg_list+='niet ' + for arg in "$@" + case $arg in + pkg_list+='python-heatclient ' + for arg in "$@" + case $arg in + pkg_list+='python-openstackclient ' + for arg in "$@" + case $arg in + pkg_list+='python-magnumclient ' + for arg in "$@" + case $arg in + pkg_list+='urllib3~=1.26.15 ' + for arg in "$@" + case $arg in + pkg_list+='yq ' + [[ -f /tmp/.os_lf_venv ]] ++ cat /tmp/.os_lf_venv + lf_venv=/tmp/venv-CM3L + echo 'lf-activate-venv(): INFO: Reuse venv:/tmp/venv-CM3L from' file:/tmp/.os_lf_venv lf-activate-venv(): INFO: Reuse venv:/tmp/venv-CM3L from file:/tmp/.os_lf_venv + echo 'lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv)' lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv) + local 'pip_opts=--upgrade --quiet' + pip_opts='--upgrade --quiet --trusted-host pypi.org' + pip_opts='--upgrade --quiet --trusted-host pypi.org --trusted-host files.pythonhosted.org' + pip_opts='--upgrade --quiet --trusted-host pypi.org --trusted-host files.pythonhosted.org --trusted-host pypi.python.org' + [[ -n '' ]] + [[ -n '' ]] + echo 'lf-activate-venv(): INFO: Attempting to install with network-safe options...' lf-activate-venv(): INFO: Attempting to install with network-safe options... + /tmp/venv-CM3L/bin/python3 -m pip install --upgrade --quiet --trusted-host pypi.org --trusted-host files.pythonhosted.org --trusted-host pypi.python.org pip 'setuptools<66' virtualenv + echo 'lf-activate-venv(): INFO: Base packages installed successfully' lf-activate-venv(): INFO: Base packages installed successfully + [[ -z lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient urllib3~=1.26.15 yq ]] + echo 'lf-activate-venv(): INFO: Installing additional packages: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient urllib3~=1.26.15 yq ' lf-activate-venv(): INFO: Installing additional packages: lftools[openstack] kubernetes niet python-heatclient python-openstackclient python-magnumclient urllib3~=1.26.15 yq + /tmp/venv-CM3L/bin/python3 -m pip install --upgrade --quiet --trusted-host pypi.org --trusted-host files.pythonhosted.org --trusted-host pypi.python.org --upgrade-strategy eager 'lftools[openstack]' kubernetes niet python-heatclient python-openstackclient python-magnumclient urllib3~=1.26.15 yq + type python3 + true + echo 'lf-activate-venv(): INFO: Adding /tmp/venv-CM3L/bin to PATH' lf-activate-venv(): INFO: Adding /tmp/venv-CM3L/bin to PATH + PATH=/tmp/venv-CM3L/bin:/opt/pyenv/shims:/opt/pyenv/bin:/home/jenkins/.local/bin:/home/jenkins/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/puppetlabs/bin + return 0 + openstack --os-cloud vex limits show --absolute +--------------------------+---------+ | Name | Value | +--------------------------+---------+ | maxTotalInstances | -1 | | maxTotalCores | 450 | | maxTotalRAMSize | 1000000 | | maxServerMeta | 128 | | maxImageMeta | 128 | | maxPersonality | 5 | | maxPersonalitySize | 10240 | | maxTotalKeypairs | 100 | | maxServerGroups | 10 | | maxServerGroupMembers | 10 | | maxTotalFloatingIps | -1 | | maxSecurityGroups | -1 | | maxSecurityGroupRules | -1 | | totalRAMUsed | 811008 | | totalCoresUsed | 198 | | totalInstancesUsed | 78 | | totalFloatingIpsUsed | 0 | | totalSecurityGroupsUsed | 0 | | totalServerGroupsUsed | 0 | | maxTotalVolumes | -1 | | maxTotalSnapshots | 10 | | maxTotalVolumeGigabytes | 4096 | | maxTotalBackups | 10 | | maxTotalBackupGigabytes | 1000 | | totalVolumesUsed | 3 | | totalGigabytesUsed | 60 | | totalSnapshotsUsed | 0 | | totalBackupsUsed | 0 | | totalBackupGigabytesUsed | 0 | +--------------------------+---------+ + pushd /opt/ciman/openstack-hot /opt/ciman/openstack-hot /w/workspace/controller-csit-3node-clustering-ask-all-scandium + lftools openstack --os-cloud vex stack create releng-controller-csit-3node-clustering-ask-all-scandium-81 csit-2-instance-type.yaml /w/workspace/controller-csit-3node-clustering-ask-all-scandium/stack-parameters.yaml Creating stack releng-controller-csit-3node-clustering-ask-all-scandium-81 Waiting to initialize infrastructure... Stack initialization successful. ------------------------------------ Stack Details ------------------------------------ {'added': None, 'capabilities': [], 'created_at': '2025-11-16T23:35:15Z', 'deleted': None, 'deleted_at': None, 'description': 'No description', 'environment': None, 'environment_files': None, 'files': None, 'files_container': None, 'id': '30580128-f3ef-46d9-94e4-91d8dddd4dcb', 'is_rollback_disabled': True, 'links': [{'href': 'https://orchestration.public.mtl1.vexxhost.net/v1/12c36e260d8e4bb2913965203b1b491f/stacks/releng-controller-csit-3node-clustering-ask-all-scandium-81/30580128-f3ef-46d9-94e4-91d8dddd4dcb', 'rel': 'self'}], 'location': Munch({'cloud': 'vex', 'region_name': 'ca-ymq-1', 'zone': None, 'project': Munch({'id': '12c36e260d8e4bb2913965203b1b491f', 'name': '61975f2c-7c17-4d69-82fa-c3ae420ad6fd', 'domain_id': None, 'domain_name': 'Default'})}), 'name': 'releng-controller-csit-3node-clustering-ask-all-scandium-81', 'notification_topics': [], 'outputs': [{'description': 'IP addresses of the 1st vm types', 'output_key': 'vm_0_ips', 'output_value': ['10.30.170.150', '10.30.170.130', '10.30.171.41']}, {'description': 'IP addresses of the 2nd vm types', 'output_key': 'vm_1_ips', 'output_value': []}], 'owner_id': ****, 'parameters': {'OS::project_id': '12c36e260d8e4bb2913965203b1b491f', 'OS::stack_id': '30580128-f3ef-46d9-94e4-91d8dddd4dcb', 'OS::stack_name': 'releng-controller-csit-3node-clustering-ask-all-scandium-81', 'job_name': '47598-81', 'silo': 'releng', 'vm_0_count': '3', 'vm_0_flavor': 'v3-standard-4', 'vm_0_image': 'ZZCI - Ubuntu 22.04 - builder - x86_64 - ' '20250917-133034.447', 'vm_1_count': '0', 'vm_1_flavor': 'v3-standard-2', 'vm_1_image': 'ZZCI - Ubuntu 22.04 - mininet-ovs-217 - x86_64 ' '- 20250917-133034.654'}, 'parent_id': None, 'replaced': None, 'status': 'CREATE_COMPLETE', 'status_reason': 'Stack CREATE completed successfully', 'tags': [], 'template': None, 'template_description': 'No description', 'template_url': None, 'timeout_mins': 15, 'unchanged': None, 'updated': None, 'updated_at': None, 'user_project_id': 'aadd39c0623a4bdd8bc25f7563233ec7'} ------------------------------------ + popd /w/workspace/controller-csit-3node-clustering-ask-all-scandium [controller-csit-3node-clustering-ask-all-scandium] $ /bin/bash -l /tmp/jenkins16748726486878780517.sh ---> Copy SSH public keys to CSIT lab Setup pyenv: system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-scandium/.python-version) lf-activate-venv(): INFO: Reuse venv:/tmp/venv-CM3L from file:/tmp/.os_lf_venv lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv) lf-activate-venv(): INFO: Attempting to install with network-safe options... lf-activate-venv(): INFO: Base packages installed successfully lf-activate-venv(): INFO: Installing additional packages: lftools[openstack] kubernetes python-heatclient python-openstackclient urllib3~=1.26.15 lf-activate-venv(): INFO: Adding /tmp/venv-CM3L/bin to PATH SSH not responding on 10.30.171.41. Retrying in 10 seconds... SSH not responding on 10.30.170.130. Retrying in 10 seconds... SSH not responding on 10.30.170.150. Retrying in 10 seconds... Ping to 10.30.171.41 successful. Ping to 10.30.170.150 successful. Ping to 10.30.170.130 successful. SSH not responding on 10.30.171.41. Retrying in 10 seconds... SSH not responding on 10.30.170.130. Retrying in 10 seconds... SSH not responding on 10.30.170.150. Retrying in 10 seconds... Ping to 10.30.171.41 successful. Ping to 10.30.170.130 successful. Ping to 10.30.170.150 successful. SSH not responding on 10.30.170.130. Retrying in 10 seconds... SSH not responding on 10.30.170.150. Retrying in 10 seconds... Warning: Permanently added '10.30.171.41' (ECDSA) to the list of known hosts. releng-47598-81-0-builder-2 Successfully copied public keys to slave 10.30.171.41 Ping to 10.30.170.130 successful. Ping to 10.30.170.150 successful. Warning: Permanently added '10.30.170.130' (ECDSA) to the list of known hosts. Warning: Permanently added '10.30.170.150' (ECDSA) to the list of known hosts. releng-47598-81-0-builder-1 Successfully copied public keys to slave 10.30.170.130 releng-47598-81-0-builder-0 Successfully copied public keys to slave 10.30.170.150 Process 6488 ready. Process 6489 ready. Process 6490 ready. SSH ready on all stack servers. [controller-csit-3node-clustering-ask-all-scandium] $ /bin/bash -l /tmp/jenkins278300690491355207.sh Setup pyenv: system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-scandium/.python-version) lf-activate-venv(): INFO: Creating python3 venv at /tmp/venv-2sLU lf-activate-venv(): INFO: Save venv in file: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/.robot_venv lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv) lf-activate-venv(): INFO: Attempting to install with network-safe options... lf-activate-venv(): INFO: Base packages installed successfully lf-activate-venv(): INFO: Installing additional packages: setuptools wheel lf-activate-venv(): INFO: Adding /tmp/venv-2sLU/bin to PATH + echo 'Installing Python Requirements' Installing Python Requirements + cat + python -m pip install -r requirements.txt Looking in indexes: https://nexus3.opendaylight.org/repository/PyPi/simple Collecting docker-py (from -r requirements.txt (line 1)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/docker-py/1.10.6/docker_py-1.10.6-py2.py3-none-any.whl (50 kB) Collecting ipaddr (from -r requirements.txt (line 2)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/ipaddr/2.2.0/ipaddr-2.2.0.tar.gz (26 kB) Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Preparing metadata (pyproject.toml): started Preparing metadata (pyproject.toml): finished with status 'done' Collecting netaddr (from -r requirements.txt (line 3)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/netaddr/1.3.0/netaddr-1.3.0-py3-none-any.whl (2.3 MB) Collecting netifaces (from -r requirements.txt (line 4)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/netifaces/0.11.0/netifaces-0.11.0.tar.gz (30 kB) Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Preparing metadata (pyproject.toml): started Preparing metadata (pyproject.toml): finished with status 'done' Collecting pyhocon (from -r requirements.txt (line 5)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/pyhocon/0.3.61/pyhocon-0.3.61-py3-none-any.whl (25 kB) Collecting requests (from -r requirements.txt (line 6)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/requests/2.32.5/requests-2.32.5-py3-none-any.whl (64 kB) Collecting robotframework (from -r requirements.txt (line 7)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework/7.3.2/robotframework-7.3.2-py3-none-any.whl (795 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 795.1/795.1 kB 38.9 MB/s 0:00:00 Collecting robotframework-httplibrary (from -r requirements.txt (line 8)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-httplibrary/0.4.2/robotframework-httplibrary-0.4.2.tar.gz (9.1 kB) Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Preparing metadata (pyproject.toml): started Preparing metadata (pyproject.toml): finished with status 'done' Collecting robotframework-requests==0.9.7 (from -r requirements.txt (line 9)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-requests/0.9.7/robotframework_requests-0.9.7-py3-none-any.whl (21 kB) Collecting robotframework-selenium2library (from -r requirements.txt (line 10)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-selenium2library/3.0.0/robotframework_selenium2library-3.0.0-py2.py3-none-any.whl (6.2 kB) Collecting robotframework-sshlibrary==3.8.0 (from -r requirements.txt (line 11)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-sshlibrary/3.8.0/robotframework-sshlibrary-3.8.0.tar.gz (51 kB) Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Preparing metadata (pyproject.toml): started Preparing metadata (pyproject.toml): finished with status 'done' Collecting scapy (from -r requirements.txt (line 12)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/scapy/2.6.1/scapy-2.6.1-py3-none-any.whl (2.4 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.4/2.4 MB 51.2 MB/s 0:00:00 Collecting jsonpath-rw (from -r requirements.txt (line 15)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/jsonpath-rw/1.4.0/jsonpath-rw-1.4.0.tar.gz (13 kB) Installing build dependencies: started Installing build dependencies: finished with status 'done' Getting requirements to build wheel: started Getting requirements to build wheel: finished with status 'done' Preparing metadata (pyproject.toml): started Preparing metadata (pyproject.toml): finished with status 'done' Collecting elasticsearch (from -r requirements.txt (line 18)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elasticsearch/9.2.0/elasticsearch-9.2.0-py3-none-any.whl (960 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 960.5/960.5 kB 34.3 MB/s 0:00:00 Collecting elasticsearch-dsl (from -r requirements.txt (line 19)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elasticsearch-dsl/8.18.0/elasticsearch_dsl-8.18.0-py3-none-any.whl (10 kB) Collecting pyangbind (from -r requirements.txt (line 22)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/pyangbind/0.8.6/pyangbind-0.8.6-py3-none-any.whl (52 kB) Collecting isodate (from -r requirements.txt (line 25)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/isodate/0.7.2/isodate-0.7.2-py3-none-any.whl (22 kB) Collecting jmespath (from -r requirements.txt (line 28)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/jmespath/1.0.1/jmespath-1.0.1-py3-none-any.whl (20 kB) Collecting jsonpatch (from -r requirements.txt (line 31)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/jsonpatch/1.33/jsonpatch-1.33-py2.py3-none-any.whl (12 kB) Collecting paramiko>=1.15.3 (from robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/paramiko/4.0.0/paramiko-4.0.0-py3-none-any.whl (223 kB) Collecting scp>=0.13.0 (from robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/scp/0.15.0/scp-0.15.0-py2.py3-none-any.whl (8.8 kB) Collecting docker-pycreds>=0.2.1 (from docker-py->-r requirements.txt (line 1)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/docker-pycreds/0.4.0/docker_pycreds-0.4.0-py2.py3-none-any.whl (9.0 kB) Collecting six>=1.4.0 (from docker-py->-r requirements.txt (line 1)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/six/1.17.0/six-1.17.0-py2.py3-none-any.whl (11 kB) Collecting websocket-client>=0.32.0 (from docker-py->-r requirements.txt (line 1)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/websocket-client/1.9.0/websocket_client-1.9.0-py3-none-any.whl (82 kB) Collecting pyparsing<4,>=2 (from pyhocon->-r requirements.txt (line 5)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/pyparsing/3.2.5/pyparsing-3.2.5-py3-none-any.whl (113 kB) Collecting charset_normalizer<4,>=2 (from requests->-r requirements.txt (line 6)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/charset-normalizer/3.4.4/charset_normalizer-3.4.4-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (151 kB) Collecting idna<4,>=2.5 (from requests->-r requirements.txt (line 6)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/idna/3.11/idna-3.11-py3-none-any.whl (71 kB) Collecting urllib3<3,>=1.21.1 (from requests->-r requirements.txt (line 6)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/urllib3/2.5.0/urllib3-2.5.0-py3-none-any.whl (129 kB) Collecting certifi>=2017.4.17 (from requests->-r requirements.txt (line 6)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/certifi/2025.11.12/certifi-2025.11.12-py3-none-any.whl (159 kB) Collecting webtest>=2.0 (from robotframework-httplibrary->-r requirements.txt (line 8)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/webtest/3.0.7/webtest-3.0.7-py3-none-any.whl (32 kB) Collecting jsonpointer (from robotframework-httplibrary->-r requirements.txt (line 8)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/jsonpointer/3.0.0/jsonpointer-3.0.0-py2.py3-none-any.whl (7.6 kB) Collecting robotframework-seleniumlibrary>=3.0.0 (from robotframework-selenium2library->-r requirements.txt (line 10)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-seleniumlibrary/6.8.0/robotframework_seleniumlibrary-6.8.0-py3-none-any.whl (104 kB) Collecting ply (from jsonpath-rw->-r requirements.txt (line 15)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/ply/3.11/ply-3.11-py2.py3-none-any.whl (49 kB) Collecting decorator (from jsonpath-rw->-r requirements.txt (line 15)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/decorator/5.2.1/decorator-5.2.1-py3-none-any.whl (9.2 kB) Collecting anyio (from elasticsearch->-r requirements.txt (line 18)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/anyio/4.11.0/anyio-4.11.0-py3-none-any.whl (109 kB) Collecting elastic-transport<10,>=9.2.0 (from elasticsearch->-r requirements.txt (line 18)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elastic-transport/9.2.0/elastic_transport-9.2.0-py3-none-any.whl (65 kB) Collecting python-dateutil (from elasticsearch->-r requirements.txt (line 18)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/python-dateutil/2.9.0.post0/python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB) Collecting sniffio (from elasticsearch->-r requirements.txt (line 18)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/sniffio/1.3.1/sniffio-1.3.1-py3-none-any.whl (10 kB) Collecting typing-extensions (from elasticsearch->-r requirements.txt (line 18)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/typing-extensions/4.15.0/typing_extensions-4.15.0-py3-none-any.whl (44 kB) Collecting elasticsearch (from -r requirements.txt (line 18)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elasticsearch/8.19.2/elasticsearch-8.19.2-py3-none-any.whl (949 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 949.7/949.7 kB 39.3 MB/s 0:00:00 INFO: pip is looking at multiple versions of elasticsearch-dsl to determine which version is compatible with other requirements. This could take a while. Collecting elasticsearch-dsl (from -r requirements.txt (line 19)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elasticsearch-dsl/8.17.1/elasticsearch_dsl-8.17.1-py3-none-any.whl (158 kB) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elasticsearch-dsl/8.17.0/elasticsearch_dsl-8.17.0-py3-none-any.whl (158 kB) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elasticsearch-dsl/8.16.0/elasticsearch_dsl-8.16.0-py3-none-any.whl (158 kB) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elasticsearch-dsl/8.15.4/elasticsearch_dsl-8.15.4-py3-none-any.whl (104 kB) Collecting elastic-transport<9,>=8.15.1 (from elasticsearch->-r requirements.txt (line 18)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/elastic-transport/8.17.1/elastic_transport-8.17.1-py3-none-any.whl (64 kB) Collecting pyang (from pyangbind->-r requirements.txt (line 22)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/pyang/2.7.1/pyang-2.7.1-py2.py3-none-any.whl (598 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 598.5/598.5 kB 34.3 MB/s 0:00:00 Collecting lxml (from pyangbind->-r requirements.txt (line 22)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/lxml/6.0.2/lxml-6.0.2-cp311-cp311-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl (5.2 MB) Collecting regex (from pyangbind->-r requirements.txt (line 22)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/regex/2025.11.3/regex-2025.11.3-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl (800 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 800.4/800.4 kB 35.5 MB/s 0:00:00 Collecting enum34 (from pyangbind->-r requirements.txt (line 22)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/enum34/1.1.10/enum34-1.1.10-py3-none-any.whl (11 kB) Collecting bcrypt>=3.2 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/bcrypt/5.0.0/bcrypt-5.0.0-cp39-abi3-manylinux_2_28_x86_64.whl (278 kB) Collecting cryptography>=3.3 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/cryptography/46.0.3/cryptography-46.0.3-cp311-abi3-manylinux_2_28_x86_64.whl (4.5 MB) Collecting invoke>=2.0 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/invoke/2.2.1/invoke-2.2.1-py3-none-any.whl (160 kB) Collecting pynacl>=1.5 (from paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/pynacl/1.6.1/pynacl-1.6.1-cp38-abi3-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl (1.4 MB) Collecting cffi>=2.0.0 (from cryptography>=3.3->paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/cffi/2.0.0/cffi-2.0.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (215 kB) Collecting pycparser (from cffi>=2.0.0->cryptography>=3.3->paramiko>=1.15.3->robotframework-sshlibrary==3.8.0->-r requirements.txt (line 11)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/pycparser/2.23/pycparser-2.23-py3-none-any.whl (118 kB) Collecting selenium>=4.3.0 (from robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/selenium/4.38.0/selenium-4.38.0-py3-none-any.whl (9.7 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 9.7/9.7 MB 73.4 MB/s 0:00:00 Collecting robotframework-pythonlibcore>=4.4.1 (from robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/robotframework-pythonlibcore/4.4.1/robotframework_pythonlibcore-4.4.1-py2.py3-none-any.whl (12 kB) Collecting click>=8.0 (from robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/click/8.3.1/click-8.3.1-py3-none-any.whl (108 kB) Collecting trio<1.0,>=0.31.0 (from selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/trio/0.32.0/trio-0.32.0-py3-none-any.whl (512 kB) Collecting trio-websocket<1.0,>=0.12.2 (from selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/trio-websocket/0.12.2/trio_websocket-0.12.2-py3-none-any.whl (21 kB) Collecting attrs>=23.2.0 (from trio<1.0,>=0.31.0->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/attrs/25.4.0/attrs-25.4.0-py3-none-any.whl (67 kB) Collecting sortedcontainers (from trio<1.0,>=0.31.0->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/sortedcontainers/2.4.0/sortedcontainers-2.4.0-py2.py3-none-any.whl (29 kB) Collecting outcome (from trio<1.0,>=0.31.0->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/outcome/1.3.0.post0/outcome-1.3.0.post0-py2.py3-none-any.whl (10 kB) Collecting wsproto>=0.14 (from trio-websocket<1.0,>=0.12.2->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/wsproto/1.3.1/wsproto-1.3.1-py3-none-any.whl (24 kB) Collecting pysocks!=1.5.7,<2.0,>=1.5.6 (from urllib3[socks]<3.0,>=2.5.0->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/pysocks/1.7.1/PySocks-1.7.1-py3-none-any.whl (16 kB) Collecting WebOb>=1.2 (from webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/webob/1.8.9/WebOb-1.8.9-py2.py3-none-any.whl (115 kB) Collecting waitress>=3.0.2 (from webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/waitress/3.0.2/waitress-3.0.2-py3-none-any.whl (56 kB) Collecting beautifulsoup4 (from webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/beautifulsoup4/4.14.2/beautifulsoup4-4.14.2-py3-none-any.whl (106 kB) Collecting h11<1,>=0.16.0 (from wsproto>=0.14->trio-websocket<1.0,>=0.12.2->selenium>=4.3.0->robotframework-seleniumlibrary>=3.0.0->robotframework-selenium2library->-r requirements.txt (line 10)) Downloading https://nexus3.opendaylight.org/repository/PyPi/packages/h11/0.16.0/h11-0.16.0-py3-none-any.whl (37 kB) Collecting soupsieve>1.2 (from beautifulsoup4->webtest>=2.0->robotframework-httplibrary->-r requirements.txt (line 8)) Using cached https://nexus3.opendaylight.org/repository/PyPi/packages/soupsieve/2.8/soupsieve-2.8-py3-none-any.whl (36 kB) Building wheels for collected packages: robotframework-sshlibrary, ipaddr, netifaces, robotframework-httplibrary, jsonpath-rw Building wheel for robotframework-sshlibrary (pyproject.toml): started Building wheel for robotframework-sshlibrary (pyproject.toml): finished with status 'done' Created wheel for robotframework-sshlibrary: filename=robotframework_sshlibrary-3.8.0-py3-none-any.whl size=55205 sha256=70954175b190f1b3ad945837fdf34100f1edda6916f1481e05382dbbdbd65673 Stored in directory: /home/jenkins/.cache/pip/wheels/f7/c9/b3/a977b7bcc410d45ae27d240df3d00a12585509180e373ecccc Building wheel for ipaddr (pyproject.toml): started Building wheel for ipaddr (pyproject.toml): finished with status 'done' Created wheel for ipaddr: filename=ipaddr-2.2.0-py3-none-any.whl size=18353 sha256=354435d9eb502b298a7e8d7e83b4f32fb73e14db889d3869ca4948732c71912d Stored in directory: /home/jenkins/.cache/pip/wheels/dc/6c/04/da2d847fa8d45c59af3e1d83e2acc29cb8adcbaf04c0898dbf Building wheel for netifaces (pyproject.toml): started Building wheel for netifaces (pyproject.toml): finished with status 'done' Created wheel for netifaces: filename=netifaces-0.11.0-cp311-cp311-linux_x86_64.whl size=41074 sha256=c71f91a8b4031e0426618393c645ea59eb66d197bda4c9109c23ae06fab65bb9 Stored in directory: /home/jenkins/.cache/pip/wheels/f8/18/88/e61d54b995bea304bdb1d040a92b72228a1bf72ca2a3eba7c9 Building wheel for robotframework-httplibrary (pyproject.toml): started Building wheel for robotframework-httplibrary (pyproject.toml): finished with status 'done' Created wheel for robotframework-httplibrary: filename=robotframework_httplibrary-0.4.2-py3-none-any.whl size=10014 sha256=af4e1cac64a8cdc11d1ba2c9d9c0f03a5f4ea9ec97a19962fe00dd6337f759e1 Stored in directory: /home/jenkins/.cache/pip/wheels/aa/bc/0d/9a20dd51effef392aae2733cb4c7b66c6fa29fca33d88b57ed Building wheel for jsonpath-rw (pyproject.toml): started Building wheel for jsonpath-rw (pyproject.toml): finished with status 'done' Created wheel for jsonpath-rw: filename=jsonpath_rw-1.4.0-py3-none-any.whl size=15176 sha256=b5e29090e22acc9bbc15d0ccdadd92661acd3bb0cd58f63101c075b64d45ad2a Stored in directory: /home/jenkins/.cache/pip/wheels/f1/54/63/9a8da38cefae13755097b36cc852decc25d8ef69c37d58d4eb Successfully built robotframework-sshlibrary ipaddr netifaces robotframework-httplibrary jsonpath-rw Installing collected packages: sortedcontainers, ply, netifaces, ipaddr, enum34, websocket-client, WebOb, waitress, urllib3, typing-extensions, soupsieve, sniffio, six, scapy, robotframework-pythonlibcore, robotframework, regex, pysocks, pyparsing, pycparser, netaddr, lxml, jsonpointer, jmespath, isodate, invoke, idna, h11, decorator, click, charset_normalizer, certifi, bcrypt, attrs, wsproto, requests, python-dateutil, pyhocon, pyang, outcome, jsonpath-rw, jsonpatch, elastic-transport, docker-pycreds, cffi, beautifulsoup4, webtest, trio, robotframework-requests, pynacl, pyangbind, elasticsearch, docker-py, cryptography, trio-websocket, robotframework-httplibrary, paramiko, elasticsearch-dsl, selenium, scp, robotframework-sshlibrary, robotframework-seleniumlibrary, robotframework-selenium2library Successfully installed WebOb-1.8.9 attrs-25.4.0 bcrypt-5.0.0 beautifulsoup4-4.14.2 certifi-2025.11.12 cffi-2.0.0 charset_normalizer-3.4.4 click-8.3.1 cryptography-46.0.3 decorator-5.2.1 docker-py-1.10.6 docker-pycreds-0.4.0 elastic-transport-8.17.1 elasticsearch-8.19.2 elasticsearch-dsl-8.15.4 enum34-1.1.10 h11-0.16.0 idna-3.11 invoke-2.2.1 ipaddr-2.2.0 isodate-0.7.2 jmespath-1.0.1 jsonpatch-1.33 jsonpath-rw-1.4.0 jsonpointer-3.0.0 lxml-6.0.2 netaddr-1.3.0 netifaces-0.11.0 outcome-1.3.0.post0 paramiko-4.0.0 ply-3.11 pyang-2.7.1 pyangbind-0.8.6 pycparser-2.23 pyhocon-0.3.61 pynacl-1.6.1 pyparsing-3.2.5 pysocks-1.7.1 python-dateutil-2.9.0.post0 regex-2025.11.3 requests-2.32.5 robotframework-7.3.2 robotframework-httplibrary-0.4.2 robotframework-pythonlibcore-4.4.1 robotframework-requests-0.9.7 robotframework-selenium2library-3.0.0 robotframework-seleniumlibrary-6.8.0 robotframework-sshlibrary-3.8.0 scapy-2.6.1 scp-0.15.0 selenium-4.38.0 six-1.17.0 sniffio-1.3.1 sortedcontainers-2.4.0 soupsieve-2.8 trio-0.32.0 trio-websocket-0.12.2 typing-extensions-4.15.0 urllib3-2.5.0 waitress-3.0.2 websocket-client-1.9.0 webtest-3.0.7 wsproto-1.3.1 + pip freeze attrs==25.4.0 bcrypt==5.0.0 beautifulsoup4==4.14.2 certifi==2025.11.12 cffi==2.0.0 charset-normalizer==3.4.4 click==8.3.1 cryptography==46.0.3 decorator==5.2.1 distlib==0.4.0 docker-py==1.10.6 docker-pycreds==0.4.0 elastic-transport==8.17.1 elasticsearch==8.19.2 elasticsearch-dsl==8.15.4 enum34==1.1.10 filelock==3.20.0 h11==0.16.0 idna==3.11 invoke==2.2.1 ipaddr==2.2.0 isodate==0.7.2 jmespath==1.0.1 jsonpatch==1.33 jsonpath-rw==1.4.0 jsonpointer==3.0.0 lxml==6.0.2 netaddr==1.3.0 netifaces==0.11.0 outcome==1.3.0.post0 paramiko==4.0.0 platformdirs==4.5.0 ply==3.11 pyang==2.7.1 pyangbind==0.8.6 pycparser==2.23 pyhocon==0.3.61 PyNaCl==1.6.1 pyparsing==3.2.5 PySocks==1.7.1 python-dateutil==2.9.0.post0 regex==2025.11.3 requests==2.32.5 robotframework==7.3.2 robotframework-httplibrary==0.4.2 robotframework-pythonlibcore==4.4.1 robotframework-requests==0.9.7 robotframework-selenium2library==3.0.0 robotframework-seleniumlibrary==6.8.0 robotframework-sshlibrary==3.8.0 scapy==2.6.1 scp==0.15.0 selenium==4.38.0 six==1.17.0 sniffio==1.3.1 sortedcontainers==2.4.0 soupsieve==2.8 trio==0.32.0 trio-websocket==0.12.2 typing_extensions==4.15.0 urllib3==2.5.0 virtualenv==20.35.4 waitress==3.0.2 WebOb==1.8.9 websocket-client==1.9.0 WebTest==3.0.7 wsproto==1.3.1 [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties file path 'env.properties' [EnvInject] - Variables injected successfully. [controller-csit-3node-clustering-ask-all-scandium] $ /bin/bash -l /tmp/jenkins14619550809444357628.sh Setup pyenv: system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-scandium/.python-version) lf-activate-venv(): INFO: Reuse venv:/tmp/venv-CM3L from file:/tmp/.os_lf_venv lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv) lf-activate-venv(): INFO: Attempting to install with network-safe options... lf-activate-venv(): INFO: Base packages installed successfully lf-activate-venv(): INFO: Installing additional packages: python-heatclient python-openstackclient yq ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. lftools 0.37.15 requires urllib3<2.1.0, but you have urllib3 2.5.0 which is incompatible. kubernetes 34.1.0 requires urllib3<2.4.0,>=1.24.2, but you have urllib3 2.5.0 which is incompatible. lf-activate-venv(): INFO: Adding /tmp/venv-CM3L/bin to PATH + ODL_SYSTEM=() + TOOLS_SYSTEM=() + OPENSTACK_SYSTEM=() + OPENSTACK_CONTROLLERS=() + mapfile -t ADDR ++ openstack stack show -f json -c outputs releng-controller-csit-3node-clustering-ask-all-scandium-81 ++ jq -r '.outputs[] | select(.output_key | match("^vm_[0-9]+_ips$")) | .output_value | .[]' + for i in "${ADDR[@]}" ++ ssh 10.30.170.150 hostname -s Warning: Permanently added '10.30.170.150' (ECDSA) to the list of known hosts. + REMHOST=releng-47598-81-0-builder-0 + case ${REMHOST} in + ODL_SYSTEM=("${ODL_SYSTEM[@]}" "${i}") + for i in "${ADDR[@]}" ++ ssh 10.30.170.130 hostname -s Warning: Permanently added '10.30.170.130' (ECDSA) to the list of known hosts. + REMHOST=releng-47598-81-0-builder-1 + case ${REMHOST} in + ODL_SYSTEM=("${ODL_SYSTEM[@]}" "${i}") + for i in "${ADDR[@]}" ++ ssh 10.30.171.41 hostname -s Warning: Permanently added '10.30.171.41' (ECDSA) to the list of known hosts. + REMHOST=releng-47598-81-0-builder-2 + case ${REMHOST} in + ODL_SYSTEM=("${ODL_SYSTEM[@]}" "${i}") + echo NUM_ODL_SYSTEM=3 + echo NUM_TOOLS_SYSTEM=0 + '[' '' == yes ']' + NUM_OPENSTACK_SYSTEM=0 + echo NUM_OPENSTACK_SYSTEM=0 + '[' 0 -eq 2 ']' + echo ODL_SYSTEM_IP=10.30.170.150 ++ seq 0 2 + for i in $(seq 0 $(( ${#ODL_SYSTEM[@]} - 1 ))) + echo ODL_SYSTEM_1_IP=10.30.170.150 + for i in $(seq 0 $(( ${#ODL_SYSTEM[@]} - 1 ))) + echo ODL_SYSTEM_2_IP=10.30.170.130 + for i in $(seq 0 $(( ${#ODL_SYSTEM[@]} - 1 ))) + echo ODL_SYSTEM_3_IP=10.30.171.41 + echo TOOLS_SYSTEM_IP= ++ seq 0 -1 + openstack_index=0 + NUM_OPENSTACK_CONTROL_NODES=1 + echo NUM_OPENSTACK_CONTROL_NODES=1 ++ seq 0 0 + for i in $(seq 0 $((NUM_OPENSTACK_CONTROL_NODES - 1))) + echo OPENSTACK_CONTROL_NODE_1_IP= + NUM_OPENSTACK_COMPUTE_NODES=-1 + echo NUM_OPENSTACK_COMPUTE_NODES=-1 + '[' -1 -ge 2 ']' ++ seq 0 -2 + NUM_OPENSTACK_HAPROXY_NODES=0 + echo NUM_OPENSTACK_HAPROXY_NODES=0 ++ seq 0 -1 + echo 'Contents of slave_addresses.txt:' Contents of slave_addresses.txt: + cat slave_addresses.txt NUM_ODL_SYSTEM=3 NUM_TOOLS_SYSTEM=0 NUM_OPENSTACK_SYSTEM=0 ODL_SYSTEM_IP=10.30.170.150 ODL_SYSTEM_1_IP=10.30.170.150 ODL_SYSTEM_2_IP=10.30.170.130 ODL_SYSTEM_3_IP=10.30.171.41 TOOLS_SYSTEM_IP= NUM_OPENSTACK_CONTROL_NODES=1 OPENSTACK_CONTROL_NODE_1_IP= NUM_OPENSTACK_COMPUTE_NODES=-1 NUM_OPENSTACK_HAPROXY_NODES=0 [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties file path 'slave_addresses.txt' [EnvInject] - Variables injected successfully. [controller-csit-3node-clustering-ask-all-scandium] $ /bin/sh /tmp/jenkins9473968709096572530.sh Preparing for JRE Version 21 Karaf artifact is karaf Karaf project is integration Java home is /usr/lib/jvm/java-21-openjdk-amd64 [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties file path 'set_variables.env' [EnvInject] - Variables injected successfully. [controller-csit-3node-clustering-ask-all-scandium] $ /bin/bash /tmp/jenkins12733758024705104744.sh 2025-11-16 23:37:15 URL:https://raw.githubusercontent.com/opendaylight/integration-distribution/stable/scandium/pom.xml [2619/2619] -> "pom.xml" [1] Bundle version is 0.21.4-SNAPSHOT --2025-11-16 23:37:15-- https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.4-SNAPSHOT/maven-metadata.xml Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6 Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 1410 (1.4K) [application/xml] Saving to: ‘maven-metadata.xml’ 0K . 100% 32.8M=0s 2025-11-16 23:37:15 (32.8 MB/s) - ‘maven-metadata.xml’ saved [1410/1410] org.opendaylight.integration karaf 0.21.4-SNAPSHOT 20251116.213008 154 20251116213008 pom 0.21.4-20251116.213008-154 20251116213008 tar.gz 0.21.4-20251116.213008-154 20251116213008 zip 0.21.4-20251116.213008-154 20251116213008 cyclonedx xml 0.21.4-20251116.213008-154 20251116213008 cyclonedx json 0.21.4-20251116.213008-154 20251116213008 Nexus timestamp is 0.21.4-20251116.213008-154 Distribution bundle URL is https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.4-SNAPSHOT/karaf-0.21.4-20251116.213008-154.zip Distribution bundle is karaf-0.21.4-20251116.213008-154.zip Distribution bundle version is 0.21.4-SNAPSHOT Distribution folder is karaf-0.21.4-SNAPSHOT Nexus prefix is https://nexus.opendaylight.org [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties file path 'detect_variables.env' [EnvInject] - Variables injected successfully. [controller-csit-3node-clustering-ask-all-scandium] $ /bin/bash -l /tmp/jenkins5261403639033034936.sh Setup pyenv: system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-scandium/.python-version) lf-activate-venv(): INFO: Reuse venv:/tmp/venv-CM3L from file:/tmp/.os_lf_venv lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv) lf-activate-venv(): INFO: Attempting to install with network-safe options... ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. lftools 0.37.15 requires urllib3<2.1.0, but you have urllib3 2.5.0 which is incompatible. lf-activate-venv(): INFO: Base packages installed successfully lf-activate-venv(): INFO: Installing additional packages: python-heatclient python-openstackclient ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. lftools 0.37.15 requires urllib3<2.1.0, but you have urllib3 2.5.0 which is incompatible. lf-activate-venv(): INFO: Adding /tmp/venv-CM3L/bin to PATH Copying common-functions.sh to /tmp Copying common-functions.sh to 10.30.170.150:/tmp Warning: Permanently added '10.30.170.150' (ECDSA) to the list of known hosts. Copying common-functions.sh to 10.30.170.130:/tmp Warning: Permanently added '10.30.170.130' (ECDSA) to the list of known hosts. Copying common-functions.sh to 10.30.171.41:/tmp Warning: Permanently added '10.30.171.41' (ECDSA) to the list of known hosts. [controller-csit-3node-clustering-ask-all-scandium] $ /bin/bash /tmp/jenkins17718749544476133328.sh common-functions.sh is being sourced common-functions environment: MAVENCONF: /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ACTUALFEATURES: FEATURESCONF: /tmp/karaf-0.21.4-SNAPSHOT/etc/org.apache.karaf.features.cfg CUSTOMPROP: /tmp/karaf-0.21.4-SNAPSHOT/etc/custom.properties LOGCONF: /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg MEMCONF: /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv CONTROLLERMEM: 2048m AKKACONF: /tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/akka.conf MODULESCONF: /tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/modules.conf MODULESHARDSCONF: /tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/module-shards.conf SUITES: ################################################# ## Configure Cluster and Start ## ################################################# ACTUALFEATURES: odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app SPACE_SEPARATED_FEATURES: odl-integration-compatible-with-all odl-jolokia odl-restconf odl-clustering-test-app Locating script plan to use... script plan exists!!! Changing the script plan path... # Place the scripts in run order: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/scripts/car_people_shard.sh Executing /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/scripts/car_people_shard.sh... Add car-people shards file Copy shard config to member-1 with IP address 10.30.170.150 Warning: Permanently added '10.30.170.150' (ECDSA) to the list of known hosts. Copy shard config to member-2 with IP address 10.30.170.130 Warning: Permanently added '10.30.170.130' (ECDSA) to the list of known hosts. Copy shard config to member-3 with IP address 10.30.171.41 Warning: Permanently added '10.30.171.41' (ECDSA) to the list of known hosts. Finished running script plans Configuring member-1 with IP address 10.30.170.150 Warning: Permanently added '10.30.170.150' (ECDSA) to the list of known hosts. Warning: Permanently added '10.30.170.150' (ECDSA) to the list of known hosts. + source /tmp/common-functions.sh karaf-0.21.4-SNAPSHOT scandium ++ [[ /tmp/common-functions.sh == \/\t\m\p\/\c\o\n\f\i\g\u\r\a\t\i\o\n\-\s\c\r\i\p\t\.\s\h ]] common-functions.sh is being sourced ++ echo 'common-functions.sh is being sourced' ++ BUNDLEFOLDER=karaf-0.21.4-SNAPSHOT ++ DISTROSTREAM=scandium ++ export MAVENCONF=/tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ++ MAVENCONF=/tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ++ export FEATURESCONF=/tmp/karaf-0.21.4-SNAPSHOT/etc/org.apache.karaf.features.cfg ++ FEATURESCONF=/tmp/karaf-0.21.4-SNAPSHOT/etc/org.apache.karaf.features.cfg ++ export CUSTOMPROP=/tmp/karaf-0.21.4-SNAPSHOT/etc/custom.properties ++ CUSTOMPROP=/tmp/karaf-0.21.4-SNAPSHOT/etc/custom.properties ++ export LOGCONF=/tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg ++ LOGCONF=/tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg ++ export MEMCONF=/tmp/karaf-0.21.4-SNAPSHOT/bin/setenv ++ MEMCONF=/tmp/karaf-0.21.4-SNAPSHOT/bin/setenv ++ export CONTROLLERMEM= ++ CONTROLLERMEM= ++ case "${DISTROSTREAM}" in ++ CLUSTER_SYSTEM=akka ++ export AKKACONF=/tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/akka.conf ++ AKKACONF=/tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/akka.conf ++ export MODULESCONF=/tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/modules.conf ++ MODULESCONF=/tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/modules.conf ++ export MODULESHARDSCONF=/tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/module-shards.conf ++ MODULESHARDSCONF=/tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/module-shards.conf ++ print_common_env ++ cat common-functions environment: MAVENCONF: /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ACTUALFEATURES: FEATURESCONF: /tmp/karaf-0.21.4-SNAPSHOT/etc/org.apache.karaf.features.cfg CUSTOMPROP: /tmp/karaf-0.21.4-SNAPSHOT/etc/custom.properties LOGCONF: /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg MEMCONF: /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv CONTROLLERMEM: AKKACONF: /tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/akka.conf MODULESCONF: /tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/modules.conf MODULESHARDSCONF: /tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/module-shards.conf SUITES: ++ SSH='ssh -t -t' ++ extra_services_cntl=' dnsmasq.service httpd.service libvirtd.service openvswitch.service ovs-vswitchd.service ovsdb-server.service rabbitmq-server.service ' ++ extra_services_cmp=' libvirtd.service openvswitch.service ovs-vswitchd.service ovsdb-server.service ' Changing to /tmp Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.4-SNAPSHOT/karaf-0.21.4-20251116.213008-154.zip + echo 'Changing to /tmp' + cd /tmp + echo 'Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.4-SNAPSHOT/karaf-0.21.4-20251116.213008-154.zip' + wget --progress=dot:mega https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.4-SNAPSHOT/karaf-0.21.4-20251116.213008-154.zip --2025-11-16 23:37:30-- https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.4-SNAPSHOT/karaf-0.21.4-20251116.213008-154.zip Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6 Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 239811271 (229M) [application/zip] Saving to: ‘karaf-0.21.4-20251116.213008-154.zip’ 0K ........ ........ ........ ........ ........ ........ 1% 26.0M 9s 3072K ........ ........ ........ ........ ........ ........ 2% 49.1M 7s 6144K ........ ........ ........ ........ ........ ........ 3% 114M 5s 9216K ........ ........ ........ ........ ........ ........ 5% 102M 4s 12288K ........ ........ ........ ........ ........ ........ 6% 149M 4s 15360K ........ ........ ........ ........ ........ ........ 7% 177M 3s 18432K ........ ........ ........ ........ ........ ........ 9% 148M 3s 21504K ........ ........ ........ ........ ........ ........ 10% 179M 3s 24576K ........ ........ ........ ........ ........ ........ 11% 158M 2s 27648K ........ ........ ........ ........ ........ ........ 13% 160M 2s 30720K ........ ........ ........ ........ ........ ........ 14% 161M 2s 33792K ........ ........ ........ ........ ........ ........ 15% 156M 2s 36864K ........ ........ ........ ........ ........ ........ 17% 198M 2s 39936K ........ ........ ........ ........ ........ ........ 18% 233M 2s 43008K ........ ........ ........ ........ ........ ........ 19% 241M 2s 46080K ........ ........ ........ ........ ........ ........ 20% 254M 2s 49152K ........ ........ ........ ........ ........ ........ 22% 266M 2s 52224K ........ ........ ........ ........ ........ ........ 23% 235M 1s 55296K ........ ........ ........ ........ ........ ........ 24% 164M 1s 58368K ........ ........ ........ ........ ........ ........ 26% 172M 1s 61440K ........ ........ ........ ........ ........ ........ 27% 228M 1s 64512K ........ ........ ........ ........ ........ ........ 28% 248M 1s 67584K ........ ........ ........ ........ ........ ........ 30% 204M 1s 70656K ........ ........ ........ ........ ........ ........ 31% 225M 1s 73728K ........ ........ ........ ........ ........ ........ 32% 182M 1s 76800K ........ ........ ........ ........ ........ ........ 34% 205M 1s 79872K ........ ........ ........ ........ ........ ........ 35% 303M 1s 82944K ........ ........ ........ ........ ........ ........ 36% 276M 1s 86016K ........ ........ ........ ........ ........ ........ 38% 255M 1s 89088K ........ ........ ........ ........ ........ ........ 39% 291M 1s 92160K ........ ........ ........ ........ ........ ........ 40% 305M 1s 95232K ........ ........ ........ ........ ........ ........ 41% 194M 1s 98304K ........ ........ ........ ........ ........ ........ 43% 262M 1s 101376K ........ ........ ........ ........ ........ ........ 44% 199M 1s 104448K ........ ........ ........ ........ ........ ........ 45% 246M 1s 107520K ........ ........ ........ ........ ........ ........ 47% 199M 1s 110592K ........ ........ ........ ........ ........ ........ 48% 218M 1s 113664K ........ ........ ........ ........ ........ ........ 49% 179M 1s 116736K ........ ........ ........ ........ ........ ........ 51% 229M 1s 119808K ........ ........ ........ ........ ........ ........ 52% 246M 1s 122880K ........ ........ ........ ........ ........ ........ 53% 318M 1s 125952K ........ ........ ........ ........ ........ ........ 55% 329M 1s 129024K ........ ........ ........ ........ ........ ........ 56% 216M 1s 132096K ........ ........ ........ ........ ........ ........ 57% 320M 1s 135168K ........ ........ ........ ........ ........ ........ 59% 173M 1s 138240K ........ ........ ........ ........ ........ ........ 60% 244M 1s 141312K ........ ........ ........ ........ ........ ........ 61% 163M 1s 144384K ........ ........ ........ ........ ........ ........ 62% 272M 1s 147456K ........ ........ ........ ........ ........ ........ 64% 111M 0s 150528K ........ ........ ........ ........ ........ ........ 65% 189M 0s 153600K ........ ........ ........ ........ ........ ........ 66% 154M 0s 156672K ........ ........ ........ ........ ........ ........ 68% 215M 0s 159744K ........ ........ ........ ........ ........ ........ 69% 232M 0s 162816K ........ ........ ........ ........ ........ ........ 70% 221M 0s 165888K ........ ........ ........ ........ ........ ........ 72% 212M 0s 168960K ........ ........ ........ ........ ........ ........ 73% 328M 0s 172032K ........ ........ ........ ........ ........ ........ 74% 207M 0s 175104K ........ ........ ........ ........ ........ ........ 76% 233M 0s 178176K ........ ........ ........ ........ ........ ........ 77% 167M 0s 181248K ........ ........ ........ ........ ........ ........ 78% 289M 0s 184320K ........ ........ ........ ........ ........ ........ 80% 318M 0s 187392K ........ ........ ........ ........ ........ ........ 81% 181M 0s 190464K ........ ........ ........ ........ ........ ........ 82% 243M 0s 193536K ........ ........ ........ ........ ........ ........ 83% 220M 0s 196608K ........ ........ ........ ........ ........ ........ 85% 336M 0s 199680K ........ ........ ........ ........ ........ ........ 86% 341M 0s 202752K ........ ........ ........ ........ ........ ........ 87% 273M 0s 205824K ........ ........ ........ ........ ........ ........ 89% 196M 0s 208896K ........ ........ ........ ........ ........ ........ 90% 170M 0s 211968K ........ ........ ........ ........ ........ ........ 91% 283M 0s 215040K ........ ........ ........ ........ ........ ........ 93% 172M 0s 218112K ........ ........ ........ ........ ........ ........ 94% 253M 0s 221184K ........ ........ ........ ........ ........ ........ 95% 281M 0s 224256K ........ ........ ........ ........ ........ ........ 97% 199M 0s 227328K ........ ........ ........ ........ ........ ........ 98% 167M 0s 230400K ........ ........ ........ ........ ........ ........ 99% 242M 0s 233472K ........ ... 100% 274M=1.3s 2025-11-16 23:37:31 (183 MB/s) - ‘karaf-0.21.4-20251116.213008-154.zip’ saved [239811271/239811271] + echo 'Extracting the new controller...' Extracting the new controller... + unzip -q karaf-0.21.4-20251116.213008-154.zip Adding external repositories... + echo 'Adding external repositories...' + sed -ie 's%org.ops4j.pax.url.mvn.repositories=%org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases%g' /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg + cat /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ################################################################################ # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # ################################################################################ # # If set to true, the following property will not allow any certificate to be used # when accessing Maven repositories through SSL # #org.ops4j.pax.url.mvn.certificateCheck= # # Path to the local Maven settings file. # The repositories defined in this file will be automatically added to the list # of default repositories if the 'org.ops4j.pax.url.mvn.repositories' property # below is not set. # The following locations are checked for the existence of the settings.xml file # * 1. looks for the specified url # * 2. if not found looks for ${user.home}/.m2/settings.xml # * 3. if not found looks for ${maven.home}/conf/settings.xml # * 4. if not found looks for ${M2_HOME}/conf/settings.xml # #org.ops4j.pax.url.mvn.settings= # # Path to the local Maven repository which is used to avoid downloading # artifacts when they already exist locally. # The value of this property will be extracted from the settings.xml file # above, or defaulted to: # System.getProperty( "user.home" ) + "/.m2/repository" # org.ops4j.pax.url.mvn.localRepository=${karaf.home}/${karaf.default.repository} # # Default this to false. It's just weird to use undocumented repos # org.ops4j.pax.url.mvn.useFallbackRepositories=false # # Uncomment if you don't wanna use the proxy settings # from the Maven conf/settings.xml file # # org.ops4j.pax.url.mvn.proxySupport=false # # Comma separated list of repositories scanned when resolving an artifact. # Those repositories will be checked before iterating through the # below list of repositories and even before the local repository # A repository url can be appended with zero or more of the following flags: # @snapshots : the repository contains snaphots # @noreleases : the repository does not contain any released artifacts # # The following property value will add the system folder as a repo. # org.ops4j.pax.url.mvn.defaultRepositories=\ file:${karaf.home}/${karaf.default.repository}@id=system.repository@snapshots,\ file:${karaf.data}/kar@id=kar.repository@multi@snapshots,\ file:${karaf.base}/${karaf.default.repository}@id=child.system.repository@snapshots # Use the default local repo (e.g.~/.m2/repository) as a "remote" repo #org.ops4j.pax.url.mvn.defaultLocalRepoAsRemote=false # # Comma separated list of repositories scanned when resolving an artifact. # The default list includes the following repositories: # http://repo1.maven.org/maven2@id=central # http://repository.springsource.com/maven/bundles/release@id=spring.ebr # http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external # http://zodiac.springsource.com/maven/bundles/release@id=gemini # http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases # https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases # https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases # To add repositories to the default ones, prepend '+' to the list of repositories # to add. # A repository url can be appended with zero or more of the following flags: # @snapshots : the repository contains snapshots # @noreleases : the repository does not contain any released artifacts # @id=repository.id : the id for the repository, just like in the settings.xml this is optional but recommended # org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases ### ^^^ No remote repositories. This is the only ODL change compared to Karaf defaults.+ [[ True == \T\r\u\e ]] Configuring the startup features... + echo 'Configuring the startup features...' + sed -ie 's/\(featuresBoot=\|featuresBoot =\)/featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app,/g' /tmp/karaf-0.21.4-SNAPSHOT/etc/org.apache.karaf.features.cfg + FEATURE_TEST_STRING=features-test + FEATURE_TEST_VERSION=0.21.4-SNAPSHOT + KARAF_VERSION=karaf4 + [[ integration == \i\n\t\e\g\r\a\t\i\o\n ]] + sed -ie 's%\(featuresRepositories=\|featuresRepositories =\)%featuresRepositories = mvn:org.opendaylight.integration/features-test/0.21.4-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features,%g' /tmp/karaf-0.21.4-SNAPSHOT/etc/org.apache.karaf.features.cfg + [[ ! -z '' ]] + cat /tmp/karaf-0.21.4-SNAPSHOT/etc/org.apache.karaf.features.cfg ################################################################################ # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # ################################################################################ # # Comma separated list of features repositories to register by default # featuresRepositories = mvn:org.opendaylight.integration/features-test/0.21.4-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features, file:${karaf.etc}/171eb417-6dc9-46bd-88b2-7f603dcbe3d2.xml # # Comma separated list of features to install at startup # featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app, 0b6a8a79-5bd3-45f5-8554-dfc18897aaa0 # # Resource repositories (OBR) that the features resolver can use # to resolve requirements/capabilities # # The format of the resourceRepositories is # resourceRepositories=[xml:url|json:url],... # for Instance: # #resourceRepositories=xml:http://host/path/to/index.xml # or #resourceRepositories=json:http://host/path/to/index.json # # # Defines if the boot features are started in asynchronous mode (in a dedicated thread) # featuresBootAsynchronous=false # # Service requirements enforcement # # By default, the feature resolver checks the service requirements/capabilities of # bundles for new features (xml schema >= 1.3.0) in order to automatically installs # the required bundles. # The following flag can have those values: # - disable: service requirements are completely ignored # - default: service requirements are ignored for old features # - enforce: service requirements are always verified # #serviceRequirements=default # # Store cfg file for config element in feature # #configCfgStore=true # # Define if the feature service automatically refresh bundles # autoRefresh=true # # Configuration of features processing mechanism (overrides, blacklisting, modification of features) # XML file defines instructions related to features processing # versions.properties may declare properties to resolve placeholders in XML file # both files are relative to ${karaf.etc} # #featureProcessing=org.apache.karaf.features.xml #featureProcessingVersions=versions.properties + configure_karaf_log karaf4 '' + local -r karaf_version=karaf4 + local -r controllerdebugmap= + local logapi=log4j + grep log4j2 /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n log4j2.rootLogger.level = INFO #log4j2.rootLogger.type = asyncRoot #log4j2.rootLogger.includeLocation = false log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi log4j2.rootLogger.appenderRef.Console.ref = Console log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF} log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL log4j2.logger.spifly.name = org.apache.aries.spifly log4j2.logger.spifly.level = WARN log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit log4j2.logger.audit.level = INFO log4j2.logger.audit.additivity = false log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile # Console appender not used by default (see log4j2.rootLogger.appenderRefs) log4j2.appender.console.type = Console log4j2.appender.console.name = Console log4j2.appender.console.layout.type = PatternLayout log4j2.appender.console.layout.pattern = ${log4j2.pattern} log4j2.appender.rolling.type = RollingRandomAccessFile log4j2.appender.rolling.name = RollingFile log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i #log4j2.appender.rolling.immediateFlush = false log4j2.appender.rolling.append = true log4j2.appender.rolling.layout.type = PatternLayout log4j2.appender.rolling.layout.pattern = ${log4j2.pattern} log4j2.appender.rolling.policies.type = Policies log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy log4j2.appender.rolling.policies.size.size = 64MB log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy log4j2.appender.rolling.strategy.max = 7 log4j2.appender.audit.type = RollingRandomAccessFile log4j2.appender.audit.name = AuditRollingFile log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i log4j2.appender.audit.append = true log4j2.appender.audit.layout.type = PatternLayout log4j2.appender.audit.layout.pattern = ${log4j2.pattern} log4j2.appender.audit.policies.type = Policies log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy log4j2.appender.audit.policies.size.size = 8MB log4j2.appender.audit.strategy.type = DefaultRolloverStrategy log4j2.appender.audit.strategy.max = 7 log4j2.appender.osgi.type = PaxOsgi log4j2.appender.osgi.name = PaxOsgi log4j2.appender.osgi.filter = * #log4j2.logger.aether.name = shaded.org.eclipse.aether #log4j2.logger.aether.level = TRACE #log4j2.logger.http-headers.name = shaded.org.apache.http.headers #log4j2.logger.http-headers.level = DEBUG #log4j2.logger.maven.name = org.ops4j.pax.url.mvn #log4j2.logger.maven.level = TRACE Configuring the karaf log... karaf_version: karaf4, logapi: log4j2 + logapi=log4j2 + echo 'Configuring the karaf log... karaf_version: karaf4, logapi: log4j2' + '[' log4j2 == log4j2 ']' + sed -ie 's/log4j2.appender.rolling.policies.size.size = 64MB/log4j2.appender.rolling.policies.size.size = 1GB/g' /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg + orgmodule=org.opendaylight.yangtools.yang.parser.repo.YangTextSchemaContextResolver + orgmodule_=org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver + echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN' + echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN' controllerdebugmap: cat /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg + unset IFS + echo 'controllerdebugmap: ' + '[' -n '' ']' + echo 'cat /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg' + cat /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg ################################################################################ # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # ################################################################################ # Common pattern layout for appenders log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n # Root logger log4j2.rootLogger.level = INFO # uncomment to use asynchronous loggers, which require mvn:com.lmax/disruptor/3.3.2 library #log4j2.rootLogger.type = asyncRoot #log4j2.rootLogger.includeLocation = false log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi log4j2.rootLogger.appenderRef.Console.ref = Console log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF} # Filters for logs marked by org.opendaylight.odlparent.Markers log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL # Loggers configuration # Spifly logger log4j2.logger.spifly.name = org.apache.aries.spifly log4j2.logger.spifly.level = WARN # Security audit logger log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit log4j2.logger.audit.level = INFO log4j2.logger.audit.additivity = false log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile # Appenders configuration # Console appender not used by default (see log4j2.rootLogger.appenderRefs) log4j2.appender.console.type = Console log4j2.appender.console.name = Console log4j2.appender.console.layout.type = PatternLayout log4j2.appender.console.layout.pattern = ${log4j2.pattern} # Rolling file appender log4j2.appender.rolling.type = RollingRandomAccessFile log4j2.appender.rolling.name = RollingFile log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i # uncomment to not force a disk flush #log4j2.appender.rolling.immediateFlush = false log4j2.appender.rolling.append = true log4j2.appender.rolling.layout.type = PatternLayout log4j2.appender.rolling.layout.pattern = ${log4j2.pattern} log4j2.appender.rolling.policies.type = Policies log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy log4j2.appender.rolling.policies.size.size = 1GB log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy log4j2.appender.rolling.strategy.max = 7 # Audit file appender log4j2.appender.audit.type = RollingRandomAccessFile log4j2.appender.audit.name = AuditRollingFile log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i log4j2.appender.audit.append = true log4j2.appender.audit.layout.type = PatternLayout log4j2.appender.audit.layout.pattern = ${log4j2.pattern} log4j2.appender.audit.policies.type = Policies log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy log4j2.appender.audit.policies.size.size = 8MB log4j2.appender.audit.strategy.type = DefaultRolloverStrategy log4j2.appender.audit.strategy.max = 7 # OSGi appender log4j2.appender.osgi.type = PaxOsgi log4j2.appender.osgi.name = PaxOsgi log4j2.appender.osgi.filter = * # help with identification of maven-related problems with pax-url-aether #log4j2.logger.aether.name = shaded.org.eclipse.aether #log4j2.logger.aether.level = TRACE #log4j2.logger.http-headers.name = shaded.org.apache.http.headers #log4j2.logger.http-headers.level = DEBUG #log4j2.logger.maven.name = org.ops4j.pax.url.mvn #log4j2.logger.maven.level = TRACE log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN Configure + set_java_vars /usr/lib/jvm/java-21-openjdk-amd64 3072m /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv + local -r java_home=/usr/lib/jvm/java-21-openjdk-amd64 + local -r controllermem=3072m + local -r memconf=/tmp/karaf-0.21.4-SNAPSHOT/bin/setenv + echo Configure java home: /usr/lib/jvm/java-21-openjdk-amd64 max memory: 3072m memconf: /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv + echo ' java home: /usr/lib/jvm/java-21-openjdk-amd64' + echo ' max memory: 3072m' + echo ' memconf: /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv' + sed -ie 's%^# export JAVA_HOME%export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-21-openjdk-amd64}%g' /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv + sed -ie 's/JAVA_MAX_MEM="2048m"/JAVA_MAX_MEM=3072m/g' /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv cat /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv + echo 'cat /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv' + cat /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv #!/bin/sh # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # # # handle specific scripts; the SCRIPT_NAME is exactly the name of the Karaf # script: client, instance, shell, start, status, stop, karaf # # if [ "${KARAF_SCRIPT}" == "SCRIPT_NAME" ]; then # Actions go here... # fi # # general settings which should be applied for all scripts go here; please keep # in mind that it is possible that scripts might be executed more than once, e.g. # in example of the start script where the start script is executed first and the # karaf script afterwards. # # # The following section shows the possible configuration options for the default # karaf scripts # export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-21-openjdk-amd64} # Location of Java installation # export JAVA_OPTS # Generic JVM options, for instance, where you can pass the memory configuration # export JAVA_NON_DEBUG_OPTS # Additional non-debug JVM options # export EXTRA_JAVA_OPTS # Additional JVM options # export KARAF_HOME # Karaf home folder # export KARAF_DATA # Karaf data folder # export KARAF_BASE # Karaf base folder # export KARAF_ETC # Karaf etc folder # export KARAF_LOG # Karaf log folder # export KARAF_SYSTEM_OPTS # First citizen Karaf options # export KARAF_OPTS # Additional available Karaf options # export KARAF_DEBUG # Enable debug mode # export KARAF_REDIRECT # Enable/set the std/err redirection when using bin/start # export KARAF_NOROOT # Prevent execution as root if set to true Set Java version + echo 'Set Java version' + sudo /usr/sbin/alternatives --install /usr/bin/java java /usr/lib/jvm/java-21-openjdk-amd64/bin/java 1 sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper sudo: a password is required + sudo /usr/sbin/alternatives --set java /usr/lib/jvm/java-21-openjdk-amd64/bin/java sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper sudo: a password is required JDK default version ... + echo 'JDK default version ...' + java -version openjdk version "21.0.8" 2025-07-15 OpenJDK Runtime Environment (build 21.0.8+9-Ubuntu-0ubuntu122.04.1) OpenJDK 64-Bit Server VM (build 21.0.8+9-Ubuntu-0ubuntu122.04.1, mixed mode, sharing) Set JAVA_HOME + echo 'Set JAVA_HOME' + export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64 ++ readlink -e /usr/lib/jvm/java-21-openjdk-amd64/bin/java Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-21-openjdk-amd64/bin/java Listing all open ports on controller system... + JAVA_RESOLVED=/usr/lib/jvm/java-21-openjdk-amd64/bin/java + echo 'Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-21-openjdk-amd64/bin/java' + echo 'Listing all open ports on controller system...' + netstat -pnatu /tmp/configuration-script.sh: line 40: netstat: command not found Custom shard config exists!!! Copying the shard config... + '[' -f /tmp/custom_shard_config.txt ']' + echo 'Custom shard config exists!!!' + echo 'Copying the shard config...' + cp /tmp/custom_shard_config.txt /tmp/karaf-0.21.4-SNAPSHOT/bin/ Configuring cluster + echo 'Configuring cluster' + /tmp/karaf-0.21.4-SNAPSHOT/bin/configure_cluster.sh 1 10.30.170.150 10.30.170.130 10.30.171.41 ################################################ ## Configure Cluster ## ################################################ NOTE: Cluster configuration files not found. Copying from /tmp/karaf-0.21.4-SNAPSHOT/system/org/opendaylight/controller/sal-clustering-config/10.0.14 Configuring unique name in akka.conf Configuring hostname in akka.conf Configuring data and rpc seed nodes in akka.conf modules = [ { name = "inventory" namespace = "urn:opendaylight:inventory" shard-strategy = "module" }, { name = "topology" namespace = "urn:TBD:params:xml:ns:yang:network-topology" shard-strategy = "module" }, { name = "toaster" namespace = "http://netconfcentral.org/ns/toaster" shard-strategy = "module" }, { name = "car" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car" shard-strategy = "module" }, { name = "people" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people" shard-strategy = "module" }, { name = "car-people" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people" shard-strategy = "module" } ] Configuring replication type in module-shards.conf ################################################ ## NOTE: Manually restart controller to ## ## apply configuration. ## ################################################ Dump akka.conf + echo 'Dump akka.conf' + cat /tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/akka.conf odl-cluster-data { akka { remote { artery { enabled = on transport = tcp canonical.hostname = "10.30.170.150" canonical.port = 2550 } } cluster { # Using artery. seed-nodes = ["akka://opendaylight-cluster-data@10.30.170.150:2550", "akka://opendaylight-cluster-data@10.30.170.130:2550", "akka://opendaylight-cluster-data@10.30.171.41:2550"] roles = ["member-1"] # when under load we might trip a false positive on the failure detector # failure-detector { # heartbeat-interval = 4 s # acceptable-heartbeat-pause = 16s # } } persistence { # By default the snapshots/journal directories live in KARAF_HOME. You can choose to put it somewhere else by # modifying the following two properties. The directory location specified may be a relative or absolute path. # The relative path is always relative to KARAF_HOME. # snapshot-store.local.dir = "target/snapshots" # Use lz4 compression for LocalSnapshotStore snapshots snapshot-store.local.use-lz4-compression = false # Size of blocks for lz4 compression: 64KB, 256KB, 1MB or 4MB snapshot-store.local.lz4-blocksize = 256KB } disable-default-actor-system-quarantined-event-handling = "false" } } Dump modules.conf + echo 'Dump modules.conf' + cat /tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/modules.conf modules = [ { name = "inventory" namespace = "urn:opendaylight:inventory" shard-strategy = "module" }, { name = "topology" namespace = "urn:TBD:params:xml:ns:yang:network-topology" shard-strategy = "module" }, { name = "toaster" namespace = "http://netconfcentral.org/ns/toaster" shard-strategy = "module" }, { name = "car" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car" shard-strategy = "module" }, { name = "people" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people" shard-strategy = "module" }, { name = "car-people" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people" shard-strategy = "module" } ] Dump module-shards.conf + echo 'Dump module-shards.conf' + cat /tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/module-shards.conf module-shards = [ { name = "default" shards = [ { name = "default" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "inventory" shards = [ { name="inventory" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "topology" shards = [ { name="topology" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "toaster" shards = [ { name="toaster" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "car" shards = [ { name="car" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "people" shards = [ { name="people" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "car-people" shards = [ { name="car-people" replicas = ["member-1", "member-2", "member-3"] } ] } ] Configuring member-2 with IP address 10.30.170.130 Warning: Permanently added '10.30.170.130' (ECDSA) to the list of known hosts. Warning: Permanently added '10.30.170.130' (ECDSA) to the list of known hosts. + source /tmp/common-functions.sh karaf-0.21.4-SNAPSHOT scandium ++ [[ /tmp/common-functions.sh == \/\t\m\p\/\c\o\n\f\i\g\u\r\a\t\i\o\n\-\s\c\r\i\p\t\.\s\h ]] common-functions.sh is being sourced ++ echo 'common-functions.sh is being sourced' ++ BUNDLEFOLDER=karaf-0.21.4-SNAPSHOT ++ DISTROSTREAM=scandium ++ export MAVENCONF=/tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ++ MAVENCONF=/tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ++ export FEATURESCONF=/tmp/karaf-0.21.4-SNAPSHOT/etc/org.apache.karaf.features.cfg ++ FEATURESCONF=/tmp/karaf-0.21.4-SNAPSHOT/etc/org.apache.karaf.features.cfg ++ export CUSTOMPROP=/tmp/karaf-0.21.4-SNAPSHOT/etc/custom.properties ++ CUSTOMPROP=/tmp/karaf-0.21.4-SNAPSHOT/etc/custom.properties ++ export LOGCONF=/tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg ++ LOGCONF=/tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg ++ export MEMCONF=/tmp/karaf-0.21.4-SNAPSHOT/bin/setenv ++ MEMCONF=/tmp/karaf-0.21.4-SNAPSHOT/bin/setenv ++ export CONTROLLERMEM= ++ CONTROLLERMEM= ++ case "${DISTROSTREAM}" in ++ CLUSTER_SYSTEM=akka ++ export AKKACONF=/tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/akka.conf ++ AKKACONF=/tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/akka.conf ++ export MODULESCONF=/tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/modules.conf ++ MODULESCONF=/tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/modules.conf ++ export MODULESHARDSCONF=/tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/module-shards.conf ++ MODULESHARDSCONF=/tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/module-shards.conf ++ print_common_env ++ cat common-functions environment: MAVENCONF: /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ACTUALFEATURES: FEATURESCONF: /tmp/karaf-0.21.4-SNAPSHOT/etc/org.apache.karaf.features.cfg CUSTOMPROP: /tmp/karaf-0.21.4-SNAPSHOT/etc/custom.properties LOGCONF: /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg MEMCONF: /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv CONTROLLERMEM: AKKACONF: /tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/akka.conf MODULESCONF: /tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/modules.conf MODULESHARDSCONF: /tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/module-shards.conf SUITES: ++ SSH='ssh -t -t' ++ extra_services_cntl=' dnsmasq.service httpd.service libvirtd.service openvswitch.service ovs-vswitchd.service ovsdb-server.service rabbitmq-server.service ' ++ extra_services_cmp=' libvirtd.service openvswitch.service ovs-vswitchd.service ovsdb-server.service ' Changing to /tmp Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.4-SNAPSHOT/karaf-0.21.4-20251116.213008-154.zip + echo 'Changing to /tmp' + cd /tmp + echo 'Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.4-SNAPSHOT/karaf-0.21.4-20251116.213008-154.zip' + wget --progress=dot:mega https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.4-SNAPSHOT/karaf-0.21.4-20251116.213008-154.zip --2025-11-16 23:37:34-- https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.4-SNAPSHOT/karaf-0.21.4-20251116.213008-154.zip Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6 Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 239811271 (229M) [application/zip] Saving to: ‘karaf-0.21.4-20251116.213008-154.zip’ 0K ........ ........ ........ ........ ........ ........ 1% 84.4M 3s 3072K ........ ........ ........ ........ ........ ........ 2% 135M 2s 6144K ........ ........ ........ ........ ........ ........ 3% 149M 2s 9216K ........ ........ ........ ........ ........ ........ 5% 120M 2s 12288K ........ ........ ........ ........ ........ ........ 6% 177M 2s 15360K ........ ........ ........ ........ ........ ........ 7% 203M 2s 18432K ........ ........ ........ ........ ........ ........ 9% 198M 1s 21504K ........ ........ ........ ........ ........ ........ 10% 262M 1s 24576K ........ ........ ........ ........ ........ ........ 11% 281M 1s 27648K ........ ........ ........ ........ ........ ........ 13% 283M 1s 30720K ........ ........ ........ ........ ........ ........ 14% 314M 1s 33792K ........ ........ ........ ........ ........ ........ 15% 294M 1s 36864K ........ ........ ........ ........ ........ ........ 17% 332M 1s 39936K ........ ........ ........ ........ ........ ........ 18% 321M 1s 43008K ........ ........ ........ ........ ........ ........ 19% 363M 1s 46080K ........ ........ ........ ........ ........ ........ 20% 264M 1s 49152K ........ ........ ........ ........ ........ ........ 22% 317M 1s 52224K ........ ........ ........ ........ ........ ........ 23% 314M 1s 55296K ........ ........ ........ ........ ........ ........ 24% 346M 1s 58368K ........ ........ ........ ........ ........ ........ 26% 311M 1s 61440K ........ ........ ........ ........ ........ ........ 27% 281M 1s 64512K ........ ........ ........ ........ ........ ........ 28% 303M 1s 67584K ........ ........ ........ ........ ........ ........ 30% 306M 1s 70656K ........ ........ ........ ........ ........ ........ 31% 251M 1s 73728K ........ ........ ........ ........ ........ ........ 32% 214M 1s 76800K ........ ........ ........ ........ ........ ........ 34% 244M 1s 79872K ........ ........ ........ ........ ........ ........ 35% 294M 1s 82944K ........ ........ ........ ........ ........ ........ 36% 248M 1s 86016K ........ ........ ........ ........ ........ ........ 38% 253M 1s 89088K ........ ........ ........ ........ ........ ........ 39% 239M 1s 92160K ........ ........ ........ ........ ........ ........ 40% 234M 1s 95232K ........ ........ ........ ........ ........ ........ 41% 232M 1s 98304K ........ ........ ........ ........ ........ ........ 43% 216M 1s 101376K ........ ........ ........ ........ ........ ........ 44% 202M 1s 104448K ........ ........ ........ ........ ........ ........ 45% 252M 1s 107520K ........ ........ ........ ........ ........ ........ 47% 318M 1s 110592K ........ ........ ........ ........ ........ ........ 48% 294M 1s 113664K ........ ........ ........ ........ ........ ........ 49% 322M 0s 116736K ........ ........ ........ ........ ........ ........ 51% 225M 0s 119808K ........ ........ ........ ........ ........ ........ 52% 264M 0s 122880K ........ ........ ........ ........ ........ ........ 53% 284M 0s 125952K ........ ........ ........ ........ ........ ........ 55% 228M 0s 129024K ........ ........ ........ ........ ........ ........ 56% 231M 0s 132096K ........ ........ ........ ........ ........ ........ 57% 267M 0s 135168K ........ ........ ........ ........ ........ ........ 59% 299M 0s 138240K ........ ........ ........ ........ ........ ........ 60% 271M 0s 141312K ........ ........ ........ ........ ........ ........ 61% 287M 0s 144384K ........ ........ ........ ........ ........ ........ 62% 290M 0s 147456K ........ ........ ........ ........ ........ ........ 64% 317M 0s 150528K ........ ........ ........ ........ ........ ........ 65% 276M 0s 153600K ........ ........ ........ ........ ........ ........ 66% 192M 0s 156672K ........ ........ ........ ........ ........ ........ 68% 290M 0s 159744K ........ ........ ........ ........ ........ ........ 69% 279M 0s 162816K ........ ........ ........ ........ ........ ........ 70% 256M 0s 165888K ........ ........ ........ ........ ........ ........ 72% 248M 0s 168960K ........ ........ ........ ........ ........ ........ 73% 263M 0s 172032K ........ ........ ........ ........ ........ ........ 74% 200M 0s 175104K ........ ........ ........ ........ ........ ........ 76% 284M 0s 178176K ........ ........ ........ ........ ........ ........ 77% 280M 0s 181248K ........ ........ ........ ........ ........ ........ 78% 273M 0s 184320K ........ ........ ........ ........ ........ ........ 80% 296M 0s 187392K ........ ........ ........ ........ ........ ........ 81% 280M 0s 190464K ........ ........ ........ ........ ........ ........ 82% 275M 0s 193536K ........ ........ ........ ........ ........ ........ 83% 230M 0s 196608K ........ ........ ........ ........ ........ ........ 85% 260M 0s 199680K ........ ........ ........ ........ ........ ........ 86% 194M 0s 202752K ........ ........ ........ ........ ........ ........ 87% 279M 0s 205824K ........ ........ ........ ........ ........ ........ 89% 285M 0s 208896K ........ ........ ........ ........ ........ ........ 90% 213M 0s 211968K ........ ........ ........ ........ ........ ........ 91% 158M 0s 215040K ........ ........ ........ ........ ........ ........ 93% 245M 0s 218112K ........ ........ ........ ........ ........ ........ 94% 271M 0s 221184K ........ ........ ........ ........ ........ ........ 95% 224M 0s 224256K ........ ........ ........ ........ ........ ........ 97% 234M 0s 227328K ........ ........ ........ ........ ........ ........ 98% 198M 0s 230400K ........ ........ ........ ........ ........ ........ 99% 242M 0s 233472K ........ ... 100% 263M=0.9s 2025-11-16 23:37:35 (241 MB/s) - ‘karaf-0.21.4-20251116.213008-154.zip’ saved [239811271/239811271] Extracting the new controller... + echo 'Extracting the new controller...' + unzip -q karaf-0.21.4-20251116.213008-154.zip Adding external repositories... + echo 'Adding external repositories...' + sed -ie 's%org.ops4j.pax.url.mvn.repositories=%org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases%g' /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg + cat /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ################################################################################ # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # ################################################################################ # # If set to true, the following property will not allow any certificate to be used # when accessing Maven repositories through SSL # #org.ops4j.pax.url.mvn.certificateCheck= # # Path to the local Maven settings file. # The repositories defined in this file will be automatically added to the list # of default repositories if the 'org.ops4j.pax.url.mvn.repositories' property # below is not set. # The following locations are checked for the existence of the settings.xml file # * 1. looks for the specified url # * 2. if not found looks for ${user.home}/.m2/settings.xml # * 3. if not found looks for ${maven.home}/conf/settings.xml # * 4. if not found looks for ${M2_HOME}/conf/settings.xml # #org.ops4j.pax.url.mvn.settings= # # Path to the local Maven repository which is used to avoid downloading # artifacts when they already exist locally. # The value of this property will be extracted from the settings.xml file # above, or defaulted to: # System.getProperty( "user.home" ) + "/.m2/repository" # org.ops4j.pax.url.mvn.localRepository=${karaf.home}/${karaf.default.repository} # # Default this to false. It's just weird to use undocumented repos # org.ops4j.pax.url.mvn.useFallbackRepositories=false # # Uncomment if you don't wanna use the proxy settings # from the Maven conf/settings.xml file # # org.ops4j.pax.url.mvn.proxySupport=false # # Comma separated list of repositories scanned when resolving an artifact. # Those repositories will be checked before iterating through the # below list of repositories and even before the local repository # A repository url can be appended with zero or more of the following flags: # @snapshots : the repository contains snaphots # @noreleases : the repository does not contain any released artifacts # # The following property value will add the system folder as a repo. # org.ops4j.pax.url.mvn.defaultRepositories=\ file:${karaf.home}/${karaf.default.repository}@id=system.repository@snapshots,\ file:${karaf.data}/kar@id=kar.repository@multi@snapshots,\ file:${karaf.base}/${karaf.default.repository}@id=child.system.repository@snapshots # Use the default local repo (e.g.~/.m2/repository) as a "remote" repo #org.ops4j.pax.url.mvn.defaultLocalRepoAsRemote=false # # Comma separated list of repositories scanned when resolving an artifact. # The default list includes the following repositories: # http://repo1.maven.org/maven2@id=central # http://repository.springsource.com/maven/bundles/release@id=spring.ebr # http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external # http://zodiac.springsource.com/maven/bundles/release@id=gemini # http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases # https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases # https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases # To add repositories to the default ones, prepend '+' to the list of repositories # to add. # A repository url can be appended with zero or more of the following flags: # @snapshots : the repository contains snapshots # @noreleases : the repository does not contain any released artifacts # @id=repository.id : the id for the repository, just like in the settings.xml this is optional but recommended # org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases ### ^^^ No remote repositories. This is the only ODL change compared to Karaf defaults.Configuring the startup features... + [[ True == \T\r\u\e ]] + echo 'Configuring the startup features...' + sed -ie 's/\(featuresBoot=\|featuresBoot =\)/featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app,/g' /tmp/karaf-0.21.4-SNAPSHOT/etc/org.apache.karaf.features.cfg + FEATURE_TEST_STRING=features-test + FEATURE_TEST_VERSION=0.21.4-SNAPSHOT + KARAF_VERSION=karaf4 + [[ integration == \i\n\t\e\g\r\a\t\i\o\n ]] + sed -ie 's%\(featuresRepositories=\|featuresRepositories =\)%featuresRepositories = mvn:org.opendaylight.integration/features-test/0.21.4-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features,%g' /tmp/karaf-0.21.4-SNAPSHOT/etc/org.apache.karaf.features.cfg + [[ ! -z '' ]] + cat /tmp/karaf-0.21.4-SNAPSHOT/etc/org.apache.karaf.features.cfg ################################################################################ # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # ################################################################################ # # Comma separated list of features repositories to register by default # featuresRepositories = mvn:org.opendaylight.integration/features-test/0.21.4-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features, file:${karaf.etc}/171eb417-6dc9-46bd-88b2-7f603dcbe3d2.xml # # Comma separated list of features to install at startup # featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app, 0b6a8a79-5bd3-45f5-8554-dfc18897aaa0 # # Resource repositories (OBR) that the features resolver can use # to resolve requirements/capabilities # # The format of the resourceRepositories is # resourceRepositories=[xml:url|json:url],... # for Instance: # #resourceRepositories=xml:http://host/path/to/index.xml # or #resourceRepositories=json:http://host/path/to/index.json # # # Defines if the boot features are started in asynchronous mode (in a dedicated thread) # featuresBootAsynchronous=false # # Service requirements enforcement # # By default, the feature resolver checks the service requirements/capabilities of # bundles for new features (xml schema >= 1.3.0) in order to automatically installs # the required bundles. # The following flag can have those values: # - disable: service requirements are completely ignored # - default: service requirements are ignored for old features # - enforce: service requirements are always verified # #serviceRequirements=default # # Store cfg file for config element in feature # #configCfgStore=true # # Define if the feature service automatically refresh bundles # autoRefresh=true # # Configuration of features processing mechanism (overrides, blacklisting, modification of features) # XML file defines instructions related to features processing # versions.properties may declare properties to resolve placeholders in XML file # both files are relative to ${karaf.etc} # #featureProcessing=org.apache.karaf.features.xml #featureProcessingVersions=versions.properties + configure_karaf_log karaf4 '' + local -r karaf_version=karaf4 + local -r controllerdebugmap= + local logapi=log4j + grep log4j2 /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n log4j2.rootLogger.level = INFO #log4j2.rootLogger.type = asyncRoot #log4j2.rootLogger.includeLocation = false log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi log4j2.rootLogger.appenderRef.Console.ref = Console log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF} log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL log4j2.logger.spifly.name = org.apache.aries.spifly log4j2.logger.spifly.level = WARN log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit log4j2.logger.audit.level = INFO log4j2.logger.audit.additivity = false log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile # Console appender not used by default (see log4j2.rootLogger.appenderRefs) log4j2.appender.console.type = Console log4j2.appender.console.name = Console log4j2.appender.console.layout.type = PatternLayout log4j2.appender.console.layout.pattern = ${log4j2.pattern} log4j2.appender.rolling.type = RollingRandomAccessFile log4j2.appender.rolling.name = RollingFile log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i #log4j2.appender.rolling.immediateFlush = false log4j2.appender.rolling.append = true log4j2.appender.rolling.layout.type = PatternLayout log4j2.appender.rolling.layout.pattern = ${log4j2.pattern} log4j2.appender.rolling.policies.type = Policies log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy log4j2.appender.rolling.policies.size.size = 64MB log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy log4j2.appender.rolling.strategy.max = 7 log4j2.appender.audit.type = RollingRandomAccessFile log4j2.appender.audit.name = AuditRollingFile log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i log4j2.appender.audit.append = true log4j2.appender.audit.layout.type = PatternLayout log4j2.appender.audit.layout.pattern = ${log4j2.pattern} log4j2.appender.audit.policies.type = Policies log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy log4j2.appender.audit.policies.size.size = 8MB log4j2.appender.audit.strategy.type = DefaultRolloverStrategy log4j2.appender.audit.strategy.max = 7 log4j2.appender.osgi.type = PaxOsgi log4j2.appender.osgi.name = PaxOsgi log4j2.appender.osgi.filter = * #log4j2.logger.aether.name = shaded.org.eclipse.aether #log4j2.logger.aether.level = TRACE #log4j2.logger.http-headers.name = shaded.org.apache.http.headers #log4j2.logger.http-headers.level = DEBUG #log4j2.logger.maven.name = org.ops4j.pax.url.mvn #log4j2.logger.maven.level = TRACE Configuring the karaf log... karaf_version: karaf4, logapi: log4j2 + logapi=log4j2 + echo 'Configuring the karaf log... karaf_version: karaf4, logapi: log4j2' + '[' log4j2 == log4j2 ']' + sed -ie 's/log4j2.appender.rolling.policies.size.size = 64MB/log4j2.appender.rolling.policies.size.size = 1GB/g' /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg + orgmodule=org.opendaylight.yangtools.yang.parser.repo.YangTextSchemaContextResolver + orgmodule_=org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver + echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN' + echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN' controllerdebugmap: + unset IFS + echo 'controllerdebugmap: ' cat /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg + '[' -n '' ']' + echo 'cat /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg' + cat /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg ################################################################################ # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # ################################################################################ # Common pattern layout for appenders log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n # Root logger log4j2.rootLogger.level = INFO # uncomment to use asynchronous loggers, which require mvn:com.lmax/disruptor/3.3.2 library #log4j2.rootLogger.type = asyncRoot #log4j2.rootLogger.includeLocation = false log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi log4j2.rootLogger.appenderRef.Console.ref = Console log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF} # Filters for logs marked by org.opendaylight.odlparent.Markers log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL # Loggers configuration # Spifly logger log4j2.logger.spifly.name = org.apache.aries.spifly log4j2.logger.spifly.level = WARN # Security audit logger log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit log4j2.logger.audit.level = INFO log4j2.logger.audit.additivity = false log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile # Appenders configuration # Console appender not used by default (see log4j2.rootLogger.appenderRefs) log4j2.appender.console.type = Console log4j2.appender.console.name = Console log4j2.appender.console.layout.type = PatternLayout log4j2.appender.console.layout.pattern = ${log4j2.pattern} # Rolling file appender log4j2.appender.rolling.type = RollingRandomAccessFile log4j2.appender.rolling.name = RollingFile log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i # uncomment to not force a disk flush #log4j2.appender.rolling.immediateFlush = false log4j2.appender.rolling.append = true log4j2.appender.rolling.layout.type = PatternLayout log4j2.appender.rolling.layout.pattern = ${log4j2.pattern} log4j2.appender.rolling.policies.type = Policies log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy log4j2.appender.rolling.policies.size.size = 1GB log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy log4j2.appender.rolling.strategy.max = 7 # Audit file appender log4j2.appender.audit.type = RollingRandomAccessFile log4j2.appender.audit.name = AuditRollingFile log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i log4j2.appender.audit.append = true log4j2.appender.audit.layout.type = PatternLayout log4j2.appender.audit.layout.pattern = ${log4j2.pattern} log4j2.appender.audit.policies.type = Policies log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy log4j2.appender.audit.policies.size.size = 8MB log4j2.appender.audit.strategy.type = DefaultRolloverStrategy log4j2.appender.audit.strategy.max = 7 # OSGi appender log4j2.appender.osgi.type = PaxOsgi log4j2.appender.osgi.name = PaxOsgi log4j2.appender.osgi.filter = * # help with identification of maven-related problems with pax-url-aether #log4j2.logger.aether.name = shaded.org.eclipse.aether #log4j2.logger.aether.level = TRACE #log4j2.logger.http-headers.name = shaded.org.apache.http.headers #log4j2.logger.http-headers.level = DEBUG #log4j2.logger.maven.name = org.ops4j.pax.url.mvn #log4j2.logger.maven.level = TRACE log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN + set_java_vars /usr/lib/jvm/java-21-openjdk-amd64 3072m /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv + local -r java_home=/usr/lib/jvm/java-21-openjdk-amd64 + local -r controllermem=3072m Configure + local -r memconf=/tmp/karaf-0.21.4-SNAPSHOT/bin/setenv + echo Configure java home: /usr/lib/jvm/java-21-openjdk-amd64 max memory: 3072m + echo ' java home: /usr/lib/jvm/java-21-openjdk-amd64' + echo ' max memory: 3072m' + echo ' memconf: /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv' memconf: /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv + sed -ie 's%^# export JAVA_HOME%export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-21-openjdk-amd64}%g' /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv + sed -ie 's/JAVA_MAX_MEM="2048m"/JAVA_MAX_MEM=3072m/g' /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv cat /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv + echo 'cat /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv' + cat /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv #!/bin/sh # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # # # handle specific scripts; the SCRIPT_NAME is exactly the name of the Karaf # script: client, instance, shell, start, status, stop, karaf # # if [ "${KARAF_SCRIPT}" == "SCRIPT_NAME" ]; then # Actions go here... # fi # # general settings which should be applied for all scripts go here; please keep # in mind that it is possible that scripts might be executed more than once, e.g. # in example of the start script where the start script is executed first and the # karaf script afterwards. # # # The following section shows the possible configuration options for the default # karaf scripts # export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-21-openjdk-amd64} # Location of Java installation # export JAVA_OPTS # Generic JVM options, for instance, where you can pass the memory configuration # export JAVA_NON_DEBUG_OPTS # Additional non-debug JVM options # export EXTRA_JAVA_OPTS # Additional JVM options # export KARAF_HOME # Karaf home folder # export KARAF_DATA # Karaf data folder # export KARAF_BASE # Karaf base folder # export KARAF_ETC # Karaf etc folder # export KARAF_LOG # Karaf log folder # export KARAF_SYSTEM_OPTS # First citizen Karaf options # export KARAF_OPTS # Additional available Karaf options # export KARAF_DEBUG # Enable debug mode # export KARAF_REDIRECT # Enable/set the std/err redirection when using bin/start # export KARAF_NOROOT # Prevent execution as root if set to true Set Java version + echo 'Set Java version' + sudo /usr/sbin/alternatives --install /usr/bin/java java /usr/lib/jvm/java-21-openjdk-amd64/bin/java 1 sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper sudo: a password is required + sudo /usr/sbin/alternatives --set java /usr/lib/jvm/java-21-openjdk-amd64/bin/java sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper sudo: a password is required JDK default version ... + echo 'JDK default version ...' + java -version openjdk version "21.0.8" 2025-07-15 OpenJDK Runtime Environment (build 21.0.8+9-Ubuntu-0ubuntu122.04.1) OpenJDK 64-Bit Server VM (build 21.0.8+9-Ubuntu-0ubuntu122.04.1, mixed mode, sharing) Set JAVA_HOME + echo 'Set JAVA_HOME' + export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64 ++ readlink -e /usr/lib/jvm/java-21-openjdk-amd64/bin/java Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-21-openjdk-amd64/bin/java + JAVA_RESOLVED=/usr/lib/jvm/java-21-openjdk-amd64/bin/java + echo 'Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-21-openjdk-amd64/bin/java' Listing all open ports on controller system... + echo 'Listing all open ports on controller system...' + netstat -pnatu /tmp/configuration-script.sh: line 40: netstat: command not found Custom shard config exists!!! + '[' -f /tmp/custom_shard_config.txt ']' + echo 'Custom shard config exists!!!' + echo 'Copying the shard config...' Copying the shard config... + cp /tmp/custom_shard_config.txt /tmp/karaf-0.21.4-SNAPSHOT/bin/ Configuring cluster + echo 'Configuring cluster' + /tmp/karaf-0.21.4-SNAPSHOT/bin/configure_cluster.sh 2 10.30.170.150 10.30.170.130 10.30.171.41 ################################################ ## Configure Cluster ## ################################################ NOTE: Cluster configuration files not found. Copying from /tmp/karaf-0.21.4-SNAPSHOT/system/org/opendaylight/controller/sal-clustering-config/10.0.14 Configuring unique name in akka.conf Configuring hostname in akka.conf Configuring data and rpc seed nodes in akka.conf modules = [ { name = "inventory" namespace = "urn:opendaylight:inventory" shard-strategy = "module" }, { name = "topology" namespace = "urn:TBD:params:xml:ns:yang:network-topology" shard-strategy = "module" }, { name = "toaster" namespace = "http://netconfcentral.org/ns/toaster" shard-strategy = "module" }, { name = "car" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car" shard-strategy = "module" }, { name = "people" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people" shard-strategy = "module" }, { name = "car-people" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people" shard-strategy = "module" } ] Configuring replication type in module-shards.conf ################################################ ## NOTE: Manually restart controller to ## ## apply configuration. ## ################################################ Dump akka.conf + echo 'Dump akka.conf' + cat /tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/akka.conf odl-cluster-data { akka { remote { artery { enabled = on transport = tcp canonical.hostname = "10.30.170.130" canonical.port = 2550 } } cluster { # Using artery. seed-nodes = ["akka://opendaylight-cluster-data@10.30.170.150:2550", "akka://opendaylight-cluster-data@10.30.170.130:2550", "akka://opendaylight-cluster-data@10.30.171.41:2550"] roles = ["member-2"] # when under load we might trip a false positive on the failure detector # failure-detector { # heartbeat-interval = 4 s # acceptable-heartbeat-pause = 16s # } } persistence { # By default the snapshots/journal directories live in KARAF_HOME. You can choose to put it somewhere else by # modifying the following two properties. The directory location specified may be a relative or absolute path. # The relative path is always relative to KARAF_HOME. # snapshot-store.local.dir = "target/snapshots" # Use lz4 compression for LocalSnapshotStore snapshots snapshot-store.local.use-lz4-compression = false # Size of blocks for lz4 compression: 64KB, 256KB, 1MB or 4MB snapshot-store.local.lz4-blocksize = 256KB } disable-default-actor-system-quarantined-event-handling = "false" } } Dump modules.conf + echo 'Dump modules.conf' + cat /tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/modules.conf modules = [ { name = "inventory" namespace = "urn:opendaylight:inventory" shard-strategy = "module" }, { name = "topology" namespace = "urn:TBD:params:xml:ns:yang:network-topology" shard-strategy = "module" }, { name = "toaster" namespace = "http://netconfcentral.org/ns/toaster" shard-strategy = "module" }, { name = "car" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car" shard-strategy = "module" }, { name = "people" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people" shard-strategy = "module" }, { name = "car-people" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people" shard-strategy = "module" } ] Dump module-shards.conf + echo 'Dump module-shards.conf' + cat /tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/module-shards.conf module-shards = [ { name = "default" shards = [ { name = "default" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "inventory" shards = [ { name="inventory" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "topology" shards = [ { name="topology" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "toaster" shards = [ { name="toaster" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "car" shards = [ { name="car" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "people" shards = [ { name="people" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "car-people" shards = [ { name="car-people" replicas = ["member-1", "member-2", "member-3"] } ] } ] Configuring member-3 with IP address 10.30.171.41 Warning: Permanently added '10.30.171.41' (ECDSA) to the list of known hosts. Warning: Permanently added '10.30.171.41' (ECDSA) to the list of known hosts. + source /tmp/common-functions.sh karaf-0.21.4-SNAPSHOT scandium ++ [[ /tmp/common-functions.sh == \/\t\m\p\/\c\o\n\f\i\g\u\r\a\t\i\o\n\-\s\c\r\i\p\t\.\s\h ]] common-functions.sh is being sourced ++ echo 'common-functions.sh is being sourced' ++ BUNDLEFOLDER=karaf-0.21.4-SNAPSHOT ++ DISTROSTREAM=scandium ++ export MAVENCONF=/tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ++ MAVENCONF=/tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ++ export FEATURESCONF=/tmp/karaf-0.21.4-SNAPSHOT/etc/org.apache.karaf.features.cfg ++ FEATURESCONF=/tmp/karaf-0.21.4-SNAPSHOT/etc/org.apache.karaf.features.cfg ++ export CUSTOMPROP=/tmp/karaf-0.21.4-SNAPSHOT/etc/custom.properties ++ CUSTOMPROP=/tmp/karaf-0.21.4-SNAPSHOT/etc/custom.properties ++ export LOGCONF=/tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg ++ LOGCONF=/tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg ++ export MEMCONF=/tmp/karaf-0.21.4-SNAPSHOT/bin/setenv ++ MEMCONF=/tmp/karaf-0.21.4-SNAPSHOT/bin/setenv ++ export CONTROLLERMEM= ++ CONTROLLERMEM= ++ case "${DISTROSTREAM}" in ++ CLUSTER_SYSTEM=akka ++ export AKKACONF=/tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/akka.conf ++ AKKACONF=/tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/akka.conf ++ export MODULESCONF=/tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/modules.conf ++ MODULESCONF=/tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/modules.conf ++ export MODULESHARDSCONF=/tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/module-shards.conf ++ MODULESHARDSCONF=/tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/module-shards.conf ++ print_common_env ++ cat common-functions environment: MAVENCONF: /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ACTUALFEATURES: FEATURESCONF: /tmp/karaf-0.21.4-SNAPSHOT/etc/org.apache.karaf.features.cfg CUSTOMPROP: /tmp/karaf-0.21.4-SNAPSHOT/etc/custom.properties LOGCONF: /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg MEMCONF: /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv CONTROLLERMEM: AKKACONF: /tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/akka.conf MODULESCONF: /tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/modules.conf MODULESHARDSCONF: /tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/module-shards.conf SUITES: ++ SSH='ssh -t -t' ++ extra_services_cntl=' dnsmasq.service httpd.service libvirtd.service openvswitch.service ovs-vswitchd.service ovsdb-server.service rabbitmq-server.service ' ++ extra_services_cmp=' libvirtd.service openvswitch.service ovs-vswitchd.service ovsdb-server.service ' Changing to /tmp Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.4-SNAPSHOT/karaf-0.21.4-20251116.213008-154.zip + echo 'Changing to /tmp' + cd /tmp + echo 'Downloading the distribution from https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.4-SNAPSHOT/karaf-0.21.4-20251116.213008-154.zip' + wget --progress=dot:mega https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.4-SNAPSHOT/karaf-0.21.4-20251116.213008-154.zip --2025-11-16 23:37:38-- https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.4-SNAPSHOT/karaf-0.21.4-20251116.213008-154.zip Resolving nexus.opendaylight.org (nexus.opendaylight.org)... 199.204.45.87, 2604:e100:1:0:f816:3eff:fe45:48d6 Connecting to nexus.opendaylight.org (nexus.opendaylight.org)|199.204.45.87|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 239811271 (229M) [application/zip] Saving to: ‘karaf-0.21.4-20251116.213008-154.zip’ 0K ........ ........ ........ ........ ........ ........ 1% 54.2M 4s 3072K ........ ........ ........ ........ ........ ........ 2% 92.0M 3s 6144K ........ ........ ........ ........ ........ ........ 3% 128M 3s 9216K ........ ........ ........ ........ ........ ........ 5% 150M 2s 12288K ........ ........ ........ ........ ........ ........ 6% 137M 2s 15360K ........ ........ ........ ........ ........ ........ 7% 171M 2s 18432K ........ ........ ........ ........ ........ ........ 9% 225M 2s 21504K ........ ........ ........ ........ ........ ........ 10% 223M 2s 24576K ........ ........ ........ ........ ........ ........ 11% 241M 2s 27648K ........ ........ ........ ........ ........ ........ 13% 234M 1s 30720K ........ ........ ........ ........ ........ ........ 14% 273M 1s 33792K ........ ........ ........ ........ ........ ........ 15% 282M 1s 36864K ........ ........ ........ ........ ........ ........ 17% 295M 1s 39936K ........ ........ ........ ........ ........ ........ 18% 310M 1s 43008K ........ ........ ........ ........ ........ ........ 19% 314M 1s 46080K ........ ........ ........ ........ ........ ........ 20% 198M 1s 49152K ........ ........ ........ ........ ........ ........ 22% 287M 1s 52224K ........ ........ ........ ........ ........ ........ 23% 319M 1s 55296K ........ ........ ........ ........ ........ ........ 24% 356M 1s 58368K ........ ........ ........ ........ ........ ........ 26% 149M 1s 61440K ........ ........ ........ ........ ........ ........ 27% 295M 1s 64512K ........ ........ ........ ........ ........ ........ 28% 356M 1s 67584K ........ ........ ........ ........ ........ ........ 30% 345M 1s 70656K ........ ........ ........ ........ ........ ........ 31% 354M 1s 73728K ........ ........ ........ ........ ........ ........ 32% 353M 1s 76800K ........ ........ ........ ........ ........ ........ 34% 360M 1s 79872K ........ ........ ........ ........ ........ ........ 35% 369M 1s 82944K ........ ........ ........ ........ ........ ........ 36% 365M 1s 86016K ........ ........ ........ ........ ........ ........ 38% 238M 1s 89088K ........ ........ ........ ........ ........ ........ 39% 412M 1s 92160K ........ ........ ........ ........ ........ ........ 40% 432M 1s 95232K ........ ........ ........ ........ ........ ........ 41% 386M 1s 98304K ........ ........ ........ ........ ........ ........ 43% 244M 1s 101376K ........ ........ ........ ........ ........ ........ 44% 351M 1s 104448K ........ ........ ........ ........ ........ ........ 45% 434M 1s 107520K ........ ........ ........ ........ ........ ........ 47% 424M 1s 110592K ........ ........ ........ ........ ........ ........ 48% 245M 1s 113664K ........ ........ ........ ........ ........ ........ 49% 333M 1s 116736K ........ ........ ........ ........ ........ ........ 51% 317M 0s 119808K ........ ........ ........ ........ ........ ........ 52% 323M 0s 122880K ........ ........ ........ ........ ........ ........ 53% 354M 0s 125952K ........ ........ ........ ........ ........ ........ 55% 425M 0s 129024K ........ ........ ........ ........ ........ ........ 56% 395M 0s 132096K ........ ........ ........ ........ ........ ........ 57% 381M 0s 135168K ........ ........ ........ ........ ........ ........ 59% 237M 0s 138240K ........ ........ ........ ........ ........ ........ 60% 215M 0s 141312K ........ ........ ........ ........ ........ ........ 61% 330M 0s 144384K ........ ........ ........ ........ ........ ........ 62% 373M 0s 147456K ........ ........ ........ ........ ........ ........ 64% 385M 0s 150528K ........ ........ ........ ........ ........ ........ 65% 382M 0s 153600K ........ ........ ........ ........ ........ ........ 66% 373M 0s 156672K ........ ........ ........ ........ ........ ........ 68% 392M 0s 159744K ........ ........ ........ ........ ........ ........ 69% 363M 0s 162816K ........ ........ ........ ........ ........ ........ 70% 283M 0s 165888K ........ ........ ........ ........ ........ ........ 72% 249M 0s 168960K ........ ........ ........ ........ ........ ........ 73% 309M 0s 172032K ........ ........ ........ ........ ........ ........ 74% 353M 0s 175104K ........ ........ ........ ........ ........ ........ 76% 365M 0s 178176K ........ ........ ........ ........ ........ ........ 77% 349M 0s 181248K ........ ........ ........ ........ ........ ........ 78% 247M 0s 184320K ........ ........ ........ ........ ........ ........ 80% 345M 0s 187392K ........ ........ ........ ........ ........ ........ 81% 384M 0s 190464K ........ ........ ........ ........ ........ ........ 82% 350M 0s 193536K ........ ........ ........ ........ ........ ........ 83% 298M 0s 196608K ........ ........ ........ ........ ........ ........ 85% 295M 0s 199680K ........ ........ ........ ........ ........ ........ 86% 308M 0s 202752K ........ ........ ........ ........ ........ ........ 87% 291M 0s 205824K ........ ........ ........ ........ ........ ........ 89% 285M 0s 208896K ........ ........ ........ ........ ........ ........ 90% 266M 0s 211968K ........ ........ ........ ........ ........ ........ 91% 290M 0s 215040K ........ ........ ........ ........ ........ ........ 93% 274M 0s 218112K ........ ........ ........ ........ ........ ........ 94% 283M 0s 221184K ........ ........ ........ ........ ........ ........ 95% 264M 0s 224256K ........ ........ ........ ........ ........ ........ 97% 277M 0s 227328K ........ ........ ........ ........ ........ ........ 98% 256M 0s 230400K ........ ........ ........ ........ ........ ........ 99% 271M 0s 233472K ........ ... 100% 273M=0.9s 2025-11-16 23:37:38 (264 MB/s) - ‘karaf-0.21.4-20251116.213008-154.zip’ saved [239811271/239811271] Extracting the new controller... + echo 'Extracting the new controller...' + unzip -q karaf-0.21.4-20251116.213008-154.zip Adding external repositories... + echo 'Adding external repositories...' + sed -ie 's%org.ops4j.pax.url.mvn.repositories=%org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases%g' /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg + cat /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ################################################################################ # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # ################################################################################ # # If set to true, the following property will not allow any certificate to be used # when accessing Maven repositories through SSL # #org.ops4j.pax.url.mvn.certificateCheck= # # Path to the local Maven settings file. # The repositories defined in this file will be automatically added to the list # of default repositories if the 'org.ops4j.pax.url.mvn.repositories' property # below is not set. # The following locations are checked for the existence of the settings.xml file # * 1. looks for the specified url # * 2. if not found looks for ${user.home}/.m2/settings.xml # * 3. if not found looks for ${maven.home}/conf/settings.xml # * 4. if not found looks for ${M2_HOME}/conf/settings.xml # #org.ops4j.pax.url.mvn.settings= # # Path to the local Maven repository which is used to avoid downloading # artifacts when they already exist locally. # The value of this property will be extracted from the settings.xml file # above, or defaulted to: # System.getProperty( "user.home" ) + "/.m2/repository" # org.ops4j.pax.url.mvn.localRepository=${karaf.home}/${karaf.default.repository} # # Default this to false. It's just weird to use undocumented repos # org.ops4j.pax.url.mvn.useFallbackRepositories=false # # Uncomment if you don't wanna use the proxy settings # from the Maven conf/settings.xml file # # org.ops4j.pax.url.mvn.proxySupport=false # # Comma separated list of repositories scanned when resolving an artifact. # Those repositories will be checked before iterating through the # below list of repositories and even before the local repository # A repository url can be appended with zero or more of the following flags: # @snapshots : the repository contains snaphots # @noreleases : the repository does not contain any released artifacts # # The following property value will add the system folder as a repo. # org.ops4j.pax.url.mvn.defaultRepositories=\ file:${karaf.home}/${karaf.default.repository}@id=system.repository@snapshots,\ file:${karaf.data}/kar@id=kar.repository@multi@snapshots,\ file:${karaf.base}/${karaf.default.repository}@id=child.system.repository@snapshots # Use the default local repo (e.g.~/.m2/repository) as a "remote" repo #org.ops4j.pax.url.mvn.defaultLocalRepoAsRemote=false # # Comma separated list of repositories scanned when resolving an artifact. # The default list includes the following repositories: # http://repo1.maven.org/maven2@id=central # http://repository.springsource.com/maven/bundles/release@id=spring.ebr # http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external # http://zodiac.springsource.com/maven/bundles/release@id=gemini # http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases # https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases # https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases # To add repositories to the default ones, prepend '+' to the list of repositories # to add. # A repository url can be appended with zero or more of the following flags: # @snapshots : the repository contains snapshots # @noreleases : the repository does not contain any released artifacts # @id=repository.id : the id for the repository, just like in the settings.xml this is optional but recommended # org.ops4j.pax.url.mvn.repositories=https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot@id=opendaylight-snapshot@snapshots, https://nexus.opendaylight.org/content/repositories/public@id=opendaylight-mirror, http://repo1.maven.org/maven2@id=central, http://repository.springsource.com/maven/bundles/release@id=spring.ebr.release, http://repository.springsource.com/maven/bundles/external@id=spring.ebr.external, http://zodiac.springsource.com/maven/bundles/release@id=gemini, http://repository.apache.org/content/groups/snapshots-group@id=apache@snapshots@noreleases, https://oss.sonatype.org/content/repositories/snapshots@id=sonatype.snapshots.deploy@snapshots@noreleases, https://oss.sonatype.org/content/repositories/ops4j-snapshots@id=ops4j.sonatype.snapshots.deploy@snapshots@noreleases ### ^^^ No remote repositories. This is the only ODL change compared to Karaf defaults.Configuring the startup features... + [[ True == \T\r\u\e ]] + echo 'Configuring the startup features...' + sed -ie 's/\(featuresBoot=\|featuresBoot =\)/featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app,/g' /tmp/karaf-0.21.4-SNAPSHOT/etc/org.apache.karaf.features.cfg + FEATURE_TEST_STRING=features-test + FEATURE_TEST_VERSION=0.21.4-SNAPSHOT + KARAF_VERSION=karaf4 + [[ integration == \i\n\t\e\g\r\a\t\i\o\n ]] + sed -ie 's%\(featuresRepositories=\|featuresRepositories =\)%featuresRepositories = mvn:org.opendaylight.integration/features-test/0.21.4-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features,%g' /tmp/karaf-0.21.4-SNAPSHOT/etc/org.apache.karaf.features.cfg + [[ ! -z '' ]] + cat /tmp/karaf-0.21.4-SNAPSHOT/etc/org.apache.karaf.features.cfg ################################################################################ # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # ################################################################################ # # Comma separated list of features repositories to register by default # featuresRepositories = mvn:org.opendaylight.integration/features-test/0.21.4-SNAPSHOT/xml/features,mvn:org.apache.karaf.decanter/apache-karaf-decanter/1.2.0/xml/features, file:${karaf.etc}/171eb417-6dc9-46bd-88b2-7f603dcbe3d2.xml # # Comma separated list of features to install at startup # featuresBoot = odl-integration-compatible-with-all,odl-jolokia,odl-restconf,odl-clustering-test-app, 0b6a8a79-5bd3-45f5-8554-dfc18897aaa0 # # Resource repositories (OBR) that the features resolver can use # to resolve requirements/capabilities # # The format of the resourceRepositories is # resourceRepositories=[xml:url|json:url],... # for Instance: # #resourceRepositories=xml:http://host/path/to/index.xml # or #resourceRepositories=json:http://host/path/to/index.json # # # Defines if the boot features are started in asynchronous mode (in a dedicated thread) # featuresBootAsynchronous=false # # Service requirements enforcement # # By default, the feature resolver checks the service requirements/capabilities of # bundles for new features (xml schema >= 1.3.0) in order to automatically installs # the required bundles. # The following flag can have those values: # - disable: service requirements are completely ignored # - default: service requirements are ignored for old features # - enforce: service requirements are always verified # #serviceRequirements=default # # Store cfg file for config element in feature # #configCfgStore=true # # Define if the feature service automatically refresh bundles # autoRefresh=true # # Configuration of features processing mechanism (overrides, blacklisting, modification of features) # XML file defines instructions related to features processing # versions.properties may declare properties to resolve placeholders in XML file # both files are relative to ${karaf.etc} # #featureProcessing=org.apache.karaf.features.xml #featureProcessingVersions=versions.properties + configure_karaf_log karaf4 '' + local -r karaf_version=karaf4 + local -r controllerdebugmap= + local logapi=log4j + grep log4j2 /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n log4j2.rootLogger.level = INFO #log4j2.rootLogger.type = asyncRoot #log4j2.rootLogger.includeLocation = false log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi log4j2.rootLogger.appenderRef.Console.ref = Console log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF} log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL log4j2.logger.spifly.name = org.apache.aries.spifly log4j2.logger.spifly.level = WARN log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit log4j2.logger.audit.level = INFO log4j2.logger.audit.additivity = false log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile # Console appender not used by default (see log4j2.rootLogger.appenderRefs) log4j2.appender.console.type = Console log4j2.appender.console.name = Console log4j2.appender.console.layout.type = PatternLayout log4j2.appender.console.layout.pattern = ${log4j2.pattern} log4j2.appender.rolling.type = RollingRandomAccessFile log4j2.appender.rolling.name = RollingFile log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i #log4j2.appender.rolling.immediateFlush = false log4j2.appender.rolling.append = true log4j2.appender.rolling.layout.type = PatternLayout log4j2.appender.rolling.layout.pattern = ${log4j2.pattern} log4j2.appender.rolling.policies.type = Policies log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy log4j2.appender.rolling.policies.size.size = 64MB log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy log4j2.appender.rolling.strategy.max = 7 log4j2.appender.audit.type = RollingRandomAccessFile log4j2.appender.audit.name = AuditRollingFile log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i log4j2.appender.audit.append = true log4j2.appender.audit.layout.type = PatternLayout log4j2.appender.audit.layout.pattern = ${log4j2.pattern} log4j2.appender.audit.policies.type = Policies log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy log4j2.appender.audit.policies.size.size = 8MB log4j2.appender.audit.strategy.type = DefaultRolloverStrategy log4j2.appender.audit.strategy.max = 7 log4j2.appender.osgi.type = PaxOsgi log4j2.appender.osgi.name = PaxOsgi log4j2.appender.osgi.filter = * #log4j2.logger.aether.name = shaded.org.eclipse.aether #log4j2.logger.aether.level = TRACE #log4j2.logger.http-headers.name = shaded.org.apache.http.headers #log4j2.logger.http-headers.level = DEBUG #log4j2.logger.maven.name = org.ops4j.pax.url.mvn #log4j2.logger.maven.level = TRACE Configuring the karaf log... karaf_version: karaf4, logapi: log4j2 + logapi=log4j2 + echo 'Configuring the karaf log... karaf_version: karaf4, logapi: log4j2' + '[' log4j2 == log4j2 ']' + sed -ie 's/log4j2.appender.rolling.policies.size.size = 64MB/log4j2.appender.rolling.policies.size.size = 1GB/g' /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg + orgmodule=org.opendaylight.yangtools.yang.parser.repo.YangTextSchemaContextResolver + orgmodule_=org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver + echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN' controllerdebugmap: cat /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg + echo 'log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN' + unset IFS + echo 'controllerdebugmap: ' + '[' -n '' ']' + echo 'cat /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg' + cat /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg ################################################################################ # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # ################################################################################ # Common pattern layout for appenders log4j2.pattern = %d{ISO8601} | %-5p | %-16t | %-32c{1} | %X{bundle.id} - %X{bundle.name} - %X{bundle.version} | %m%n # Root logger log4j2.rootLogger.level = INFO # uncomment to use asynchronous loggers, which require mvn:com.lmax/disruptor/3.3.2 library #log4j2.rootLogger.type = asyncRoot #log4j2.rootLogger.includeLocation = false log4j2.rootLogger.appenderRef.RollingFile.ref = RollingFile log4j2.rootLogger.appenderRef.PaxOsgi.ref = PaxOsgi log4j2.rootLogger.appenderRef.Console.ref = Console log4j2.rootLogger.appenderRef.Console.filter.threshold.type = ThresholdFilter log4j2.rootLogger.appenderRef.Console.filter.threshold.level = ${karaf.log.console:-OFF} # Filters for logs marked by org.opendaylight.odlparent.Markers log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.type = ContextMapFilter log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.type = KeyValuePair log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.key = slf4j.marker log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.pair1.value = CONFIDENTIAL log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.operator = or log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMatch = DENY log4j2.rootLogger.appenderRef.RollingFile.filter.confidential.onMismatch = NEUTRAL # Loggers configuration # Spifly logger log4j2.logger.spifly.name = org.apache.aries.spifly log4j2.logger.spifly.level = WARN # Security audit logger log4j2.logger.audit.name = org.apache.karaf.jaas.modules.audit log4j2.logger.audit.level = INFO log4j2.logger.audit.additivity = false log4j2.logger.audit.appenderRef.AuditRollingFile.ref = AuditRollingFile # Appenders configuration # Console appender not used by default (see log4j2.rootLogger.appenderRefs) log4j2.appender.console.type = Console log4j2.appender.console.name = Console log4j2.appender.console.layout.type = PatternLayout log4j2.appender.console.layout.pattern = ${log4j2.pattern} # Rolling file appender log4j2.appender.rolling.type = RollingRandomAccessFile log4j2.appender.rolling.name = RollingFile log4j2.appender.rolling.fileName = ${karaf.data}/log/karaf.log log4j2.appender.rolling.filePattern = ${karaf.data}/log/karaf.log.%i # uncomment to not force a disk flush #log4j2.appender.rolling.immediateFlush = false log4j2.appender.rolling.append = true log4j2.appender.rolling.layout.type = PatternLayout log4j2.appender.rolling.layout.pattern = ${log4j2.pattern} log4j2.appender.rolling.policies.type = Policies log4j2.appender.rolling.policies.size.type = SizeBasedTriggeringPolicy log4j2.appender.rolling.policies.size.size = 1GB log4j2.appender.rolling.strategy.type = DefaultRolloverStrategy log4j2.appender.rolling.strategy.max = 7 # Audit file appender log4j2.appender.audit.type = RollingRandomAccessFile log4j2.appender.audit.name = AuditRollingFile log4j2.appender.audit.fileName = ${karaf.data}/security/audit.log log4j2.appender.audit.filePattern = ${karaf.data}/security/audit.log.%i log4j2.appender.audit.append = true log4j2.appender.audit.layout.type = PatternLayout log4j2.appender.audit.layout.pattern = ${log4j2.pattern} log4j2.appender.audit.policies.type = Policies log4j2.appender.audit.policies.size.type = SizeBasedTriggeringPolicy log4j2.appender.audit.policies.size.size = 8MB log4j2.appender.audit.strategy.type = DefaultRolloverStrategy log4j2.appender.audit.strategy.max = 7 # OSGi appender log4j2.appender.osgi.type = PaxOsgi log4j2.appender.osgi.name = PaxOsgi log4j2.appender.osgi.filter = * # help with identification of maven-related problems with pax-url-aether #log4j2.logger.aether.name = shaded.org.eclipse.aether #log4j2.logger.aether.level = TRACE #log4j2.logger.http-headers.name = shaded.org.apache.http.headers #log4j2.logger.http-headers.level = DEBUG #log4j2.logger.maven.name = org.ops4j.pax.url.mvn #log4j2.logger.maven.level = TRACE log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.name = WARN log4j2.logger.org_opendaylight_yangtools_yang_parser_repo_YangTextSchemaContextResolver.level = WARN Configure + set_java_vars /usr/lib/jvm/java-21-openjdk-amd64 3072m /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv + local -r java_home=/usr/lib/jvm/java-21-openjdk-amd64 + local -r controllermem=3072m + local -r memconf=/tmp/karaf-0.21.4-SNAPSHOT/bin/setenv + echo Configure java home: /usr/lib/jvm/java-21-openjdk-amd64 max memory: 3072m memconf: /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv + echo ' java home: /usr/lib/jvm/java-21-openjdk-amd64' + echo ' max memory: 3072m' + echo ' memconf: /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv' + sed -ie 's%^# export JAVA_HOME%export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-21-openjdk-amd64}%g' /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv + sed -ie 's/JAVA_MAX_MEM="2048m"/JAVA_MAX_MEM=3072m/g' /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv cat /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv + echo 'cat /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv' + cat /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv #!/bin/sh # # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # # # handle specific scripts; the SCRIPT_NAME is exactly the name of the Karaf # script: client, instance, shell, start, status, stop, karaf # # if [ "${KARAF_SCRIPT}" == "SCRIPT_NAME" ]; then # Actions go here... # fi # # general settings which should be applied for all scripts go here; please keep # in mind that it is possible that scripts might be executed more than once, e.g. # in example of the start script where the start script is executed first and the # karaf script afterwards. # # # The following section shows the possible configuration options for the default # karaf scripts # export JAVA_HOME=${JAVA_HOME:-/usr/lib/jvm/java-21-openjdk-amd64} # Location of Java installation # export JAVA_OPTS # Generic JVM options, for instance, where you can pass the memory configuration # export JAVA_NON_DEBUG_OPTS # Additional non-debug JVM options # export EXTRA_JAVA_OPTS # Additional JVM options # export KARAF_HOME # Karaf home folder # export KARAF_DATA # Karaf data folder # export KARAF_BASE # Karaf base folder # export KARAF_ETC # Karaf etc folder # export KARAF_LOG # Karaf log folder # export KARAF_SYSTEM_OPTS # First citizen Karaf options # export KARAF_OPTS # Additional available Karaf options # export KARAF_DEBUG # Enable debug mode # export KARAF_REDIRECT # Enable/set the std/err redirection when using bin/start # export KARAF_NOROOT # Prevent execution as root if set to true Set Java version + echo 'Set Java version' + sudo /usr/sbin/alternatives --install /usr/bin/java java /usr/lib/jvm/java-21-openjdk-amd64/bin/java 1 sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper sudo: a password is required + sudo /usr/sbin/alternatives --set java /usr/lib/jvm/java-21-openjdk-amd64/bin/java sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper sudo: a password is required JDK default version ... + echo 'JDK default version ...' + java -version openjdk version "21.0.8" 2025-07-15 OpenJDK Runtime Environment (build 21.0.8+9-Ubuntu-0ubuntu122.04.1) OpenJDK 64-Bit Server VM (build 21.0.8+9-Ubuntu-0ubuntu122.04.1, mixed mode, sharing) Set JAVA_HOME + echo 'Set JAVA_HOME' + export JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64 + JAVA_HOME=/usr/lib/jvm/java-21-openjdk-amd64 ++ readlink -e /usr/lib/jvm/java-21-openjdk-amd64/bin/java Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-21-openjdk-amd64/bin/java Listing all open ports on controller system... + JAVA_RESOLVED=/usr/lib/jvm/java-21-openjdk-amd64/bin/java + echo 'Java binary pointed at by JAVA_HOME: /usr/lib/jvm/java-21-openjdk-amd64/bin/java' + echo 'Listing all open ports on controller system...' + netstat -pnatu /tmp/configuration-script.sh: line 40: netstat: command not found Custom shard config exists!!! + '[' -f /tmp/custom_shard_config.txt ']' + echo 'Custom shard config exists!!!' + echo 'Copying the shard config...' Copying the shard config... + cp /tmp/custom_shard_config.txt /tmp/karaf-0.21.4-SNAPSHOT/bin/ Configuring cluster + echo 'Configuring cluster' + /tmp/karaf-0.21.4-SNAPSHOT/bin/configure_cluster.sh 3 10.30.170.150 10.30.170.130 10.30.171.41 ################################################ ## Configure Cluster ## ################################################ NOTE: Cluster configuration files not found. Copying from /tmp/karaf-0.21.4-SNAPSHOT/system/org/opendaylight/controller/sal-clustering-config/10.0.14 Configuring unique name in akka.conf Configuring hostname in akka.conf Configuring data and rpc seed nodes in akka.conf modules = [ { name = "inventory" namespace = "urn:opendaylight:inventory" shard-strategy = "module" }, { name = "topology" namespace = "urn:TBD:params:xml:ns:yang:network-topology" shard-strategy = "module" }, { name = "toaster" namespace = "http://netconfcentral.org/ns/toaster" shard-strategy = "module" }, { name = "car" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car" shard-strategy = "module" }, { name = "people" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people" shard-strategy = "module" }, { name = "car-people" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people" shard-strategy = "module" } ] Configuring replication type in module-shards.conf ################################################ ## NOTE: Manually restart controller to ## ## apply configuration. ## ################################################ Dump akka.conf + echo 'Dump akka.conf' + cat /tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/akka.conf odl-cluster-data { akka { remote { artery { enabled = on transport = tcp canonical.hostname = "10.30.171.41" canonical.port = 2550 } } cluster { # Using artery. seed-nodes = ["akka://opendaylight-cluster-data@10.30.170.150:2550", "akka://opendaylight-cluster-data@10.30.170.130:2550", "akka://opendaylight-cluster-data@10.30.171.41:2550"] roles = ["member-3"] # when under load we might trip a false positive on the failure detector # failure-detector { # heartbeat-interval = 4 s # acceptable-heartbeat-pause = 16s # } } persistence { # By default the snapshots/journal directories live in KARAF_HOME. You can choose to put it somewhere else by # modifying the following two properties. The directory location specified may be a relative or absolute path. # The relative path is always relative to KARAF_HOME. # snapshot-store.local.dir = "target/snapshots" # Use lz4 compression for LocalSnapshotStore snapshots snapshot-store.local.use-lz4-compression = false # Size of blocks for lz4 compression: 64KB, 256KB, 1MB or 4MB snapshot-store.local.lz4-blocksize = 256KB } disable-default-actor-system-quarantined-event-handling = "false" } } Dump modules.conf + echo 'Dump modules.conf' + cat /tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/modules.conf modules = [ { name = "inventory" namespace = "urn:opendaylight:inventory" shard-strategy = "module" }, { name = "topology" namespace = "urn:TBD:params:xml:ns:yang:network-topology" shard-strategy = "module" }, { name = "toaster" namespace = "http://netconfcentral.org/ns/toaster" shard-strategy = "module" }, { name = "car" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car" shard-strategy = "module" }, { name = "people" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:people" shard-strategy = "module" }, { name = "car-people" namespace = "urn:opendaylight:params:xml:ns:yang:controller:config:sal-clustering-it:car-people" shard-strategy = "module" } ] Dump module-shards.conf + echo 'Dump module-shards.conf' + cat /tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/module-shards.conf module-shards = [ { name = "default" shards = [ { name = "default" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "inventory" shards = [ { name="inventory" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "topology" shards = [ { name="topology" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "toaster" shards = [ { name="toaster" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "car" shards = [ { name="car" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "people" shards = [ { name="people" replicas = ["member-1", "member-2", "member-3"] } ] }, { name = "car-people" shards = [ { name="car-people" replicas = ["member-1", "member-2", "member-3"] } ] } ] Locating config plan to use... Finished running config plans Starting member-1 with IP address 10.30.170.150 Warning: Permanently added '10.30.170.150' (ECDSA) to the list of known hosts. Warning: Permanently added '10.30.170.150' (ECDSA) to the list of known hosts. Redirecting karaf console output to karaf_console.log Starting controller... start: Redirecting Karaf output to /tmp/karaf-0.21.4-SNAPSHOT/data/log/karaf_console.log Starting member-2 with IP address 10.30.170.130 Warning: Permanently added '10.30.170.130' (ECDSA) to the list of known hosts. Warning: Permanently added '10.30.170.130' (ECDSA) to the list of known hosts. Redirecting karaf console output to karaf_console.log Starting controller... start: Redirecting Karaf output to /tmp/karaf-0.21.4-SNAPSHOT/data/log/karaf_console.log Starting member-3 with IP address 10.30.171.41 Warning: Permanently added '10.30.171.41' (ECDSA) to the list of known hosts. Warning: Permanently added '10.30.171.41' (ECDSA) to the list of known hosts. Redirecting karaf console output to karaf_console.log Starting controller... start: Redirecting Karaf output to /tmp/karaf-0.21.4-SNAPSHOT/data/log/karaf_console.log [controller-csit-3node-clustering-ask-all-scandium] $ /bin/bash /tmp/jenkins8209621920782153717.sh common-functions.sh is being sourced common-functions environment: MAVENCONF: /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.url.mvn.cfg ACTUALFEATURES: FEATURESCONF: /tmp/karaf-0.21.4-SNAPSHOT/etc/org.apache.karaf.features.cfg CUSTOMPROP: /tmp/karaf-0.21.4-SNAPSHOT/etc/custom.properties LOGCONF: /tmp/karaf-0.21.4-SNAPSHOT/etc/org.ops4j.pax.logging.cfg MEMCONF: /tmp/karaf-0.21.4-SNAPSHOT/bin/setenv CONTROLLERMEM: 2048m AKKACONF: /tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/akka.conf MODULESCONF: /tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/modules.conf MODULESHARDSCONF: /tmp/karaf-0.21.4-SNAPSHOT/configuration/initial/module-shards.conf SUITES: + echo '#################################################' ################################################# + echo '## Verify Cluster is UP ##' ## Verify Cluster is UP ## + echo '#################################################' ################################################# + create_post_startup_script + cat + copy_and_run_post_startup_script + seed_index=1 ++ seq 1 3 + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_1_IP + echo 'Execute the post startup script on controller 10.30.170.150' Execute the post startup script on controller 10.30.170.150 + scp /w/workspace/controller-csit-3node-clustering-ask-all-scandium/post-startup-script.sh 10.30.170.150:/tmp/ Warning: Permanently added '10.30.170.150' (ECDSA) to the list of known hosts. + ssh 10.30.170.150 'bash /tmp/post-startup-script.sh 1' Warning: Permanently added '10.30.170.150' (ECDSA) to the list of known hosts. /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found Waiting up to 3 minutes for controller to come up, checking every 5 seconds... 2025-11-16T23:38:28,285 | INFO | SystemReadyService-0 | SimpleSystemReadyMonitor | 282 - org.opendaylight.infrautils.ready-api - 7.1.7 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.] Controller is UP 2025-11-16T23:38:28,285 | INFO | SystemReadyService-0 | SimpleSystemReadyMonitor | 282 - org.opendaylight.infrautils.ready-api - 7.1.7 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.] Listing all open ports on controller system... /tmp/post-startup-script.sh: line 51: netstat: command not found looking for "BindException: Address already in use" in log file looking for "server is unhealthy" in log file + '[' 1 == 0 ']' + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_2_IP + echo 'Execute the post startup script on controller 10.30.170.130' Execute the post startup script on controller 10.30.170.130 + scp /w/workspace/controller-csit-3node-clustering-ask-all-scandium/post-startup-script.sh 10.30.170.130:/tmp/ Warning: Permanently added '10.30.170.130' (ECDSA) to the list of known hosts. + ssh 10.30.170.130 'bash /tmp/post-startup-script.sh 2' Warning: Permanently added '10.30.170.130' (ECDSA) to the list of known hosts. /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found Waiting up to 3 minutes for controller to come up, checking every 5 seconds... 2025-11-16T23:38:28,385 | INFO | SystemReadyService-0 | SimpleSystemReadyMonitor | 282 - org.opendaylight.infrautils.ready-api - 7.1.7 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.] Controller is UP 2025-11-16T23:38:28,385 | INFO | SystemReadyService-0 | SimpleSystemReadyMonitor | 282 - org.opendaylight.infrautils.ready-api - 7.1.7 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.] Listing all open ports on controller system... /tmp/post-startup-script.sh: line 51: netstat: command not found looking for "BindException: Address already in use" in log file looking for "server is unhealthy" in log file + '[' 2 == 0 ']' + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_3_IP + echo 'Execute the post startup script on controller 10.30.171.41' Execute the post startup script on controller 10.30.171.41 + scp /w/workspace/controller-csit-3node-clustering-ask-all-scandium/post-startup-script.sh 10.30.171.41:/tmp/ Warning: Permanently added '10.30.171.41' (ECDSA) to the list of known hosts. + ssh 10.30.171.41 'bash /tmp/post-startup-script.sh 3' Warning: Permanently added '10.30.171.41' (ECDSA) to the list of known hosts. /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found /tmp/post-startup-script.sh: line 4: netstat: command not found Waiting up to 3 minutes for controller to come up, checking every 5 seconds... 2025-11-16T23:38:27,271 | INFO | SystemReadyService-0 | SimpleSystemReadyMonitor | 282 - org.opendaylight.infrautils.ready-api - 7.1.7 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.] Controller is UP 2025-11-16T23:38:27,271 | INFO | SystemReadyService-0 | SimpleSystemReadyMonitor | 282 - org.opendaylight.infrautils.ready-api - 7.1.7 | System ready; AKA: Aye captain, all warp coils are now operating at peak efficiency! [M.] Listing all open ports on controller system... /tmp/post-startup-script.sh: line 51: netstat: command not found looking for "BindException: Address already in use" in log file looking for "server is unhealthy" in log file + '[' 0 == 0 ']' + seed_index=1 + dump_controller_threads ++ seq 1 3 + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_1_IP + echo 'Let'\''s take the karaf thread dump' Let's take the karaf thread dump + ssh 10.30.170.150 'sudo ps aux' Warning: Permanently added '10.30.170.150' (ECDSA) to the list of known hosts. ++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-ask-all-scandium/ps_before.log ++ grep -v grep ++ tr -s ' ' ++ cut -f2 '-d ' + pid=2091 + echo 'karaf main: org.apache.karaf.main.Main, pid:2091' karaf main: org.apache.karaf.main.Main, pid:2091 + ssh 10.30.170.150 '/usr/lib/jvm/java-21-openjdk-amd64/bin/jstack -l 2091' Warning: Permanently added '10.30.170.150' (ECDSA) to the list of known hosts. + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_2_IP + echo 'Let'\''s take the karaf thread dump' Let's take the karaf thread dump + ssh 10.30.170.130 'sudo ps aux' Warning: Permanently added '10.30.170.130' (ECDSA) to the list of known hosts. ++ tr -s ' ' ++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-ask-all-scandium/ps_before.log ++ grep -v grep ++ cut -f2 '-d ' + pid=2093 + echo 'karaf main: org.apache.karaf.main.Main, pid:2093' karaf main: org.apache.karaf.main.Main, pid:2093 + ssh 10.30.170.130 '/usr/lib/jvm/java-21-openjdk-amd64/bin/jstack -l 2093' Warning: Permanently added '10.30.170.130' (ECDSA) to the list of known hosts. + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_3_IP + echo 'Let'\''s take the karaf thread dump' Let's take the karaf thread dump + ssh 10.30.171.41 'sudo ps aux' Warning: Permanently added '10.30.171.41' (ECDSA) to the list of known hosts. ++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-ask-all-scandium/ps_before.log ++ grep -v grep ++ tr -s ' ' ++ cut -f2 '-d ' + pid=2095 + echo 'karaf main: org.apache.karaf.main.Main, pid:2095' karaf main: org.apache.karaf.main.Main, pid:2095 + ssh 10.30.171.41 '/usr/lib/jvm/java-21-openjdk-amd64/bin/jstack -l 2095' Warning: Permanently added '10.30.171.41' (ECDSA) to the list of known hosts. + '[' 0 -gt 0 ']' + echo 'Generating controller variables...' Generating controller variables... ++ seq 1 3 + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_1_IP + odl_variables=' -v ODL_SYSTEM_1_IP:10.30.170.150' + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_2_IP + odl_variables=' -v ODL_SYSTEM_1_IP:10.30.170.150 -v ODL_SYSTEM_2_IP:10.30.170.130' + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_3_IP + odl_variables=' -v ODL_SYSTEM_1_IP:10.30.170.150 -v ODL_SYSTEM_2_IP:10.30.170.130 -v ODL_SYSTEM_3_IP:10.30.171.41' + echo 'Generating mininet variables...' Generating mininet variables... ++ seq 1 0 + get_test_suites SUITES + local __suite_list=SUITES + echo 'Locating test plan to use...' Locating test plan to use... + testplan_filepath=/w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/testplans/controller-clustering-ask-scandium.txt + '[' '!' -f /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/testplans/controller-clustering-ask-scandium.txt ']' + testplan_filepath=/w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/testplans/controller-clustering-ask.txt + '[' disabled '!=' disabled ']' + echo 'Changing the testplan path...' Changing the testplan path... + sed s:integration:/w/workspace/controller-csit-3node-clustering-ask-all-scandium: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/testplans/controller-clustering-ask.txt + cat testplan.txt # Place the suites in run order: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/action_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/action_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/singleton_service/global_rpc_kill.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/singleton_service/global_rpc_freeze.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/singleton_service/global_rpc_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot + '[' -z '' ']' ++ grep -E -v '(^[[:space:]]*#|^[[:space:]]*$)' testplan.txt ++ tr '\012' ' ' + suite_list='/w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/action_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/action_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/singleton_service/global_rpc_kill.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/singleton_service/global_rpc_freeze.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/singleton_service/global_rpc_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot ' + eval 'SUITES='\''/w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/action_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/action_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/singleton_service/global_rpc_kill.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/singleton_service/global_rpc_freeze.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/singleton_service/global_rpc_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot '\''' ++ SUITES='/w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/action_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/action_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/singleton_service/global_rpc_kill.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/singleton_service/global_rpc_freeze.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/singleton_service/global_rpc_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot ' + echo 'Starting Robot test suites /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/action_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/action_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/singleton_service/global_rpc_kill.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/singleton_service/global_rpc_freeze.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/singleton_service/global_rpc_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot ...' Starting Robot test suites /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/action_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/action_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/singleton_service/global_rpc_kill.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/singleton_service/global_rpc_freeze.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/singleton_service/global_rpc_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot ... + robot -N controller-clustering-ask.txt --removekeywords wuks -e exclude -e skip_if_scandium -v BUNDLEFOLDER:karaf-0.21.4-SNAPSHOT -v BUNDLE_URL:https://nexus.opendaylight.org/content/repositories/opendaylight.snapshot/org/opendaylight/integration/karaf/0.21.4-SNAPSHOT/karaf-0.21.4-20251116.213008-154.zip -v CONTROLLER:10.30.170.150 -v CONTROLLER1:10.30.170.130 -v CONTROLLER2:10.30.171.41 -v CONTROLLER_USER:jenkins -v JAVA_HOME:/usr/lib/jvm/java-21-openjdk-amd64 -v JDKVERSION:openjdk21 -v JENKINS_WORKSPACE:/w/workspace/controller-csit-3node-clustering-ask-all-scandium -v MININET: -v MININET1: -v MININET2: -v MININET_USER:jenkins -v NEXUSURL_PREFIX:https://nexus.opendaylight.org -v NUM_ODL_SYSTEM:3 -v NUM_TOOLS_SYSTEM:0 -v ODL_STREAM:scandium -v ODL_SYSTEM_IP:10.30.170.150 -v ODL_SYSTEM_1_IP:10.30.170.150 -v ODL_SYSTEM_2_IP:10.30.170.130 -v ODL_SYSTEM_3_IP:10.30.171.41 -v ODL_SYSTEM_USER:jenkins -v TOOLS_SYSTEM_IP: -v TOOLS_SYSTEM_USER:jenkins -v USER_HOME:/home/jenkins -v IS_KARAF_APPL:True -v WORKSPACE:/tmp /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/rpc_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/action_provider_precedence.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_rpc_broker/action_provider_partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/cluster_singleton/master_stability.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/cluster_singleton/partition_and_heal.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/cluster_singleton/chasing_the_leader.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/singleton_service/global_rpc_kill.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/singleton_service/global_rpc_freeze.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/singleton_service/global_rpc_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/dom_data_broker/restart_odl_with_tell_based_false.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/carpeople_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_persistence_recovery.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/buycar_failover.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/entity_isolate.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/buycar_failover_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_failover_crud_isolation.robot /w/workspace/controller-csit-3node-clustering-ask-all-scandium/test/csit/suites/controller/Clustering_Datastore/car_outage_corners.robot ============================================================================== controller-clustering-ask.txt ============================================================================== controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te... ============================================================================== Stop_All_Members :: Stop every odl node. | FAIL | Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1 ------------------------------------------------------------------------------ Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS | ------------------------------------------------------------------------------ Start_All_And_Sync :: Start each member and wait for sync. | FAIL | Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== controller-clustering-ask.txt.Rpc Provider Precedence :: DOMRpcBroker testi... ============================================================================== Register_Rpc_On_Each_Node :: Register global rpc on each node of t... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_rpc_provider_precedence_register_rpc_on_each_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_rpc_provider_precedence_register_rpc_on_each_node&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Invoke_Rpc_On_Each_Node :: Verify that the rpc response comes from... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_rpc_provider_precedence_invoke_rpc_on_each_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_rpc_provider_precedence_invoke_rpc_on_each_node&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Unregister_Rpc_On_Node :: Unregister the rpc on one of the cluster... | PASS | ------------------------------------------------------------------------------ Invoke_Rpc_On_Node_With_Unregistered_Rpc :: Invoke rcp on the node... | PASS | ------------------------------------------------------------------------------ Invoke_Rpc_On_Remaining_Nodes :: Verify that the rpc response come... | PASS | ------------------------------------------------------------------------------ Reregister_Rpc_On_Node :: Reregister the rpc. | PASS | ------------------------------------------------------------------------------ Invoke_Rpc_On_Each_Node_Again :: Verify that the rpc response come... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_rpc_provider_precedence_invoke_rpc_on_each_node_again" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_rpc_provider_precedence_invoke_rpc_on_each_node_again&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Unregister_Rpc_On_Each_Node :: Unregister rpc on every node. | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_rpc_provider_precedence_unregister_rpc_on_each_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_rpc_provider_precedence_unregister_rpc_on_each_node&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Rpc Provider Precedence :: DOMRpcBro... | FAIL | 8 tests, 4 passed, 4 failed ============================================================================== controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te... ============================================================================== Stop_All_Members :: Stop every odl node. | FAIL | Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1 ------------------------------------------------------------------------------ Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS | ------------------------------------------------------------------------------ Start_All_And_Sync :: Start each member and wait for sync. | FAIL | Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== controller-clustering-ask.txt.Rpc Provider Partition And Heal :: DOMRpcBrok... ============================================================================== Register_Rpc_On_Two_Nodes :: Register rpc on two nodes of the odl ... | PASS | ------------------------------------------------------------------------------ Invoke_Rpc_On_Each_Node :: Invoke get-constant rpc on every node o... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_rpc_provider_partition_and_heal_invoke_rpc_on_each_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_rpc_provider_partition_and_heal_invoke_rpc_on_each_node&order=bug_status" Keyword 'Verify_Constant_On_Active_Nodes' failed after retrying for 10 seconds. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Isolate_One_Node :: Isolate one node with registered rpc. From the... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_rpc_provider_partition_and_heal_isolate_one_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_rpc_provider_partition_and_heal_isolate_one_node&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Invoke_Rpc_On_Nonisolated_Nodes :: Invoke rpc on non-islolated nodes. | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_rpc_provider_partition_and_heal_invoke_rpc_on_nonisolated_nodes" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_rpc_provider_partition_and_heal_invoke_rpc_on_nonisolated_nodes&order=bug_status" Keyword 'DrbCommons.Verify_Constant_On_Active_Nodes' failed after retrying for 1 minute. The last error was: Keyword 'Verify_Constant_On_Active_Nodes' failed after retrying for 10 seconds. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Rejoin_Isolated_Member :: Rejoin isolated node | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_rpc_provider_partition_and_heal_rejoin_isolated_member" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_rpc_provider_partition_and_heal_rejoin_isolated_member&order=bug_status" Variable '${isolated_idx}' not found. ------------------------------------------------------------------------------ Invoke_Rpc_On_Each_Node_Again :: Invoke rpc get-constant on every ... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_rpc_provider_partition_and_heal_invoke_rpc_on_each_node_again" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_rpc_provider_partition_and_heal_invoke_rpc_on_each_node_again&order=bug_status" Keyword 'Verify_Constant_On_Active_Nodes' failed after retrying for 10 seconds. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Unregister_Rpc_On_Each_Node :: Inregister rpc on both nodes. | PASS | ------------------------------------------------------------------------------ controller-clustering-ask.txt.Rpc Provider Partition And Heal :: D... | FAIL | 7 tests, 2 passed, 5 failed ============================================================================== controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te... ============================================================================== Stop_All_Members :: Stop every odl node. | FAIL | Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1 ------------------------------------------------------------------------------ Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS | ------------------------------------------------------------------------------ Start_All_And_Sync :: Start each member and wait for sync. | FAIL | Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== controller-clustering-ask.txt.Action Provider Precedence :: DOMRpcBroker te... ============================================================================== Register_Rpc_On_Each_Node :: Register routed rpc on each node of t... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_action_provider_precedence_register_rpc_on_each_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_precedence_register_rpc_on_each_node&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Invoke_Rpc_On_Each_Node :: Verify that the rpc response comes from... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_action_provider_precedence_invoke_rpc_on_each_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_precedence_invoke_rpc_on_each_node&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Unregister_Rpc_On_Node :: Unregister the rpc on one of the cluster... | PASS | ------------------------------------------------------------------------------ Invoke_Rpc_On_Node_With_Unregistered_Rpc :: Invoke rcp on the node... | PASS | ------------------------------------------------------------------------------ Invoke_Rpc_On_Remaining_Nodes :: Verify that the rpc response come... | PASS | ------------------------------------------------------------------------------ Reregister_Rpc_On_Node :: Reregister the rpc. | PASS | ------------------------------------------------------------------------------ Invoke_Rpc_On_Each_Node_Again :: Verify that the rpc response come... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_action_provider_precedence_invoke_rpc_on_each_node_again" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_precedence_invoke_rpc_on_each_node_again&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Unregister_Rpc_On_Each_Node :: Unregister rpc on every node. | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_action_provider_precedence_unregister_rpc_on_each_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_precedence_unregister_rpc_on_each_node&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Action Provider Precedence :: DOMRpc... | FAIL | 8 tests, 4 passed, 4 failed ============================================================================== controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te... ============================================================================== Stop_All_Members :: Stop every odl node. | FAIL | Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1 ------------------------------------------------------------------------------ Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS | ------------------------------------------------------------------------------ Start_All_And_Sync :: Start each member and wait for sync. | FAIL | Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== controller-clustering-ask.txt.Action Provider Partition And Heal :: DOMRpcB... ============================================================================== Register_Rpc_On_Two_Nodes :: Register rpc on two nodes of the odl ... | PASS | ------------------------------------------------------------------------------ Invoke_Rpc_On_Each_Node :: Invoke get-contexted-constant rpc on ev... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_action_provider_partition_and_heal_invoke_rpc_on_each_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_partition_and_heal_invoke_rpc_on_each_node&order=bug_status" Keyword 'Verify_Contexted_Constant_On_Active_Nodes' failed after retrying for 10 seconds. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Isolate_One_Node :: Isolate one node with registered rpc. | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_action_provider_partition_and_heal_isolate_one_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_partition_and_heal_isolate_one_node&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Invoke_Rpc_On_Remaining_Nodes :: Invoke rpc on non-islolated nodes... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_action_provider_partition_and_heal_invoke_rpc_on_remaining_nodes" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_partition_and_heal_invoke_rpc_on_remaining_nodes&order=bug_status" Keyword 'DrbCommons.Verify_Contexted_Constant_On_Active_Nodes' failed after retrying for 1 minute. The last error was: Keyword 'Verify_Contexted_Constant_On_Active_Nodes' failed after retrying for 10 seconds. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Rejoin_Isolated_Member :: Rejoin isolated node | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_action_provider_partition_and_heal_rejoin_isolated_member" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_partition_and_heal_rejoin_isolated_member&order=bug_status" Variable '${isolated_idx}' not found. ------------------------------------------------------------------------------ Invoke_Rpc_On_Each_Node_Again :: Invoke rpc get-contexted-constant... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_action_provider_partition_and_heal_invoke_rpc_on_each_node_again" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_action_provider_partition_and_heal_invoke_rpc_on_each_node_again&order=bug_status" Keyword 'Verify_Contexted_Constant_On_Active_Nodes' failed after retrying for 10 seconds. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Unregister_Rpc_On_Each_Node :: Inregister rpc on both nodes. | PASS | ------------------------------------------------------------------------------ controller-clustering-ask.txt.Action Provider Partition And Heal :... | FAIL | 7 tests, 2 passed, 5 failed ============================================================================== controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te... ============================================================================== Stop_All_Members :: Stop every odl node. | FAIL | Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1 ------------------------------------------------------------------------------ Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS | ------------------------------------------------------------------------------ Start_All_And_Sync :: Start each member and wait for sync. | FAIL | Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== controller-clustering-ask.txt.Master Stability :: Cluster Singleton testing... ============================================================================== Register_Singleton_Constant_On_Each_Node_And_Verify :: Register a ... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_master_stability_register_singleton_constant_on_each_node_and_verify" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_master_stability_register_singleton_constant_on_each_node_and_verify&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Unregister_Singleton_Constant_On_Non_Master_Node :: Unregister the... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_master_stability_unregister_singleton_constant_on_non_master_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_master_stability_unregister_singleton_constant_on_non_master_node&order=bug_status" Variable '@{cs_candidates}' not found. Did you mean: @{cs_exp_candidates} ------------------------------------------------------------------------------ Monitor_Stability_While_Unregistered :: Verify that the owner rema... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_master_stability_monitor_stability_while_unregistered" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_master_stability_monitor_stability_while_unregistered&order=bug_status" Variable '${cs_owner}' not found. ------------------------------------------------------------------------------ Reregister_Singleton_Constant :: Re-register the unregistered cand... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_master_stability_reregister_singleton_constant" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_master_stability_reregister_singleton_constant&order=bug_status" Variable '${unregistered_node}' not found. ------------------------------------------------------------------------------ Verify_Stability_After_Reregistration :: Verify that the owner rem... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_master_stability_verify_stability_after_reregistration" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_master_stability_verify_stability_after_reregistration&order=bug_status" Variable '${cs_owner}' not found. ------------------------------------------------------------------------------ Unregister_Singleton_Constant_On_Each_Node :: Unregister the appli... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_master_stability_unregister_singleton_constant_on_each_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_master_stability_unregister_singleton_constant_on_each_node&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Master Stability :: Cluster Singleto... | FAIL | 6 tests, 0 passed, 6 failed ============================================================================== controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te... ============================================================================== Stop_All_Members :: Stop every odl node. | FAIL | Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1 ------------------------------------------------------------------------------ Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS | ------------------------------------------------------------------------------ Start_All_And_Sync :: Start each member and wait for sync. | FAIL | Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== controller-clustering-ask.txt.Partition And Heal :: Cluster Singleton testi... ============================================================================== Register_Singleton_Constant_On_Each_Node :: Register a candidate a... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_partition_and_heal_register_singleton_constant_on_each_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_partition_and_heal_register_singleton_constant_on_each_node&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ Verify_Singleton_Constant_On_Each_Node :: Store the owner and cand... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_partition_and_heal_verify_singleton_constant_on_each_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_partition_and_heal_verify_singleton_constant_on_each_node&order=bug_status" Could not parse owner and candidates for device get-singleton-constant-service'] ------------------------------------------------------------------------------ Isolate_Owner_Node :: Isolate the cluster node which is the owner.... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_partition_and_heal_isolate_owner_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_partition_and_heal_isolate_owner_node&order=bug_status" Variable '${cs_owner}' not found. ------------------------------------------------------------------------------ Monitor_Stability_While_Isolated :: Monitor the stability of the s... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_partition_and_heal_monitor_stability_while_isolated" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_partition_and_heal_monitor_stability_while_isolated&order=bug_status" Invalid IF condition: Evaluating expression '"${index}" == "${cs_isolated_index}"' failed: Variable '${cs_isolated_index}' not found. ------------------------------------------------------------------------------ Rejoin_Isolated_node :: Rejoin isolated node. | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_partition_and_heal_rejoin_isolated_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_partition_and_heal_rejoin_isolated_node&order=bug_status" Variable '${cs_isolated_index}' not found. ------------------------------------------------------------------------------ Unregister_Singleton_Constant_On_Each_Node :: Unregister the appli... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_partition_and_heal_unregister_singleton_constant_on_each_node" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_partition_and_heal_unregister_singleton_constant_on_each_node&order=bug_status" ReadTimeout: HTTPConnectionPool(host='10.30.171.41', port=8181): Read timed out. (read timeout=125.0) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Partition And Heal :: Cluster Single... | FAIL | 6 tests, 0 passed, 6 failed ============================================================================== controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te... ============================================================================== Stop_All_Members :: Stop every odl node. | FAIL | Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1 ------------------------------------------------------------------------------ Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS | ------------------------------------------------------------------------------ Start_All_And_Sync :: Start each member and wait for sync. | FAIL | Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore ------------------------------------------------------------------------------ controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== controller-clustering-ask.txt.Chasing The Leader :: Cluster Singleton testi... ============================================================================== Register_Candidates :: Register a candidate application on each no... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_chasing_the_leader_register_candidates" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_chasing_the_leader_register_candidates&order=bug_status" HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/operations/odl-mdsal-lowlevel-control:register-flapping-singleton ------------------------------------------------------------------------------ Do_Nothing :: Do nothing for the time of the test duration, becaus... | PASS | ------------------------------------------------------------------------------ Unregister_Candidates_And_Validate_Criteria :: Unregister the test... | FAIL | ... click for list of related bugs or create a new one if needed (with the "controller_clustering_ask_txt_chasing_the_leader_unregister_candidates_and_validate_criteria" reference somewhere inside) "https://bugs.opendaylight.org/buglist.cgi?f1=cf_external_ref&o1=substring&v1=controller_clustering_ask_txt_chasing_the_leader_unregister_candidates_and_validate_criteria&order=bug_status" HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/operations/odl-mdsal-lowlevel-control:unregister-flapping-singleton ------------------------------------------------------------------------------ controller-clustering-ask.txt.Chasing The Leader :: Cluster Single... | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te... ============================================================================== Stop_All_Members :: Stop every odl node. | FAIL | Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1 ------------------------------------------------------------------------------ Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS | ------------------------------------------------------------------------------ Start_All_And_Sync :: Start each member and wait for sync. | FAIL | Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore ------------------------------------------------------------------------------ controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== controller-clustering-ask.txt.Global Rpc Kill :: Controller functional HA t... ============================================================================== Get_Basic_Rpc_Test_Owner :: Find a service owner and successors. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_Before_Stopping_On_Owner :: Run rpc on the service owner. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_Before_Stop_On_Successors :: Run rpc on non owher cluster nodes. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Stop_Current_Owner_Member :: Stop cluster node which is the owner. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Verify_New_Basic_Rpc_Test_Owner_Elected :: Verify new owner of the... | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_On_Remained_Cluster_Nodes :: Run rpc on remained cluster nodes. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Restart_Stopped_Member :: Restart stopped node | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Verify_New_Owner_Remained_After_Rejoin :: Verify no owner change h... | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_After_Rejoin_On_New_Owner :: Run rpc on the new service owner ... | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_After_Rejoin_On_Old_Owner :: Run rpc on rejoined cluster node. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_After_Rejoin_On_All :: Run rpc again on all nodes. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ controller-clustering-ask.txt.Global Rpc Kill :: Controller functi... | FAIL | Suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . 11 tests, 0 passed, 11 failed ============================================================================== controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te... ============================================================================== Stop_All_Members :: Stop every odl node. | FAIL | Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1 ------------------------------------------------------------------------------ Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS | ------------------------------------------------------------------------------ Start_All_And_Sync :: Start each member and wait for sync. | FAIL | Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore ------------------------------------------------------------------------------ controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== controller-clustering-ask.txt.Global Rpc Freeze :: Controller functional HA... ============================================================================== Get_Basic_Rpc_Test_Owner :: Find a service owner and successors. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_Before_Freezing_On_Owner :: Run rpc on the service owner. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_Before_Freeze_On_Successors :: Run rpc on non owher cluster no... | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Freeze_Current_Owner_Member :: Stop cluster node which is the owner. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Verify_New_Basic_Rpc_Test_Owner_Elected :: Verify new owner of the... | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_On_Remained_Cluster_Nodes :: Run rpc on remained cluster nodes. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Unfreeze_Frozen_Member :: Restart frozen node | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_After_Rejoin_On_New_Owner :: Run rpc on the new service owner ... | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_After_Rejoin_On_Old_Owner :: Run rpc on rejoined cluster node. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_After_Rejoin_On_All :: Run rpc again on all nodes. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ controller-clustering-ask.txt.Global Rpc Freeze :: Controller func... | FAIL | Suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . 10 tests, 0 passed, 10 failed ============================================================================== controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te... ============================================================================== Stop_All_Members :: Stop every odl node. | FAIL | Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1 ------------------------------------------------------------------------------ Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS | ------------------------------------------------------------------------------ Start_All_And_Sync :: Start each member and wait for sync. | FAIL | Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore ------------------------------------------------------------------------------ controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== controller-clustering-ask.txt.Global Rpc Isolate :: Controller functional H... ============================================================================== Get_Basic_Rpc_Test_Owner :: Find a service owner and successors. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_Before_Isolation_On_Owner :: Run rpc on the service owner. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_Before_Isolation_On_Successors :: Run rpc on non owher cluster... | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Isolate_Current_Owner_Member :: Isolating cluster node which is th... | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Verify_New_Basic_Rpc_Test_Owner_Elected :: Verify new owner of the... | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_On_Isolated_Node :: Run rpc on isolated cluster node. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_On_Non_Isolated_Cluster_Nodes :: Run rpc on remained cluster n... | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rejoin_Isolated_Member :: Rejoin isolated node | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_After_Rejoin_On_New_Owner :: Run rpc on the new service owner ... | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_After_Rejoin_On_Old_Owner :: Run rpc on rejoined cluster node. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ Rpc_After_Rejoin_On_All :: Run rpc again on all nodes. | FAIL | Parent suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . ------------------------------------------------------------------------------ controller-clustering-ask.txt.Global Rpc Isolate :: Controller fun... | FAIL | Suite setup failed: No match found for 'opendaylight-user' in 10 seconds Output: . 11 tests, 0 passed, 11 failed ============================================================================== controller-clustering-ask.txt.Restart Odl With Tell Based False :: Unset te... ============================================================================== Stop_All_Members :: Stop every odl node. | FAIL | Keyword 'Verify_Karaf_Is_Not_Running_On_Member' failed after retrying for 6 minutes. The last error was: Found running Karaf count: 1: 0 != 1 ------------------------------------------------------------------------------ Unset_Tell_Based_Protocol_Usage :: Comment out the flag usage in c... | PASS | ------------------------------------------------------------------------------ Start_All_And_Sync :: Start each member and wait for sync. | FAIL | Keyword 'Verify_Members_Are_Ready' failed after retrying for 6 minutes. The last error was: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/jolokia/read/org.opendaylight.controller:Category=ShardManager,name=shard-manager-config,type=DistributedConfigDatastore ------------------------------------------------------------------------------ controller-clustering-ask.txt.Restart Odl With Tell Based False ::... | FAIL | 3 tests, 1 passed, 2 failed ============================================================================== controller-clustering-ask.txt.Carpeople Crud :: Suite for performing basic ... ============================================================================== Add_Cars_To_Leader :: Add 30 cars to car Leader by one big PUT. | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Added_Cars_On_Leader :: GET response from Leader should match ... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Added_Cars_On_Followers :: The same check on other members. | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Add_People_To_First_Follower :: Add 30 people to people first Foll... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Added_People_On_Leader :: GET response from Leader should matc... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Added_People_On_Followers :: The same check on other members. | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Buy_Cars_On_Leader :: Buy some cars on car-people Leader, loop of ... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Buy_Cars_On_Followers :: On each Follower buy corresponding ID seg... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Added_CarPeople_On_Leader :: GET car-person mappings from Lead... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Added_CarPeople_On_Followers :: The same check on other members. | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_All_CarPeople_On_Leader :: DELETE car-people container. No ... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_All_People_On_Leader :: DELETE people container. No verific... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_All_Cars_On_Leader :: DELETE cars container. No verificatio... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ controller-clustering-ask.txt.Carpeople Crud :: Suite for performi... | FAIL | Suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig 13 tests, 0 passed, 13 failed ============================================================================== controller-clustering-ask.txt.Car Failover Crud :: Suite mixing basic opera... ============================================================================== Add_Original_Cars_On_Old_Leader_And_Verify :: Add initial cars on ... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Stop_Original_Car_Leader :: Stop the car Leader to cause a new lea... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Wait_For_New_Leader :: Wait until new car Leader is elected. | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Original_Cars_On_New_Leader :: GET cars from new Leader, shoul... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Original_Cars_On_New_Followers :: The same check on other exis... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_Original_Cars_On_New_Leader :: Delete cars on the new Leader. | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Add_Leader_Cars_On_New_Leader :: Add cars on the new Leader. | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Leader_Cars_On_New_Leader :: GET cars from new Leader, should ... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Leader_Cars_On_New_Followers :: The same check on other existi... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_Leader_Cars_On_New_First_Follower :: Delete cars in new fir... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Add_Follower_Cars_On_New_First_Follower :: Add cars on the new fir... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Folower_Cars_On_New_Leader :: Get cars from the new Leader, sh... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Follower_Cars_On_New_Followers :: The same check on other exis... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Start_Old_Car_Leader :: Start the stopped member without deleting ... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Folower_Cars_On_Old_Leader :: GET cars from the restarted memb... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_Follower_Cars_On_New_Leader :: Delete cars on the last Leader. | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ controller-clustering-ask.txt.Car Failover Crud :: Suite mixing ba... | FAIL | Suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig 16 tests, 0 passed, 16 failed ============================================================================== controller-clustering-ask.txt.Car Persistence Recovery :: This test restart... ============================================================================== Add_Cars_On_Leader_And_Verify :: Single big PUT to datastore to ad... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Stop_All_Members :: Stop all controllers. | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Start_All_Members :: Start all controllers (should restore the per... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Memorize_Leader_And_Followers :: Locate current Leader of car Shard. | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Cars_On_Leader :: GET cars from Leader, should match the PUT d... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Cars_On_Followers :: The same check on other members. | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_Cars_On_Leader :: Delete cars on the new Leader. | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ controller-clustering-ask.txt.Car Persistence Recovery :: This tes... | FAIL | Suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig 7 tests, 0 passed, 7 failed ============================================================================== controller-clustering-ask.txt.Buycar Failover :: This test focuses on testi... ============================================================================== Add_Cars_To_Leader_And_Verify :: Add all needed cars to car Leader... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Add_People_To_First_Follower_And_Verify :: Add all needed people t... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Buy_Cars_On_Leader_And_Verify :: Buy some cars on the leader member. | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Buy_Cars_On_Follower_And_Verify :: Buy some cars on the first foll... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Reboot_People_Leader :: Previous people Leader is rebooted. We sho... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Buy_Cars_On_Leader_After_Reboot_And_Verify :: Buy some cars on the... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Buy_Cars_On_Follower_After_Reboot_And_Verify :: Buy some cars on t... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_All_CarPeople :: DELETE car-people container. No verificati... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_All_People :: DELETE people container. No verification beyo... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_All_Cars :: DELETE cars container. No verification beyond h... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ controller-clustering-ask.txt.Buycar Failover :: This test focuses... | FAIL | Suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig 10 tests, 0 passed, 10 failed ============================================================================== controller-clustering-ask.txt.Entity Isolate :: Suite for performing member... ============================================================================== Check All Shards Before Isolate :: Check all shards in controller. | FAIL | Parent suite setup failed: Evaluating expression 'json.loads(\'\'\'{\n "error": "javax.management.InstanceNotFoundException : org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "error_type": "javax.management.InstanceNotFoundException",\n "request": {\n "mbean": "org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "type": "read"\n },\n "stacktrace": "javax.management.InstanceNotFoundException: org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1073)\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBeanInfo(DefaultMBeanServerInterceptor.java:1343)\\n\\tat java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.getMBeanInfo(JmxMBeanServer.java:921)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:46)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:41)\\n\\tat org.jolokia.backend.executor.AbstractMBeanServerExecutor.call(AbstractMBeanServerExecutor.java:90)\\n\\tat org.jolokia.handler.ReadHandler.getMBeanInfo(ReadHandler.java:233)\\n\\tat org.jolokia.handler.ReadHandler.getAllAttributesNames(ReadHandler.java:245)\\n\\tat org.jolokia.handler.ReadHandler.resolveAttributes(ReadHandler.java:221)\\n\\tat org.jolokia.handler.ReadHandler.f... [ Message content over the limit has been removed. ] ...lipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\\n\\tat org.eclipse.jetty.server.Server.handle(Server.java:516)\\n\\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)\\n\\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)\\n\\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)\\n\\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)\\n\\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)\\n\\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)\\n\\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)\\n\\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)\\n\\tat java.base/java.lang.Thread.run(Thread.java:1583)\\n",\n "status": 404\n}\n\'\'\')' failed: JSONDecodeError: Invalid control character at: line 8 column 190 (char 619) ------------------------------------------------------------------------------ Isolate Entity Leader :: Isolate the entity-ownership Leader to ca... | FAIL | Parent suite setup failed: Evaluating expression 'json.loads(\'\'\'{\n "error": "javax.management.InstanceNotFoundException : org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "error_type": "javax.management.InstanceNotFoundException",\n "request": {\n "mbean": "org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "type": "read"\n },\n "stacktrace": "javax.management.InstanceNotFoundException: org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1073)\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBeanInfo(DefaultMBeanServerInterceptor.java:1343)\\n\\tat java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.getMBeanInfo(JmxMBeanServer.java:921)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:46)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:41)\\n\\tat org.jolokia.backend.executor.AbstractMBeanServerExecutor.call(AbstractMBeanServerExecutor.java:90)\\n\\tat org.jolokia.handler.ReadHandler.getMBeanInfo(ReadHandler.java:233)\\n\\tat org.jolokia.handler.ReadHandler.getAllAttributesNames(ReadHandler.java:245)\\n\\tat org.jolokia.handler.ReadHandler.resolveAttributes(ReadHandler.java:221)\\n\\tat org.jolokia.handler.ReadHandler.f... [ Message content over the limit has been removed. ] ...lipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\\n\\tat org.eclipse.jetty.server.Server.handle(Server.java:516)\\n\\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)\\n\\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)\\n\\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)\\n\\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)\\n\\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)\\n\\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)\\n\\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)\\n\\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)\\n\\tat java.base/java.lang.Thread.run(Thread.java:1583)\\n",\n "status": 404\n}\n\'\'\')' failed: JSONDecodeError: Invalid control character at: line 8 column 190 (char 619) ------------------------------------------------------------------------------ Check All Shards After Isolate :: Check all shards in controller. | FAIL | Parent suite setup failed: Evaluating expression 'json.loads(\'\'\'{\n "error": "javax.management.InstanceNotFoundException : org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "error_type": "javax.management.InstanceNotFoundException",\n "request": {\n "mbean": "org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "type": "read"\n },\n "stacktrace": "javax.management.InstanceNotFoundException: org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1073)\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBeanInfo(DefaultMBeanServerInterceptor.java:1343)\\n\\tat java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.getMBeanInfo(JmxMBeanServer.java:921)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:46)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:41)\\n\\tat org.jolokia.backend.executor.AbstractMBeanServerExecutor.call(AbstractMBeanServerExecutor.java:90)\\n\\tat org.jolokia.handler.ReadHandler.getMBeanInfo(ReadHandler.java:233)\\n\\tat org.jolokia.handler.ReadHandler.getAllAttributesNames(ReadHandler.java:245)\\n\\tat org.jolokia.handler.ReadHandler.resolveAttributes(ReadHandler.java:221)\\n\\tat org.jolokia.handler.ReadHandler.f... [ Message content over the limit has been removed. ] ...lipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\\n\\tat org.eclipse.jetty.server.Server.handle(Server.java:516)\\n\\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)\\n\\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)\\n\\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)\\n\\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)\\n\\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)\\n\\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)\\n\\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)\\n\\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)\\n\\tat java.base/java.lang.Thread.run(Thread.java:1583)\\n",\n "status": 404\n}\n\'\'\')' failed: JSONDecodeError: Invalid control character at: line 8 column 190 (char 619) ------------------------------------------------------------------------------ Rejoin Entity Leader :: Rejoin the entity-ownership Leader. | FAIL | Parent suite setup failed: Evaluating expression 'json.loads(\'\'\'{\n "error": "javax.management.InstanceNotFoundException : org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "error_type": "javax.management.InstanceNotFoundException",\n "request": {\n "mbean": "org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "type": "read"\n },\n "stacktrace": "javax.management.InstanceNotFoundException: org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1073)\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBeanInfo(DefaultMBeanServerInterceptor.java:1343)\\n\\tat java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.getMBeanInfo(JmxMBeanServer.java:921)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:46)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:41)\\n\\tat org.jolokia.backend.executor.AbstractMBeanServerExecutor.call(AbstractMBeanServerExecutor.java:90)\\n\\tat org.jolokia.handler.ReadHandler.getMBeanInfo(ReadHandler.java:233)\\n\\tat org.jolokia.handler.ReadHandler.getAllAttributesNames(ReadHandler.java:245)\\n\\tat org.jolokia.handler.ReadHandler.resolveAttributes(ReadHandler.java:221)\\n\\tat org.jolokia.handler.ReadHandler.f... [ Message content over the limit has been removed. ] ...lipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\\n\\tat org.eclipse.jetty.server.Server.handle(Server.java:516)\\n\\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)\\n\\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)\\n\\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)\\n\\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)\\n\\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)\\n\\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)\\n\\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)\\n\\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)\\n\\tat java.base/java.lang.Thread.run(Thread.java:1583)\\n",\n "status": 404\n}\n\'\'\')' failed: JSONDecodeError: Invalid control character at: line 8 column 190 (char 619) ------------------------------------------------------------------------------ Check All Shards After Rejoin :: Check all shards in controller. | FAIL | Parent suite setup failed: Evaluating expression 'json.loads(\'\'\'{\n "error": "javax.management.InstanceNotFoundException : org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "error_type": "javax.management.InstanceNotFoundException",\n "request": {\n "mbean": "org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "type": "read"\n },\n "stacktrace": "javax.management.InstanceNotFoundException: org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1073)\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBeanInfo(DefaultMBeanServerInterceptor.java:1343)\\n\\tat java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.getMBeanInfo(JmxMBeanServer.java:921)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:46)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:41)\\n\\tat org.jolokia.backend.executor.AbstractMBeanServerExecutor.call(AbstractMBeanServerExecutor.java:90)\\n\\tat org.jolokia.handler.ReadHandler.getMBeanInfo(ReadHandler.java:233)\\n\\tat org.jolokia.handler.ReadHandler.getAllAttributesNames(ReadHandler.java:245)\\n\\tat org.jolokia.handler.ReadHandler.resolveAttributes(ReadHandler.java:221)\\n\\tat org.jolokia.handler.ReadHandler.f... [ Message content over the limit has been removed. ] ...lipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\\n\\tat org.eclipse.jetty.server.Server.handle(Server.java:516)\\n\\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)\\n\\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)\\n\\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)\\n\\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)\\n\\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)\\n\\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)\\n\\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)\\n\\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)\\n\\tat java.base/java.lang.Thread.run(Thread.java:1583)\\n",\n "status": 404\n}\n\'\'\')' failed: JSONDecodeError: Invalid control character at: line 8 column 190 (char 619) ------------------------------------------------------------------------------ controller-clustering-ask.txt.Entity Isolate :: Suite for performi... | FAIL | Suite setup failed: Evaluating expression 'json.loads(\'\'\'{\n "error": "javax.management.InstanceNotFoundException : org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "error_type": "javax.management.InstanceNotFoundException",\n "request": {\n "mbean": "org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore",\n "type": "read"\n },\n "stacktrace": "javax.management.InstanceNotFoundException: org.opendaylight.controller:Category=Shards,name=member-1-shard-entity-ownership-operational,type=DistributedOperationalDatastore\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1073)\\n\\tat java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBeanInfo(DefaultMBeanServerInterceptor.java:1343)\\n\\tat java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.getMBeanInfo(JmxMBeanServer.java:921)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:46)\\n\\tat org.jolokia.handler.ReadHandler$1.execute(ReadHandler.java:41)\\n\\tat org.jolokia.backend.executor.AbstractMBeanServerExecutor.call(AbstractMBeanServerExecutor.java:90)\\n\\tat org.jolokia.handler.ReadHandler.getMBeanInfo(ReadHandler.java:233)\\n\\tat org.jolokia.handler.ReadHandler.getAllAttributesNames(ReadHandler.java:245)\\n\\tat org.jolokia.handler.ReadHandler.resolveAttributes(ReadHandler.java:221)\\n\\tat org.jolokia.handler.ReadHandler.f... [ Message content over the limit has been removed. ] ...lipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)\\n\\tat org.eclipse.jetty.server.Server.handle(Server.java:516)\\n\\tat org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)\\n\\tat org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)\\n\\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)\\n\\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)\\n\\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)\\n\\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)\\n\\tat org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173)\\n\\tat org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)\\n\\tat org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)\\n\\tat org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)\\n\\tat java.base/java.lang.Thread.run(Thread.java:1583)\\n",\n "status": 404\n}\n\'\'\')' failed: JSONDecodeError: Invalid control character at: line 8 column 190 (char 619) 5 tests, 0 passed, 5 failed ============================================================================== controller-clustering-ask.txt.Buycar Failover Isolation :: This test focuse... ============================================================================== Add_Cars_To_Leader_And_Verify :: Add all needed cars to car Leader... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Add_People_To_First_Follower_And_Verify :: Add all needed people t... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Buy_Cars_On_Leader_And_Verify :: Buy some cars on the leader member. | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Buy_Cars_On_Follower_And_Verify :: Buy some cars on the first foll... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Isolate_and_Rejoin_People_Leader :: Previous people Leader is isol... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Buy_Cars_On_Leader_After_Rejoin_And_Verify :: Buy some cars on the... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Buy_Cars_On_Follower_After_Rejoin_And_Verify :: Buy some cars on t... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_All_CarPeople :: DELETE car-people container. No verificati... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_All_People :: DELETE people container. No verification beyo... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_All_Cars :: DELETE cars container. No verification beyond h... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ controller-clustering-ask.txt.Buycar Failover Isolation :: This te... | FAIL | Suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig 10 tests, 0 passed, 10 failed ============================================================================== controller-clustering-ask.txt.Car Failover Crud Isolation :: Suite mixing b... ============================================================================== Add_Original_Cars_On_Old_Leader_And_Verify :: Add initial cars on ... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Isolate_Original_Car_Leader :: Isolate the car Leader to cause a n... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Wait_For_New_Leader :: Wait until new car Leader is elected. | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Original_Cars_On_New_Leader :: GET cars from new Leader, shoul... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Original_Cars_On_New_Followers :: The same check on other exis... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_Original_Cars_On_New_Leader :: Delete cars on the new Leader. | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Add_Leader_Cars_On_New_Leader :: Add cars on the new Leader. | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Leader_Cars_On_New_Leader :: GET cars from new Leader, should ... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Leader_Cars_On_New_Followers :: The same check on other existi... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_Leader_Cars_On_New_First_Follower :: Delete cars in new fir... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Add_Follower_Cars_On_New_First_Follower :: Add cars on the new fir... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Folower_Cars_On_New_Leader :: Get cars from the new Leader, sh... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Follower_Cars_On_New_Followers :: The same check on other exis... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Rejoin_Old_Car_Leader :: Rejoin the isolated member without deleti... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Folower_Cars_On_Old_Leader :: GET cars from the restarted memb... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_Follower_Cars_On_New_Leader :: Delete cars on the last Leader. | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ controller-clustering-ask.txt.Car Failover Crud Isolation :: Suite... | FAIL | Suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig 16 tests, 0 passed, 16 failed ============================================================================== controller-clustering-ask.txt.Car Outage Corners :: Cluster suite for testi... ============================================================================== Stop_Majority_Of_The_Followers :: Stop half plus one car Follower ... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Attempt_To_Add_Cars_To_Leader :: Adding cars should fail, as major... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Start_Tipping_Follower :: Start one Follower member without persis... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Add_Cars_On_Tipping_Follower :: Add cars on the tipping Follower. | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Cars_On_Existing_Members :: On each up member: GET cars, shoul... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Start_Other_Followers :: Start other followers without persisted d... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ See_Cars_On_New_Follower_Leader :: GET cars from a new follower to... | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ Delete_Cars_On_Leader :: Delete cars on Leader. | FAIL | Parent suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig ------------------------------------------------------------------------------ controller-clustering-ask.txt.Car Outage Corners :: Cluster suite ... | FAIL | Suite setup failed: HTTPError: 401 Client Error: Unauthorized for url: http://10.30.170.130:8181/rests/data/ietf-yang-library:modules-state?content=nonconfig 8 tests, 0 passed, 8 failed ============================================================================== controller-clustering-ask.txt | FAIL | 195 tests, 24 passed, 171 failed ============================================================================== Output: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/output.xml Log: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/log.html Report: /w/workspace/controller-csit-3node-clustering-ask-all-scandium/report.html + true + echo 'Examining the files in data/log and checking filesize' Examining the files in data/log and checking filesize + ssh 10.30.170.150 'ls -altr /tmp/karaf-0.21.4-SNAPSHOT/data/log/' Warning: Permanently added '10.30.170.150' (ECDSA) to the list of known hosts. total 8692 drwxrwxr-x 2 jenkins jenkins 4096 Nov 16 23:37 . -rw-rw-r-- 1 jenkins jenkins 1720 Nov 16 23:37 karaf_console.log drwxrwxr-x 9 jenkins jenkins 4096 Nov 17 03:30 .. -rw-rw-r-- 1 jenkins jenkins 8882636 Nov 17 03:42 karaf.log + ssh 10.30.170.150 'du -hs /tmp/karaf-0.21.4-SNAPSHOT/data/log/*' Warning: Permanently added '10.30.170.150' (ECDSA) to the list of known hosts. 8.5M /tmp/karaf-0.21.4-SNAPSHOT/data/log/karaf.log 4.0K /tmp/karaf-0.21.4-SNAPSHOT/data/log/karaf_console.log + ssh 10.30.170.130 'ls -altr /tmp/karaf-0.21.4-SNAPSHOT/data/log/' Warning: Permanently added '10.30.170.130' (ECDSA) to the list of known hosts. total 48252 drwxrwxr-x 2 jenkins jenkins 4096 Nov 16 23:37 . -rw-rw-r-- 1 jenkins jenkins 1720 Nov 16 23:37 karaf_console.log drwxrwxr-x 4 jenkins jenkins 4096 Nov 17 03:29 .. -rw-rw-r-- 1 jenkins jenkins 49392959 Nov 17 03:43 karaf.log + ssh 10.30.170.130 'du -hs /tmp/karaf-0.21.4-SNAPSHOT/data/log/*' Warning: Permanently added '10.30.170.130' (ECDSA) to the list of known hosts. 48M /tmp/karaf-0.21.4-SNAPSHOT/data/log/karaf.log 4.0K /tmp/karaf-0.21.4-SNAPSHOT/data/log/karaf_console.log + ssh 10.30.171.41 'ls -altr /tmp/karaf-0.21.4-SNAPSHOT/data/log/' Warning: Permanently added '10.30.171.41' (ECDSA) to the list of known hosts. total 6240 drwxrwxr-x 2 jenkins jenkins 4096 Nov 16 23:37 . -rw-rw-r-- 1 jenkins jenkins 145919 Nov 16 23:48 karaf_console.log drwxrwxr-x 4 jenkins jenkins 4096 Nov 17 03:29 .. -rw-rw-r-- 1 jenkins jenkins 6233261 Nov 17 03:42 karaf.log + ssh 10.30.171.41 'du -hs /tmp/karaf-0.21.4-SNAPSHOT/data/log/*' Warning: Permanently added '10.30.171.41' (ECDSA) to the list of known hosts. 6.0M /tmp/karaf-0.21.4-SNAPSHOT/data/log/karaf.log 144K /tmp/karaf-0.21.4-SNAPSHOT/data/log/karaf_console.log + set +e ++ seq 1 3 + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_1_IP + echo 'Let'\''s take the karaf thread dump again' Let's take the karaf thread dump again + ssh 10.30.170.150 'sudo ps aux' Warning: Permanently added '10.30.170.150' (ECDSA) to the list of known hosts. ++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-ask-all-scandium/ps_after.log ++ grep -v grep ++ cut -f2 '-d ' ++ tr -s ' ' + pid=29077 + echo 'karaf main: org.apache.karaf.main.Main, pid:29077' karaf main: org.apache.karaf.main.Main, pid:29077 + ssh 10.30.170.150 '/usr/lib/jvm/java-21-openjdk-amd64/bin/jstack -l 29077' Warning: Permanently added '10.30.170.150' (ECDSA) to the list of known hosts. + echo 'killing karaf process...' killing karaf process... + ssh 10.30.170.150 bash -c 'ps axf | grep karaf | grep -v grep | awk '\''{print "kill -9 " $1}'\'' | sh' Warning: Permanently added '10.30.170.150' (ECDSA) to the list of known hosts. + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_2_IP + echo 'Let'\''s take the karaf thread dump again' Let's take the karaf thread dump again + ssh 10.30.170.130 'sudo ps aux' Warning: Permanently added '10.30.170.130' (ECDSA) to the list of known hosts. ++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-ask-all-scandium/ps_after.log ++ cut -f2 '-d ' ++ grep -v grep ++ tr -s ' ' + pid=16821 + echo 'karaf main: org.apache.karaf.main.Main, pid:16821' karaf main: org.apache.karaf.main.Main, pid:16821 + ssh 10.30.170.130 '/usr/lib/jvm/java-21-openjdk-amd64/bin/jstack -l 16821' Warning: Permanently added '10.30.170.130' (ECDSA) to the list of known hosts. + echo 'killing karaf process...' killing karaf process... + ssh 10.30.170.130 bash -c 'ps axf | grep karaf | grep -v grep | awk '\''{print "kill -9 " $1}'\'' | sh' Warning: Permanently added '10.30.170.130' (ECDSA) to the list of known hosts. + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_3_IP + echo 'Let'\''s take the karaf thread dump again' Let's take the karaf thread dump again + ssh 10.30.171.41 'sudo ps aux' Warning: Permanently added '10.30.171.41' (ECDSA) to the list of known hosts. ++ grep org.apache.karaf.main.Main /w/workspace/controller-csit-3node-clustering-ask-all-scandium/ps_after.log ++ grep -v grep ++ tr -s ' ' ++ cut -f2 '-d ' + pid=2095 + echo 'karaf main: org.apache.karaf.main.Main, pid:2095' karaf main: org.apache.karaf.main.Main, pid:2095 + ssh 10.30.171.41 '/usr/lib/jvm/java-21-openjdk-amd64/bin/jstack -l 2095' Warning: Permanently added '10.30.171.41' (ECDSA) to the list of known hosts. + echo 'killing karaf process...' killing karaf process... + ssh 10.30.171.41 bash -c 'ps axf | grep karaf | grep -v grep | awk '\''{print "kill -9 " $1}'\'' | sh' Warning: Permanently added '10.30.171.41' (ECDSA) to the list of known hosts. + sleep 5 ++ seq 1 3 + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_1_IP + echo 'Compressing karaf.log 1' Compressing karaf.log 1 + ssh 10.30.170.150 gzip --best /tmp/karaf-0.21.4-SNAPSHOT/data/log/karaf.log Warning: Permanently added '10.30.170.150' (ECDSA) to the list of known hosts. + echo 'Fetching compressed karaf.log 1' Fetching compressed karaf.log 1 + scp 10.30.170.150:/tmp/karaf-0.21.4-SNAPSHOT/data/log/karaf.log.gz odl1_karaf.log.gz Warning: Permanently added '10.30.170.150' (ECDSA) to the list of known hosts. + ssh 10.30.170.150 rm -f /tmp/karaf-0.21.4-SNAPSHOT/data/log/karaf.log.gz Warning: Permanently added '10.30.170.150' (ECDSA) to the list of known hosts. + scp 10.30.170.150:/tmp/karaf-0.21.4-SNAPSHOT/data/log/karaf_console.log odl1_karaf_console.log Warning: Permanently added '10.30.170.150' (ECDSA) to the list of known hosts. + ssh 10.30.170.150 rm -f /tmp/karaf-0.21.4-SNAPSHOT/data/log/karaf_console.log Warning: Permanently added '10.30.170.150' (ECDSA) to the list of known hosts. + echo 'Fetch GC logs' Fetch GC logs + mkdir -p gclogs-1 + scp '10.30.170.150:/tmp/karaf-0.21.4-SNAPSHOT/data/log/*.log' gclogs-1/ Warning: Permanently added '10.30.170.150' (ECDSA) to the list of known hosts. scp: /tmp/karaf-0.21.4-SNAPSHOT/data/log/*.log: No such file or directory + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_2_IP + echo 'Compressing karaf.log 2' Compressing karaf.log 2 + ssh 10.30.170.130 gzip --best /tmp/karaf-0.21.4-SNAPSHOT/data/log/karaf.log Warning: Permanently added '10.30.170.130' (ECDSA) to the list of known hosts. + echo 'Fetching compressed karaf.log 2' Fetching compressed karaf.log 2 + scp 10.30.170.130:/tmp/karaf-0.21.4-SNAPSHOT/data/log/karaf.log.gz odl2_karaf.log.gz Warning: Permanently added '10.30.170.130' (ECDSA) to the list of known hosts. + ssh 10.30.170.130 rm -f /tmp/karaf-0.21.4-SNAPSHOT/data/log/karaf.log.gz Warning: Permanently added '10.30.170.130' (ECDSA) to the list of known hosts. + scp 10.30.170.130:/tmp/karaf-0.21.4-SNAPSHOT/data/log/karaf_console.log odl2_karaf_console.log Warning: Permanently added '10.30.170.130' (ECDSA) to the list of known hosts. + ssh 10.30.170.130 rm -f /tmp/karaf-0.21.4-SNAPSHOT/data/log/karaf_console.log Warning: Permanently added '10.30.170.130' (ECDSA) to the list of known hosts. + echo 'Fetch GC logs' Fetch GC logs + mkdir -p gclogs-2 + scp '10.30.170.130:/tmp/karaf-0.21.4-SNAPSHOT/data/log/*.log' gclogs-2/ Warning: Permanently added '10.30.170.130' (ECDSA) to the list of known hosts. scp: /tmp/karaf-0.21.4-SNAPSHOT/data/log/*.log: No such file or directory + for i in $(seq 1 "${NUM_ODL_SYSTEM}") + CONTROLLERIP=ODL_SYSTEM_3_IP + echo 'Compressing karaf.log 3' Compressing karaf.log 3 + ssh 10.30.171.41 gzip --best /tmp/karaf-0.21.4-SNAPSHOT/data/log/karaf.log Warning: Permanently added '10.30.171.41' (ECDSA) to the list of known hosts. + echo 'Fetching compressed karaf.log 3' Fetching compressed karaf.log 3 + scp 10.30.171.41:/tmp/karaf-0.21.4-SNAPSHOT/data/log/karaf.log.gz odl3_karaf.log.gz Warning: Permanently added '10.30.171.41' (ECDSA) to the list of known hosts. + ssh 10.30.171.41 rm -f /tmp/karaf-0.21.4-SNAPSHOT/data/log/karaf.log.gz Warning: Permanently added '10.30.171.41' (ECDSA) to the list of known hosts. + scp 10.30.171.41:/tmp/karaf-0.21.4-SNAPSHOT/data/log/karaf_console.log odl3_karaf_console.log Warning: Permanently added '10.30.171.41' (ECDSA) to the list of known hosts. + ssh 10.30.171.41 rm -f /tmp/karaf-0.21.4-SNAPSHOT/data/log/karaf_console.log Warning: Permanently added '10.30.171.41' (ECDSA) to the list of known hosts. + echo 'Fetch GC logs' Fetch GC logs + mkdir -p gclogs-3 + scp '10.30.171.41:/tmp/karaf-0.21.4-SNAPSHOT/data/log/*.log' gclogs-3/ Warning: Permanently added '10.30.171.41' (ECDSA) to the list of known hosts. scp: /tmp/karaf-0.21.4-SNAPSHOT/data/log/*.log: No such file or directory + echo 'Examine copied files' Examine copied files + ls -lt total 60016 drwxrwxr-x. 2 jenkins jenkins 6 Nov 17 03:43 gclogs-3 -rw-rw-r--. 1 jenkins jenkins 145919 Nov 17 03:43 odl3_karaf_console.log -rw-rw-r--. 1 jenkins jenkins 243486 Nov 17 03:43 odl3_karaf.log.gz drwxrwxr-x. 2 jenkins jenkins 6 Nov 17 03:43 gclogs-2 -rw-rw-r--. 1 jenkins jenkins 1720 Nov 17 03:43 odl2_karaf_console.log -rw-rw-r--. 1 jenkins jenkins 874787 Nov 17 03:43 odl2_karaf.log.gz drwxrwxr-x. 2 jenkins jenkins 6 Nov 17 03:43 gclogs-1 -rw-rw-r--. 1 jenkins jenkins 1720 Nov 17 03:43 odl1_karaf_console.log -rw-rw-r--. 1 jenkins jenkins 655549 Nov 17 03:43 odl1_karaf.log.gz -rw-rw-r--. 1 jenkins jenkins 143176 Nov 17 03:43 karaf_3_2095_threads_after.log -rw-rw-r--. 1 jenkins jenkins 13850 Nov 17 03:43 ps_after.log -rw-rw-r--. 1 jenkins jenkins 139449 Nov 17 03:43 karaf_2_16821_threads_after.log -rw-rw-r--. 1 jenkins jenkins 128630 Nov 17 03:43 karaf_1_29077_threads_after.log -rw-rw-r--. 1 jenkins jenkins 301238 Nov 17 03:43 report.html -rw-rw-r--. 1 jenkins jenkins 3618523 Nov 17 03:43 log.html -rw-rw-r--. 1 jenkins jenkins 54728425 Nov 17 03:43 output.xml -rw-rw-r--. 1 jenkins jenkins 4120 Nov 16 23:41 testplan.txt -rw-rw-r--. 1 jenkins jenkins 115599 Nov 16 23:41 karaf_3_2095_threads_before.log -rw-rw-r--. 1 jenkins jenkins 14096 Nov 16 23:41 ps_before.log -rw-rw-r--. 1 jenkins jenkins 114451 Nov 16 23:41 karaf_2_2093_threads_before.log -rw-rw-r--. 1 jenkins jenkins 114410 Nov 16 23:41 karaf_1_2091_threads_before.log -rw-rw-r--. 1 jenkins jenkins 3106 Nov 16 23:37 post-startup-script.sh -rw-rw-r--. 1 jenkins jenkins 252 Nov 16 23:37 startup-script.sh -rw-rw-r--. 1 jenkins jenkins 3471 Nov 16 23:37 configuration-script.sh -rw-rw-r--. 1 jenkins jenkins 695 Nov 16 23:37 custom_shard_config.txt -rw-rw-r--. 1 jenkins jenkins 135 Nov 16 23:37 scriptplan.txt -rw-rw-r--. 1 jenkins jenkins 337 Nov 16 23:37 detect_variables.env -rw-rw-r--. 1 jenkins jenkins 2619 Nov 16 23:37 pom.xml -rw-rw-r--. 1 jenkins jenkins 92 Nov 16 23:37 set_variables.env -rw-rw-r--. 1 jenkins jenkins 313 Nov 16 23:37 slave_addresses.txt -rw-rw-r--. 1 jenkins jenkins 570 Nov 16 23:36 requirements.txt -rw-rw-r--. 1 jenkins jenkins 26 Nov 16 23:36 env.properties -rw-rw-r--. 1 jenkins jenkins 333 Nov 16 23:34 stack-parameters.yaml drwxrwxr-x. 7 jenkins jenkins 4096 Nov 16 23:33 test drwxrwxr-x. 2 jenkins jenkins 6 Nov 16 23:33 test@tmp -rw-rw-r--. 1 jenkins jenkins 1410 Nov 16 21:30 maven-metadata.xml + true [controller-csit-3node-clustering-ask-all-scandium] $ /bin/sh /tmp/jenkins17378831309683019674.sh Cleaning up Robot installation... $ ssh-agent -k unset SSH_AUTH_SOCK; unset SSH_AGENT_PID; echo Agent pid 5280 killed; [ssh-agent] Stopped. Recording plot data Robot results publisher started... INFO: Checking test criticality is deprecated and will be dropped in a future release! -Parsing output xml: Done! -Copying log files to build dir: Done! -Assigning results to build: Done! -Checking thresholds: Done! Done publishing Robot results. Build step 'Publish Robot Framework test results' changed build result to UNSTABLE [PostBuildScript] - [INFO] Executing post build scripts. [controller-csit-3node-clustering-ask-all-scandium] $ /bin/bash /tmp/jenkins15265427462349560824.sh Archiving csit artifacts mv: cannot stat '*_1.png': No such file or directory mv: cannot stat '/tmp/odl1_*': No such file or directory mv: cannot stat '*_2.png': No such file or directory mv: cannot stat '/tmp/odl2_*': No such file or directory mv: cannot stat '*_3.png': No such file or directory mv: cannot stat '/tmp/odl3_*': No such file or directory % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 776k 0 776k 0 0 4911k 0 --:--:-- --:--:-- --:--:-- 4880k 100 3401k 0 3401k 0 0 1901k 0 --:--:-- 0:00:01 --:--:-- 1900k 100 4004k 0 4004k 0 0 1024k 0 --:--:-- 0:00:03 --:--:-- 1023k 100 4012k 0 4012k 0 0 956k 0 --:--:-- 0:00:04 --:--:-- 956k 100 5030k 0 5030k 0 0 1170k 0 --:--:-- 0:00:04 --:--:-- 1169k Archive: robot-plugin.zip inflating: ./archives/robot-plugin/log.html inflating: ./archives/robot-plugin/output.xml inflating: ./archives/robot-plugin/report.html mv: cannot stat '*.log.gz': No such file or directory mv: cannot stat '*.csv': No such file or directory mv: cannot stat '*.png': No such file or directory [PostBuildScript] - [INFO] Executing post build scripts. [controller-csit-3node-clustering-ask-all-scandium] $ /bin/bash /tmp/jenkins15469196985477579502.sh [PostBuildScript] - [INFO] Executing post build scripts. [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties content OS_CLOUD=vex OS_STACK_NAME=releng-controller-csit-3node-clustering-ask-all-scandium-81 [EnvInject] - Variables injected successfully. provisioning config files... copy managed file [clouds-yaml] to file:/home/jenkins/.config/openstack/clouds.yaml [controller-csit-3node-clustering-ask-all-scandium] $ /bin/bash /tmp/jenkins843365897792676426.sh ---> openstack-stack-delete.sh Setup pyenv: system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-scandium/.python-version) lf-activate-venv(): INFO: Reuse venv:/tmp/venv-CM3L from file:/tmp/.os_lf_venv lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv) lf-activate-venv(): INFO: Attempting to install with network-safe options... ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. lftools 0.37.15 requires urllib3<2.1.0, but you have urllib3 2.5.0 which is incompatible. lf-activate-venv(): INFO: Base packages installed successfully lf-activate-venv(): INFO: Installing additional packages: lftools[openstack] kubernetes python-heatclient python-openstackclient urllib3~=1.26.15 lf-activate-venv(): INFO: Adding /tmp/venv-CM3L/bin to PATH INFO: Stack cost retrieval disabled, setting cost to 0 INFO: Deleting stack releng-controller-csit-3node-clustering-ask-all-scandium-81 Successfully deleted stack releng-controller-csit-3node-clustering-ask-all-scandium-81 [PostBuildScript] - [INFO] Executing post build scripts. [controller-csit-3node-clustering-ask-all-scandium] $ /bin/bash /tmp/jenkins476915096931660022.sh ---> sysstat.sh [controller-csit-3node-clustering-ask-all-scandium] $ /bin/bash /tmp/jenkins2139303015878956027.sh ---> package-listing.sh ++ facter osfamily ++ tr '[:upper:]' '[:lower:]' + OS_FAMILY=redhat + workspace=/w/workspace/controller-csit-3node-clustering-ask-all-scandium + START_PACKAGES=/tmp/packages_start.txt + END_PACKAGES=/tmp/packages_end.txt + DIFF_PACKAGES=/tmp/packages_diff.txt + PACKAGES=/tmp/packages_start.txt + '[' /w/workspace/controller-csit-3node-clustering-ask-all-scandium ']' + PACKAGES=/tmp/packages_end.txt + case "${OS_FAMILY}" in + rpm -qa + sort + '[' -f /tmp/packages_start.txt ']' + '[' -f /tmp/packages_end.txt ']' + diff /tmp/packages_start.txt /tmp/packages_end.txt + '[' /w/workspace/controller-csit-3node-clustering-ask-all-scandium ']' + mkdir -p /w/workspace/controller-csit-3node-clustering-ask-all-scandium/archives/ + cp -f /tmp/packages_diff.txt /tmp/packages_end.txt /tmp/packages_start.txt /w/workspace/controller-csit-3node-clustering-ask-all-scandium/archives/ [controller-csit-3node-clustering-ask-all-scandium] $ /bin/bash /tmp/jenkins8993656654455422541.sh ---> capture-instance-metadata.sh Setup pyenv: system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-scandium/.python-version) lf-activate-venv(): INFO: Reuse venv:/tmp/venv-CM3L from file:/tmp/.os_lf_venv lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv) lf-activate-venv(): INFO: Attempting to install with network-safe options... lf-activate-venv(): INFO: Base packages installed successfully lf-activate-venv(): INFO: Installing additional packages: lftools lf-activate-venv(): INFO: Adding /tmp/venv-CM3L/bin to PATH INFO: Running in OpenStack, capturing instance metadata [controller-csit-3node-clustering-ask-all-scandium] $ /bin/bash /tmp/jenkins7711162728645529682.sh provisioning config files... Could not find credentials [logs] for controller-csit-3node-clustering-ask-all-scandium #81 copy managed file [jenkins-log-archives-settings] to file:/w/workspace/controller-csit-3node-clustering-ask-all-scandium@tmp/config5741397890974951877tmp Regular expression run condition: Expression=[^.*logs-s3.*], Label=[odl-logs-s3-cloudfront-index] Run condition [Regular expression match] enabling perform for step [Provide Configuration files] provisioning config files... copy managed file [jenkins-s3-log-ship] to file:/home/jenkins/.aws/credentials [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties content SERVER_ID=logs [EnvInject] - Variables injected successfully. [controller-csit-3node-clustering-ask-all-scandium] $ /bin/bash /tmp/jenkins441616489011543558.sh ---> create-netrc.sh WARN: Log server credential not found. [controller-csit-3node-clustering-ask-all-scandium] $ /bin/bash /tmp/jenkins6636541358389145539.sh ---> python-tools-install.sh Setup pyenv: system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-scandium/.python-version) lf-activate-venv(): INFO: Reuse venv:/tmp/venv-CM3L from file:/tmp/.os_lf_venv lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv) lf-activate-venv(): INFO: Attempting to install with network-safe options... lf-activate-venv(): INFO: Base packages installed successfully lf-activate-venv(): INFO: Installing additional packages: lftools lf-activate-venv(): INFO: Adding /tmp/venv-CM3L/bin to PATH [controller-csit-3node-clustering-ask-all-scandium] $ /bin/bash /tmp/jenkins1998752605241940623.sh ---> sudo-logs.sh Archiving 'sudo' log.. [controller-csit-3node-clustering-ask-all-scandium] $ /bin/bash /tmp/jenkins5497515555424404186.sh ---> job-cost.sh Setup pyenv: system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-scandium/.python-version) lf-activate-venv(): INFO: Reuse venv:/tmp/venv-CM3L from file:/tmp/.os_lf_venv lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv) lf-activate-venv(): INFO: Attempting to install with network-safe options... lf-activate-venv(): INFO: Base packages installed successfully lf-activate-venv(): INFO: Installing additional packages: zipp==1.1.0 python-openstackclient urllib3~=1.26.15 lf-activate-venv(): INFO: Adding /tmp/venv-CM3L/bin to PATH DEBUG: total: 0 INFO: Retrieving Stack Cost... INFO: Retrieving Pricing Info for: v3-standard-2 INFO: Archiving Costs [controller-csit-3node-clustering-ask-all-scandium] $ /bin/bash -l /tmp/jenkins18257786811472891590.sh ---> logs-deploy.sh Setup pyenv: system 3.8.13 3.9.13 3.10.13 * 3.11.7 (set by /w/workspace/controller-csit-3node-clustering-ask-all-scandium/.python-version) lf-activate-venv(): INFO: Reuse venv:/tmp/venv-CM3L from file:/tmp/.os_lf_venv lf-activate-venv(): INFO: Installing base packages (pip, setuptools, virtualenv) lf-activate-venv(): INFO: Attempting to install with network-safe options... lf-activate-venv(): INFO: Base packages installed successfully lf-activate-venv(): INFO: Installing additional packages: lftools urllib3~=1.26.15 lf-activate-venv(): INFO: Adding /tmp/venv-CM3L/bin to PATH WARNING: Nexus logging server not set INFO: S3 path logs/releng/vex-yul-odl-jenkins-1/controller-csit-3node-clustering-ask-all-scandium/81/ INFO: archiving logs to S3 ---> uname -a: Linux prd-centos8-robot-2c-8g-11129.novalocal 4.18.0-553.5.1.el8.x86_64 #1 SMP Tue May 21 05:46:01 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux ---> lscpu: Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 2 On-line CPU(s) list: 0,1 Thread(s) per core: 1 Core(s) per socket: 1 Socket(s): 2 NUMA node(s): 1 Vendor ID: AuthenticAMD CPU family: 23 Model: 49 Model name: AMD EPYC-Rome Processor Stepping: 0 CPU MHz: 2800.000 BogoMIPS: 5600.00 Virtualization: AMD-V Hypervisor vendor: KVM Virtualization type: full L1d cache: 32K L1i cache: 32K L2 cache: 512K L3 cache: 16384K NUMA node0 CPU(s): 0,1 Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqdq ssse3 fma cx16 sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm cmp_legacy svm cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw topoext perfctr_core ssbd ibrs ibpb stibp vmmcall fsgsbase tsc_adjust bmi1 avx2 smep bmi2 rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves clzero xsaveerptr wbnoinvd arat npt nrip_save umip rdpid arch_capabilities ---> nproc: 2 ---> df -h: Filesystem Size Used Avail Use% Mounted on devtmpfs 3.8G 0 3.8G 0% /dev tmpfs 3.8G 0 3.8G 0% /dev/shm tmpfs 3.8G 17M 3.8G 1% /run tmpfs 3.8G 0 3.8G 0% /sys/fs/cgroup /dev/vda1 40G 8.4G 32G 21% / tmpfs 770M 0 770M 0% /run/user/1001 ---> free -m: total used free shared buff/cache available Mem: 7697 631 4749 19 2316 6767 Swap: 1023 0 1023 ---> ip addr: 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: mtu 1458 qdisc mq state UP group default qlen 1000 link/ether fa:16:3e:09:83:95 brd ff:ff:ff:ff:ff:ff altname enp0s3 altname ens3 inet 10.30.170.100/23 brd 10.30.171.255 scope global dynamic noprefixroute eth0 valid_lft 71282sec preferred_lft 71282sec inet6 fe80::f816:3eff:fe09:8395/64 scope link valid_lft forever preferred_lft forever ---> sar -b -r -n DEV: Linux 4.18.0-553.5.1.el8.x86_64 (prd-centos8-robot-2c-8g-11129.novalocal) 11/17/2025 _x86_64_ (2 CPU) 12:00:01 AM tps rtps wtps bread/s bwrtn/s 12:01:01 AM 1.00 0.02 0.98 0.13 196.44 12:02:01 AM 0.45 0.07 0.38 1.73 8.13 12:03:01 AM 0.23 0.00 0.23 0.00 6.05 12:04:01 AM 0.38 0.00 0.38 0.00 11.98 12:05:01 AM 0.23 0.00 0.23 0.00 10.16 12:06:01 AM 0.37 0.00 0.37 0.00 4.63 12:07:01 AM 0.25 0.00 0.25 0.00 3.97 12:08:01 AM 0.45 0.07 0.38 7.60 12.18 12:09:01 AM 0.42 0.00 0.42 0.00 6.90 12:10:01 AM 0.17 0.00 0.17 0.00 6.03 12:11:01 AM 0.22 0.00 0.22 0.00 8.78 12:12:01 AM 0.45 0.00 0.45 0.00 20.61 12:13:01 AM 0.53 0.00 0.53 0.00 18.28 12:14:01 AM 0.53 0.00 0.53 0.00 18.33 12:15:01 AM 0.27 0.00 0.27 0.00 16.25 12:16:02 AM 0.22 0.00 0.22 0.00 16.01 12:17:01 AM 0.24 0.00 0.24 0.00 21.96 12:18:01 AM 0.13 0.00 0.13 0.00 8.40 12:19:01 AM 0.30 0.00 0.30 0.00 5.28 12:20:01 AM 0.17 0.00 0.17 0.00 4.02 12:21:01 AM 0.30 0.00 0.30 0.00 3.68 12:22:01 AM 0.23 0.00 0.23 0.00 2.95 12:23:01 AM 0.17 0.00 0.17 0.00 4.28 12:24:01 AM 0.32 0.00 0.32 0.00 4.90 12:25:01 AM 0.17 0.00 0.17 0.00 9.21 12:26:01 AM 0.15 0.00 0.15 0.00 5.87 12:27:01 AM 0.17 0.00 0.17 0.00 7.36 12:28:01 AM 0.22 0.00 0.22 0.00 2.03 12:29:01 AM 0.30 0.00 0.30 0.00 7.31 12:30:01 AM 0.18 0.00 0.18 0.00 2.80 12:31:01 AM 0.23 0.00 0.23 0.00 3.08 12:32:01 AM 0.22 0.00 0.22 0.00 5.10 12:33:01 AM 0.37 0.00 0.37 0.00 9.60 12:34:01 AM 1.62 0.32 1.30 6.93 26.86 12:35:01 AM 0.55 0.00 0.55 0.00 8.43 12:36:01 AM 0.35 0.00 0.35 0.00 8.53 12:37:01 AM 0.32 0.00 0.32 0.00 3.05 12:38:01 AM 0.33 0.00 0.33 0.00 12.25 12:39:01 AM 0.30 0.00 0.30 0.00 6.25 12:40:01 AM 0.18 0.02 0.17 0.13 18.68 12:41:01 AM 8.71 5.73 2.98 555.41 83.81 12:42:01 AM 0.30 0.00 0.30 0.00 13.88 12:43:01 AM 0.23 0.00 0.23 0.00 18.11 12:44:01 AM 0.42 0.00 0.42 0.00 13.76 12:45:01 AM 0.22 0.00 0.22 0.00 17.46 12:46:01 AM 0.18 0.00 0.18 0.00 24.71 12:47:01 AM 0.18 0.00 0.18 0.00 2.67 12:48:01 AM 0.13 0.00 0.13 0.00 1.68 12:49:01 AM 0.23 0.00 0.23 0.00 5.23 12:50:01 AM 0.25 0.00 0.25 0.00 4.65 12:51:01 AM 0.25 0.00 0.25 0.00 5.35 12:52:01 AM 0.20 0.00 0.20 0.00 3.18 12:53:01 AM 0.15 0.00 0.15 0.00 7.70 12:54:01 AM 0.32 0.00 0.32 0.00 7.37 12:55:01 AM 0.28 0.00 0.28 0.00 9.95 12:56:01 AM 0.15 0.00 0.15 0.00 1.22 12:57:01 AM 0.18 0.00 0.18 0.00 3.15 12:58:01 AM 0.30 0.00 0.30 0.00 8.41 12:59:01 AM 0.37 0.00 0.37 0.00 6.08 01:00:01 AM 0.15 0.00 0.15 0.00 6.41 01:01:01 AM 0.23 0.00 0.23 0.00 6.85 01:02:01 AM 0.28 0.00 0.28 0.00 9.86 01:03:01 AM 0.25 0.00 0.25 0.00 7.75 01:04:01 AM 0.33 0.00 0.33 0.00 5.78 01:05:01 AM 0.12 0.00 0.12 0.00 3.45 01:06:01 AM 0.17 0.00 0.17 0.00 5.90 01:07:01 AM 0.12 0.00 0.12 0.00 1.35 01:08:01 AM 0.23 0.00 0.23 0.00 6.72 01:09:01 AM 0.35 0.00 0.35 0.00 17.86 01:10:01 AM 0.13 0.00 0.13 0.00 16.53 01:11:01 AM 0.28 0.00 0.28 0.00 18.14 01:12:01 AM 0.25 0.00 0.25 0.00 13.23 01:13:01 AM 0.20 0.00 0.20 0.00 16.71 01:14:01 AM 0.43 0.00 0.43 0.00 14.85 01:15:01 AM 0.17 0.00 0.17 0.00 25.65 01:16:01 AM 0.25 0.00 0.25 0.00 4.80 01:17:01 AM 0.17 0.00 0.17 0.00 2.38 01:18:02 AM 0.17 0.00 0.17 0.00 4.15 01:19:01 AM 0.34 0.00 0.34 0.00 6.37 01:20:01 AM 0.17 0.00 0.17 0.00 3.90 01:21:01 AM 0.20 0.00 0.20 0.00 2.05 01:22:01 AM 0.18 0.00 0.18 0.00 8.40 01:23:01 AM 0.23 0.00 0.23 0.00 4.52 01:24:01 AM 0.42 0.00 0.42 0.00 16.11 01:25:01 AM 0.12 0.00 0.12 0.00 0.82 01:26:01 AM 0.15 0.00 0.15 0.00 1.65 01:27:01 AM 0.22 0.00 0.22 0.00 5.65 01:28:01 AM 0.13 0.00 0.13 0.00 1.32 01:29:01 AM 0.27 0.00 0.27 0.00 4.90 01:30:01 AM 0.25 0.00 0.25 0.00 8.55 01:31:01 AM 0.50 0.00 0.50 0.00 6.61 01:32:01 AM 0.43 0.00 0.43 0.00 7.22 01:33:01 AM 0.28 0.00 0.28 0.00 8.68 01:34:01 AM 0.33 0.00 0.33 0.00 5.13 01:35:01 AM 0.22 0.00 0.22 0.00 2.97 01:36:01 AM 0.18 0.00 0.18 0.00 12.15 01:37:01 AM 0.47 0.00 0.47 0.00 15.98 01:38:01 AM 0.52 0.00 0.52 0.00 17.90 01:39:01 AM 0.65 0.00 0.65 0.00 20.24 01:40:01 AM 0.63 0.00 0.63 0.00 19.06 01:41:01 AM 0.55 0.00 0.55 0.00 18.66 01:42:01 AM 0.50 0.00 0.50 0.00 17.71 01:43:01 AM 0.23 0.00 0.23 0.00 23.88 01:44:01 AM 0.32 0.00 0.32 0.00 7.57 01:45:01 AM 0.13 0.00 0.13 0.00 1.30 01:46:01 AM 0.92 0.00 0.92 0.00 16.78 01:47:01 AM 0.22 0.00 0.22 0.00 3.80 01:48:01 AM 0.15 0.00 0.15 0.00 3.63 01:49:01 AM 0.32 0.00 0.32 0.00 5.85 01:50:01 AM 0.12 0.00 0.12 0.00 0.78 01:51:01 AM 0.40 0.00 0.40 0.00 13.18 01:52:01 AM 0.15 0.00 0.15 0.00 6.82 01:53:01 AM 0.12 0.00 0.12 0.00 1.20 01:54:01 AM 0.37 0.00 0.37 0.00 7.05 01:55:01 AM 0.17 0.00 0.17 0.00 3.10 01:56:01 AM 0.17 0.00 0.17 0.00 4.68 01:57:01 AM 0.12 0.00 0.12 0.00 2.83 01:58:01 AM 0.17 0.00 0.17 0.00 5.85 01:59:01 AM 0.20 0.00 0.20 0.00 3.10 02:00:01 AM 0.45 0.00 0.45 0.00 13.90 02:01:01 AM 0.30 0.00 0.30 0.00 19.26 02:02:01 AM 0.17 0.00 0.17 0.00 9.76 02:03:01 AM 0.27 0.00 0.27 0.00 20.08 02:04:01 AM 0.33 0.00 0.33 0.00 12.01 02:05:01 AM 0.37 0.00 0.37 0.00 22.73 02:06:01 AM 0.30 0.00 0.30 0.00 16.28 02:07:01 AM 0.17 0.00 0.17 0.00 18.03 02:08:01 AM 0.17 0.00 0.17 0.00 4.43 02:09:01 AM 0.28 0.00 0.28 0.00 5.26 02:10:01 AM 0.13 0.00 0.13 0.00 3.73 02:11:01 AM 0.30 0.00 0.30 0.00 3.63 02:12:01 AM 0.20 0.00 0.20 0.00 5.36 02:13:01 AM 0.18 0.00 0.18 0.00 2.15 02:14:01 AM 0.33 0.00 0.33 0.00 11.13 02:15:01 AM 0.15 0.00 0.15 0.00 7.73 02:16:01 AM 0.12 0.00 0.12 0.00 1.37 02:17:01 AM 0.13 0.00 0.13 0.00 0.95 02:18:01 AM 0.15 0.00 0.15 0.00 2.97 02:19:01 AM 0.37 0.00 0.37 0.00 10.51 02:20:01 AM 0.13 0.00 0.13 0.00 3.07 02:21:01 AM 0.28 0.00 0.28 0.00 8.95 02:22:01 AM 0.40 0.00 0.40 0.00 5.58 02:23:01 AM 0.37 0.00 0.37 0.00 7.90 02:24:01 AM 0.70 0.00 0.70 0.00 19.30 02:25:01 AM 0.52 0.00 0.52 0.00 18.11 02:26:01 AM 0.50 0.00 0.50 0.00 18.01 02:27:01 AM 0.52 0.00 0.52 0.00 16.53 02:28:02 AM 0.45 0.00 0.45 0.00 14.00 02:29:01 AM 0.36 0.00 0.36 0.00 20.17 02:30:01 AM 0.10 0.00 0.10 0.00 21.86 02:31:01 AM 0.20 0.00 0.20 0.00 12.45 02:32:01 AM 0.22 0.00 0.22 0.00 8.68 02:33:01 AM 0.27 0.00 0.27 0.00 13.63 02:34:01 AM 0.43 0.00 0.43 0.00 15.05 02:35:01 AM 0.33 0.00 0.33 0.00 12.26 02:36:01 AM 0.13 0.00 0.13 0.00 7.50 02:37:01 AM 0.18 0.00 0.18 0.00 12.28 02:38:01 AM 0.13 0.00 0.13 0.00 5.90 02:39:01 AM 0.32 0.00 0.32 0.00 5.87 02:40:01 AM 0.20 0.00 0.20 0.00 3.13 02:41:01 AM 0.20 0.00 0.20 0.00 9.91 02:42:01 AM 0.27 0.00 0.27 0.00 18.78 02:43:01 AM 0.23 0.00 0.23 0.00 18.26 02:44:01 AM 0.33 0.00 0.33 0.00 20.36 02:45:01 AM 0.15 0.00 0.15 0.00 11.10 02:46:01 AM 0.15 0.00 0.15 0.00 16.26 02:47:01 AM 0.23 0.00 0.23 0.00 27.98 02:48:01 AM 0.17 0.00 0.17 0.00 8.18 02:49:01 AM 0.38 0.00 0.38 0.00 14.91 02:50:01 AM 0.17 0.00 0.17 0.00 7.91 02:51:01 AM 0.33 0.00 0.33 0.00 16.33 02:52:01 AM 0.27 0.00 0.27 0.00 12.81 02:53:01 AM 0.27 0.00 0.27 0.00 16.18 02:54:01 AM 0.45 0.00 0.45 0.00 10.36 02:55:01 AM 0.33 0.00 0.33 0.00 8.55 02:56:01 AM 0.27 0.00 0.27 0.00 15.65 02:57:01 AM 0.27 0.00 0.27 0.00 14.38 02:58:01 AM 0.12 0.00 0.12 0.00 8.53 02:59:01 AM 0.42 0.00 0.42 0.00 22.24 03:00:01 AM 0.27 0.00 0.27 0.00 19.76 03:01:01 AM 0.55 0.00 0.55 0.00 18.79 03:02:01 AM 0.20 0.00 0.20 0.00 21.84 03:03:01 AM 0.37 0.00 0.37 0.00 9.58 03:04:01 AM 0.37 0.00 0.37 0.00 16.76 03:05:01 AM 0.18 0.00 0.18 0.00 14.11 03:06:01 AM 0.23 0.00 0.23 0.00 9.37 03:07:01 AM 0.13 0.00 0.13 0.00 13.73 03:08:01 AM 0.15 0.00 0.15 0.00 10.28 03:09:01 AM 0.28 0.00 0.28 0.00 9.43 03:10:01 AM 0.17 0.00 0.17 0.00 12.16 03:11:01 AM 0.22 0.00 0.22 0.00 16.84 03:12:01 AM 0.32 0.00 0.32 0.00 17.80 03:13:01 AM 0.13 0.00 0.13 0.00 8.92 03:14:01 AM 0.30 0.00 0.30 0.00 19.79 03:15:01 AM 0.28 0.00 0.28 0.00 11.88 03:16:01 AM 0.15 0.00 0.15 0.00 28.24 03:17:01 AM 0.22 0.00 0.22 0.00 9.73 03:18:01 AM 0.12 0.00 0.12 0.00 7.33 03:19:01 AM 0.23 0.00 0.23 0.00 15.50 03:20:01 AM 0.35 0.00 0.35 0.00 11.71 03:21:01 AM 0.23 0.00 0.23 0.00 14.38 03:22:01 AM 0.22 0.00 0.22 0.00 9.77 03:23:01 AM 0.30 0.00 0.30 0.00 12.83 03:24:01 AM 0.98 0.00 0.98 0.00 21.21 03:25:01 AM 0.40 0.00 0.40 0.00 22.38 03:26:01 AM 0.20 0.00 0.20 0.00 13.23 03:27:01 AM 0.17 0.00 0.17 0.00 12.80 03:28:01 AM 0.13 0.00 0.13 0.00 15.19 03:29:01 AM 0.32 0.00 0.32 0.00 13.31 03:30:01 AM 0.15 0.00 0.15 0.00 16.39 03:31:01 AM 0.25 0.00 0.25 0.00 19.63 03:32:01 AM 0.27 0.00 0.27 0.00 16.03 03:33:01 AM 0.13 0.00 0.13 0.00 7.50 03:34:02 AM 0.40 0.00 0.40 0.00 19.05 03:35:01 AM 0.15 0.00 0.15 0.00 7.64 03:36:01 AM 0.18 0.00 0.18 0.00 14.01 03:37:01 AM 0.13 0.00 0.13 0.00 11.06 03:38:01 AM 0.15 0.00 0.15 0.00 11.60 03:39:01 AM 0.37 0.00 0.37 0.00 16.36 03:40:01 AM 0.27 0.00 0.27 0.00 21.31 03:41:01 AM 0.20 0.00 0.20 0.00 13.40 03:42:01 AM 0.20 0.00 0.20 0.00 12.95 03:43:01 AM 0.12 0.00 0.12 0.00 10.26 03:44:01 AM 13.70 0.28 13.41 33.86 608.40 03:45:01 AM 33.44 0.42 33.02 48.25 3445.05 Average: 0.52 0.03 0.49 2.91 29.88 12:00:01 AM kbmemfree kbavail kbmemused %memused kbbuffers kbcached kbcommit %commit kbactive kbinact kbdirty 12:01:01 AM 4879660 6992260 3002772 38.09 2688 2278988 735208 8.23 266856 2348692 4 12:02:01 AM 4879612 6992428 3002820 38.10 2688 2279204 735208 8.23 267044 2348688 92 12:03:01 AM 4879796 6992716 3002636 38.09 2688 2279308 735208 8.23 267044 2348520 96 12:04:01 AM 4879468 6992584 3002964 38.10 2688 2279504 735208 8.23 267044 2348872 108 12:05:01 AM 4879564 6992832 3002868 38.10 2688 2279656 735208 8.23 267044 2348888 4 12:06:01 AM 4879376 6992644 3003056 38.10 2688 2279656 735208 8.23 267044 2349100 4 12:07:01 AM 4878904 6992220 3003528 38.10 2688 2279704 751056 8.41 267044 2349296 20 12:08:01 AM 4879052 6992792 3003380 38.10 2688 2280128 737296 8.26 267092 2349324 8 12:09:01 AM 4878960 6992700 3003472 38.10 2688 2280128 737296 8.26 267092 2349516 4 12:10:01 AM 4878568 6992452 3003864 38.11 2688 2280272 737296 8.26 267092 2349600 4 12:11:01 AM 4878080 6992320 3004352 38.11 2688 2280628 754224 8.45 267092 2350244 144 12:12:01 AM 4877800 6992416 3004632 38.12 2688 2281012 754224 8.45 267092 2350580 40 12:13:01 AM 4877412 6992420 3005020 38.12 2688 2281396 754224 8.45 267092 2351036 24 12:14:01 AM 4876616 6991996 3005816 38.13 2688 2281776 754224 8.45 267092 2351868 92 12:15:01 AM 4876052 6991816 3006380 38.14 2688 2282160 754224 8.45 267092 2352176 76 12:16:02 AM 4875884 6992032 3006548 38.14 2688 2282544 754224 8.45 267092 2352564 24 12:17:01 AM 4875240 6992064 3007192 38.15 2688 2283212 754224 8.45 267092 2353128 120 12:18:01 AM 4875360 6992296 3007072 38.15 2688 2283324 754224 8.45 267092 2353244 4 12:19:01 AM 4875032 6991972 3007400 38.15 2688 2283328 754224 8.45 267092 2353316 4 12:20:01 AM 4874704 6991724 3007728 38.16 2688 2283408 754224 8.45 267092 2353696 4 12:21:01 AM 4875096 6992120 3007336 38.15 2688 2283412 754224 8.45 267092 2353332 4 12:22:01 AM 4875040 6992144 3007392 38.15 2688 2283492 754224 8.45 267092 2353420 88 12:23:01 AM 4875068 6992172 3007364 38.15 2688 2283492 754224 8.45 267092 2353420 4 12:24:01 AM 4874764 6992012 3007668 38.16 2688 2283636 733400 8.21 267092 2353640 148 12:25:01 AM 4874576 6991912 3007856 38.16 2688 2283724 733400 8.21 267092 2353864 4 12:26:01 AM 4874496 6991992 3007936 38.16 2688 2283884 733400 8.21 267092 2353956 24 12:27:01 AM 4874256 6991888 3008176 38.16 2688 2284020 733400 8.21 267092 2354176 4 12:28:01 AM 4874360 6991992 3008072 38.16 2688 2284020 733400 8.21 267092 2353972 4 12:29:01 AM 4874336 6992064 3008096 38.16 2688 2284116 733400 8.21 267092 2354268 40 12:30:01 AM 4874360 6992088 3008072 38.16 2688 2284116 726844 8.14 267092 2354180 4 12:31:01 AM 4873912 6991692 3008520 38.17 2688 2284168 726844 8.14 267092 2354316 64 12:32:01 AM 4873900 6991832 3008532 38.17 2688 2284320 726844 8.14 267092 2354552 152 12:33:01 AM 4873672 6991608 3008760 38.17 2688 2284324 726844 8.14 267092 2354632 4 12:34:01 AM 4873384 6991564 3009048 38.17 2688 2284568 726844 8.14 267236 2354732 4 12:35:01 AM 4873696 6991940 3008736 38.17 2688 2284632 726844 8.14 267236 2354476 4 12:36:01 AM 4873824 6992204 3008608 38.17 2688 2284768 726844 8.14 267236 2354628 4 12:37:01 AM 4873732 6992112 3008700 38.17 2688 2284768 726844 8.14 267240 2354624 4 12:38:01 AM 4873452 6992096 3008980 38.17 2688 2285032 726844 8.14 267240 2354872 4 12:39:01 AM 4873048 6991904 3009384 38.18 2688 2285252 750200 8.40 267240 2355184 184 12:40:01 AM 4871900 6991140 3010532 38.19 2688 2285636 738944 8.27 267240 2355936 56 12:41:01 AM 4855540 6991876 3026892 38.40 2688 2302680 735832 8.24 269384 2370452 24 12:42:01 AM 4854772 6991488 3027660 38.41 2688 2303060 735832 8.24 269384 2371072 140 12:43:01 AM 4854644 6991756 3027788 38.41 2688 2303456 735832 8.24 269384 2371380 76 12:44:01 AM 4854288 6991792 3028144 38.42 2688 2303848 735832 8.24 269384 2371792 232 12:45:01 AM 4853084 6991224 3029348 38.43 2688 2304476 735832 8.24 269384 2372700 392 12:46:01 AM 4853392 6991836 3029040 38.43 2688 2304780 735832 8.24 269384 2372740 4 12:47:01 AM 4853024 6991472 3029408 38.43 2688 2304784 735832 8.24 269388 2372836 4 12:48:01 AM 4853176 6991700 3029256 38.43 2688 2304860 735832 8.24 269384 2372820 88 12:49:01 AM 4853008 6991532 3029424 38.43 2688 2304860 735832 8.24 269384 2372988 4 12:50:01 AM 4852820 6991356 3029612 38.43 2688 2304872 752808 8.43 269384 2372928 16 12:51:01 AM 4852968 6991580 3029464 38.43 2688 2304944 752808 8.43 269384 2372904 4 12:52:01 AM 4852976 6991592 3029456 38.43 2688 2304948 752808 8.43 269384 2372908 4 12:53:01 AM 4852792 6991592 3029640 38.44 2688 2305132 735324 8.23 269384 2373092 16 12:54:01 AM 4852068 6991076 3030364 38.44 2688 2305340 735324 8.23 269420 2373640 168 12:55:01 AM 4852192 6991204 3030240 38.44 2688 2305344 735324 8.23 269420 2373648 4 12:56:01 AM 4851464 6990516 3030968 38.45 2688 2305384 735324 8.23 269420 2374348 52 12:57:01 AM 4851260 6990460 3031172 38.45 2688 2305532 735324 8.23 269428 2374488 140 12:58:01 AM 4851364 6990568 3031068 38.45 2688 2305536 735324 8.23 269428 2374492 8 12:59:01 AM 4851172 6990424 3031260 38.46 2688 2305584 735324 8.23 269428 2374608 56 01:00:01 AM 4850892 6990256 3031540 38.46 2688 2305696 751168 8.41 269428 2374772 4 01:01:01 AM 4850052 6989616 3032380 38.47 2688 2305892 751168 8.41 269428 2375316 96 01:02:01 AM 4850784 6990460 3031648 38.46 2688 2306004 734588 8.23 269460 2374932 24 01:03:01 AM 4850580 6990404 3031852 38.46 2688 2306152 734588 8.23 269460 2375076 4 01:04:01 AM 4850496 6990320 3031936 38.46 2688 2306152 734588 8.23 269460 2375296 16 01:05:01 AM 4849948 6989968 3032484 38.47 2688 2306348 734588 8.23 269468 2375688 148 01:06:01 AM 4850284 6990304 3032148 38.47 2688 2306348 734588 8.23 269468 2375348 4 01:07:01 AM 4850036 6990164 3032396 38.47 2688 2306456 734588 8.23 269468 2375564 112 01:08:01 AM 4850184 6990496 3032248 38.47 2688 2306640 737096 8.25 269468 2375636 168 01:09:01 AM 4849612 6990300 3032820 38.48 2688 2307024 737096 8.25 269468 2376164 196 01:10:01 AM 4849152 6990228 3033280 38.48 2688 2307408 736260 8.24 269468 2376528 116 01:11:01 AM 4849020 6990480 3033412 38.48 2688 2307792 736260 8.24 269468 2376804 40 01:12:01 AM 4848536 6990384 3033896 38.49 2688 2308176 736260 8.24 269468 2377364 132 01:13:01 AM 4848264 6990492 3034168 38.49 2688 2308556 736260 8.24 269468 2377440 52 01:14:01 AM 4848140 6990756 3034292 38.49 2688 2308936 736260 8.24 269464 2377980 204 01:15:01 AM 4847416 6990636 3035016 38.50 2688 2309540 736260 8.24 269464 2378404 72 01:16:01 AM 4847504 6990724 3034928 38.50 2688 2309540 736260 8.24 269464 2378404 4 01:17:01 AM 4847328 6990624 3035104 38.50 2688 2309616 736260 8.24 269464 2378692 84 01:18:02 AM 4847516 6990816 3034916 38.50 2688 2309620 736260 8.24 269464 2378500 4 01:19:01 AM 4847260 6990564 3035172 38.51 2688 2309624 736260 8.24 269464 2378708 4 01:20:01 AM 4847068 6990452 3035364 38.51 2688 2309704 727564 8.15 269464 2378700 4 01:21:01 AM 4846932 6990324 3035500 38.51 2688 2309712 727564 8.15 269464 2378864 16 01:22:01 AM 4847080 6990656 3035352 38.51 2688 2309896 727564 8.15 269464 2378768 20 01:23:01 AM 4846948 6990728 3035484 38.51 2688 2310100 727564 8.15 269464 2378980 160 01:24:01 AM 4846876 6990812 3035556 38.51 2688 2310256 727564 8.15 269464 2379212 4 01:25:01 AM 4846560 6990496 3035872 38.51 2688 2310256 727564 8.15 269464 2379144 4 01:26:01 AM 4846708 6990688 3035724 38.51 2688 2310308 727564 8.15 269464 2379188 56 01:27:01 AM 4846696 6990732 3035736 38.51 2688 2310364 727564 8.15 269464 2379460 4 01:28:01 AM 4846776 6990812 3035656 38.51 2688 2310364 727564 8.15 269464 2379256 4 01:29:01 AM 4846600 6990696 3035832 38.51 2688 2310424 727564 8.15 269464 2379384 72 01:30:01 AM 4846444 6990680 3035988 38.52 2688 2310564 727536 8.15 269464 2379580 4 01:31:01 AM 4846172 6990412 3036260 38.52 2688 2310568 727536 8.15 269464 2379672 4 01:32:01 AM 4846528 6990864 3035904 38.51 2688 2310664 727536 8.15 269464 2379556 16 01:33:01 AM 4846000 6990496 3036432 38.52 2688 2310824 727536 8.15 269464 2379876 4 01:34:01 AM 4846152 6990652 3036280 38.52 2688 2310828 727536 8.15 269464 2379820 12 01:35:01 AM 4845956 6990512 3036476 38.52 2688 2310884 727536 8.15 269464 2379932 64 01:36:01 AM 4846112 6990924 3036320 38.52 2688 2311140 727536 8.15 269464 2380048 8 01:37:01 AM 4845696 6990864 3036736 38.53 2688 2311504 728272 8.15 269464 2380412 24 01:38:01 AM 4845132 6990688 3037300 38.53 2688 2311884 728272 8.15 269464 2380800 4 01:39:01 AM 4844692 6990636 3037740 38.54 2688 2312280 728272 8.15 269468 2381332 24 01:40:01 AM 4844300 6990632 3038132 38.54 2688 2312664 740616 8.29 269464 2381688 36 01:41:01 AM 4844044 6990760 3038388 38.55 2688 2313048 740616 8.29 269464 2381972 24 01:42:01 AM 4843352 6990448 3039080 38.56 2688 2313428 740616 8.29 269464 2382652 24 01:43:01 AM 4842840 6990576 3039592 38.56 2688 2314064 740616 8.29 269464 2383064 16 01:44:01 AM 4842664 6990480 3039768 38.56 2688 2314136 740616 8.29 269464 2383296 4 01:45:01 AM 4842864 6990680 3039568 38.56 2688 2314136 740616 8.29 269464 2383160 4 01:46:01 AM 4842756 6990660 3039676 38.56 2688 2314224 731444 8.19 278704 2374008 4 01:47:01 AM 4842816 6990720 3039616 38.56 2688 2314224 731444 8.19 278704 2374012 4 01:48:01 AM 4842544 6990524 3039888 38.57 2688 2314300 731444 8.19 278704 2374088 4 01:49:01 AM 4842508 6990492 3039924 38.57 2688 2314304 731444 8.19 278704 2374172 4 01:50:01 AM 4842080 6990240 3040352 38.57 2688 2314480 725756 8.13 278704 2374388 184 01:51:01 AM 4842312 6990532 3040120 38.57 2688 2314540 725756 8.13 278704 2374340 16 01:52:01 AM 4842184 6990544 3040248 38.57 2688 2314680 741212 8.30 278712 2374476 4 01:53:01 AM 4842272 6990632 3040160 38.57 2688 2314680 741212 8.30 278712 2374476 4 01:54:01 AM 4841916 6990336 3040516 38.57 2688 2314740 741212 8.30 278712 2374608 4 01:55:01 AM 4842088 6990604 3040344 38.57 2688 2314836 741212 8.30 278712 2374636 48 01:56:01 AM 4842084 6990664 3040348 38.57 2688 2314900 741212 8.30 278720 2374792 4 01:57:01 AM 4841756 6990528 3040676 38.58 2688 2315092 741212 8.30 278716 2374940 136 01:58:01 AM 4841708 6990480 3040724 38.58 2688 2315092 741212 8.30 278716 2374876 4 01:59:01 AM 4841428 6990292 3041004 38.58 2688 2315192 741212 8.30 278716 2375124 108 02:00:01 AM 4841160 6990252 3041272 38.58 2688 2315420 736164 8.24 278716 2375460 120 02:01:01 AM 4839852 6989324 3042580 38.60 2688 2315800 736164 8.24 278716 2376460 40 02:02:01 AM 4840496 6990364 3041936 38.59 2688 2316188 736164 8.24 278716 2376028 192 02:03:01 AM 4840268 6990512 3042164 38.59 2688 2316572 736164 8.24 278716 2376420 56 02:04:01 AM 4839768 6990400 3042664 38.60 2688 2316952 736164 8.24 278716 2376876 208 02:05:01 AM 4839476 6990492 3042956 38.60 2688 2317336 736164 8.24 278716 2377224 68 02:06:01 AM 4838832 6990432 3043600 38.61 2688 2317920 736164 8.24 278716 2377908 256 02:07:01 AM 4838556 6990488 3043876 38.62 2688 2318252 736164 8.24 278716 2378240 80 02:08:01 AM 4838500 6990436 3043932 38.62 2688 2318256 736164 8.24 278716 2378248 4 02:09:01 AM 4838476 6990492 3043956 38.62 2688 2318336 736164 8.24 278716 2378396 96 02:10:01 AM 4838396 6990412 3044036 38.62 2688 2318336 728020 8.15 278716 2378444 8 02:11:01 AM 4838412 6990444 3044020 38.62 2688 2318348 728020 8.15 278716 2378336 16 02:12:01 AM 4838132 6990236 3044300 38.62 2688 2318420 728020 8.15 278716 2378412 4 02:13:01 AM 4838044 6990152 3044388 38.62 2688 2318424 743396 8.32 278716 2378416 4 02:14:01 AM 4837672 6989960 3044760 38.63 2688 2318604 743396 8.32 278716 2378720 16 02:15:01 AM 4837680 6990160 3044752 38.63 2688 2318796 743396 8.32 278716 2378844 4 02:16:01 AM 4837804 6990288 3044628 38.63 2688 2318800 743396 8.32 278716 2378848 4 02:17:01 AM 4837828 6990352 3044604 38.63 2688 2318840 743396 8.32 278716 2378888 48 02:18:01 AM 4837812 6990416 3044620 38.63 2688 2318920 743396 8.32 278732 2378952 72 02:19:01 AM 4837408 6990120 3045024 38.63 2688 2319028 743396 8.32 278732 2379132 56 02:20:01 AM 4837420 6990192 3045012 38.63 2688 2319088 761504 8.53 278732 2379240 52 02:21:01 AM 4837228 6990140 3045204 38.63 2688 2319224 761504 8.53 278732 2379260 4 02:22:01 AM 4837212 6990128 3045220 38.63 2688 2319228 761504 8.53 278732 2379380 4 02:23:01 AM 4837164 6990224 3045268 38.63 2688 2319372 743612 8.33 278732 2379400 16 02:24:01 AM 4836496 6989900 3045936 38.64 2688 2319724 743612 8.33 278732 2380052 40 02:25:01 AM 4836480 6990268 3045952 38.64 2688 2320108 743612 8.33 278732 2380232 24 02:26:01 AM 4835672 6989848 3046760 38.65 2688 2320488 743612 8.33 278732 2380824 4 02:27:01 AM 4835624 6990176 3046808 38.65 2688 2320872 743612 8.33 278732 2380996 40 02:28:02 AM 4835168 6990080 3047264 38.66 2688 2321224 743612 8.33 278732 2381360 128 02:29:01 AM 4834984 6990208 3047448 38.66 2688 2321544 743612 8.33 278732 2381736 4 02:30:01 AM 4833616 6989496 3048816 38.68 2688 2322192 736312 8.24 278732 2382836 40 02:31:01 AM 4833472 6989648 3048960 38.68 2688 2322488 736312 8.24 278732 2383068 8 02:32:01 AM 4833308 6989768 3049124 38.68 2688 2322772 736312 8.24 278732 2383308 116 02:33:01 AM 4832868 6989612 3049564 38.69 2688 2323064 736312 8.24 278732 2383596 56 02:34:01 AM 4832864 6989892 3049568 38.69 2688 2323348 736312 8.24 278732 2383676 60 02:35:01 AM 4832688 6990004 3049744 38.69 2688 2323636 736312 8.24 278732 2383896 108 02:36:01 AM 4832504 6990196 3049928 38.69 2688 2324004 736312 8.24 278732 2384256 276 02:37:01 AM 4832288 6990036 3050144 38.70 2688 2324060 736312 8.24 278732 2384320 16 02:38:01 AM 4832120 6990016 3050312 38.70 2688 2324208 736312 8.24 278736 2384464 24 02:39:01 AM 4832156 6990088 3050276 38.70 2688 2324244 736312 8.24 278736 2384568 40 02:40:01 AM 4831688 6989860 3050744 38.70 2688 2324484 733856 8.22 278740 2384844 228 02:41:01 AM 4831352 6989800 3051080 38.71 2688 2324760 733856 8.22 278740 2385012 264 02:42:01 AM 4831036 6989872 3051396 38.71 2688 2325144 733856 8.22 278740 2385396 192 02:43:01 AM 4830700 6989924 3051732 38.72 2688 2325524 733856 8.22 278740 2385784 112 02:44:01 AM 4830212 6989816 3052220 38.72 2688 2325912 733856 8.22 278740 2386440 40 02:45:01 AM 4829624 6989608 3052808 38.73 2688 2326292 733856 8.22 278740 2386888 128 02:46:01 AM 4829572 6989936 3052860 38.73 2688 2326672 733856 8.22 278740 2386960 52 02:47:01 AM 4828860 6989952 3053572 38.74 2688 2327392 733856 8.22 278740 2387692 4 02:48:01 AM 4828272 6989564 3054160 38.75 2688 2327592 733856 8.22 278740 2388116 4 02:49:01 AM 4828320 6989896 3054112 38.75 2688 2327876 733856 8.22 278740 2388264 4 02:50:01 AM 4827960 6989828 3054472 38.75 2688 2328168 751540 8.41 278740 2388720 104 02:51:01 AM 4827704 6989864 3054728 38.75 2688 2328460 751540 8.41 278740 2388780 4 02:52:01 AM 4827448 6989896 3054984 38.76 2688 2328748 734884 8.23 278740 2389068 4 02:53:01 AM 4826996 6989872 3055436 38.76 2688 2329176 734884 8.23 278740 2389480 4 02:54:01 AM 4826756 6989752 3055676 38.77 2688 2329296 734884 8.23 278740 2389864 4 02:55:01 AM 4826616 6989820 3055816 38.77 2688 2329508 750480 8.40 278740 2389840 56 02:56:01 AM 4826164 6989748 3056268 38.77 2688 2329888 750480 8.40 278740 2390264 52 02:57:01 AM 4825816 6989784 3056616 38.78 2688 2330272 750480 8.40 278740 2390688 88 02:58:01 AM 4825368 6989716 3057064 38.78 2688 2330652 750480 8.40 278740 2391068 236 02:59:01 AM 4825080 6989812 3057352 38.79 2688 2331036 750480 8.40 278740 2391524 160 03:00:01 AM 4824332 6989448 3058100 38.80 2688 2331420 744504 8.34 278740 2391956 24 03:01:01 AM 4822976 6988488 3059456 38.81 2688 2331804 744504 8.34 278740 2393160 4 03:02:01 AM 4823704 6989796 3058728 38.80 2688 2332392 744504 8.34 278740 2392828 4 03:03:01 AM 4823336 6989724 3059096 38.81 2688 2332680 744504 8.34 278740 2393108 152 03:04:01 AM 4822872 6989548 3059560 38.81 2688 2332968 744504 8.34 278740 2393480 100 03:05:01 AM 4822780 6989744 3059652 38.82 2688 2333256 744504 8.34 278740 2393700 4 03:06:01 AM 4822520 6989772 3059912 38.82 2688 2333544 744504 8.34 278740 2393988 104 03:07:01 AM 4822184 6989724 3060248 38.82 2688 2333832 744504 8.34 278740 2394392 4 03:08:01 AM 4822040 6989856 3060392 38.83 2688 2334108 725908 8.13 278740 2394540 16 03:09:01 AM 4821672 6989656 3060760 38.83 2688 2334276 725908 8.13 278740 2394772 60 03:10:01 AM 4821228 6989556 3061204 38.84 2688 2334628 758552 8.49 278740 2395180 88 03:11:01 AM 4821012 6989728 3061420 38.84 2688 2335012 743172 8.32 278740 2395452 16 03:12:01 AM 4820476 6989576 3061956 38.85 2688 2335396 743172 8.32 278740 2395952 4 03:13:01 AM 4820092 6989572 3062340 38.85 2688 2335776 743172 8.32 278740 2396332 160 03:14:01 AM 4819648 6989512 3062784 38.86 2688 2336160 743172 8.32 278740 2396676 24 03:15:01 AM 4819452 6989700 3062980 38.86 2688 2336544 743172 8.32 278740 2397108 176 03:16:01 AM 4818804 6989692 3063628 38.87 2688 2337176 743172 8.32 278740 2397712 20 03:17:01 AM 4818644 6989808 3063788 38.87 2688 2337460 743172 8.32 278740 2397988 56 03:18:01 AM 4818156 6989612 3064276 38.87 2688 2337752 743172 8.32 278740 2398280 152 03:19:01 AM 4817732 6989468 3064700 38.88 2688 2338040 743172 8.32 278740 2398696 56 03:20:01 AM 4817204 6989224 3065228 38.89 2688 2338324 758548 8.49 278740 2398960 152 03:21:01 AM 4817284 6989600 3065148 38.89 2688 2338616 758548 8.49 278740 2399136 80 03:22:01 AM 4816960 6989660 3065472 38.89 2688 2338984 735124 8.23 278740 2399504 228 03:23:01 AM 4816800 6989672 3065632 38.89 2688 2339156 735124 8.23 278740 2399680 80 03:24:01 AM 4816416 6989496 3066016 38.90 2688 2339372 750368 8.40 279388 2399448 100 03:25:01 AM 4815892 6989340 3066540 38.90 2688 2339740 750368 8.40 279388 2399840 4 03:26:01 AM 4815824 6989656 3066608 38.90 2688 2340124 750368 8.40 279388 2400020 36 03:27:01 AM 4815552 6989768 3066880 38.91 2688 2340508 750368 8.40 279388 2400404 72 03:28:01 AM 4814628 6989224 3067804 38.92 2688 2340888 750368 8.40 279388 2401268 36 03:29:01 AM 4814580 6989568 3067852 38.92 2688 2341272 750368 8.40 279388 2401272 176 03:30:01 AM 4813328 6989080 3069104 38.94 2688 2342032 744640 8.34 279388 2402232 476 03:31:01 AM 4813236 6989200 3069196 38.94 2688 2342252 744640 8.34 279388 2402416 200 03:32:01 AM 4813368 6989616 3069064 38.94 2688 2342536 744640 8.34 279388 2402496 100 03:33:01 AM 4812604 6989140 3069828 38.95 2688 2342824 744640 8.34 279388 2403196 200 03:34:02 AM 4812392 6989176 3070040 38.95 2688 2343064 744640 8.34 279388 2403236 52 03:35:01 AM 4812416 6989488 3070016 38.95 2688 2343352 744640 8.34 279388 2403340 152 03:36:01 AM 4812244 6989604 3070188 38.95 2688 2343640 744640 8.34 279388 2403628 56 03:37:01 AM 4811088 6988928 3071344 38.96 2688 2344120 744640 8.34 279392 2404468 276 03:38:01 AM 4808428 6986556 3074004 39.00 2688 2344408 777944 8.71 279392 2406948 272 03:39:01 AM 4807636 6986084 3074796 39.01 2688 2344728 795904 8.91 279392 2407520 284 03:40:01 AM 4806812 6985552 3075620 39.02 2688 2345012 818960 9.17 279392 2407920 4 03:41:01 AM 4804292 6983404 3078140 39.05 2688 2345380 852264 9.54 279392 2410532 32 03:42:01 AM 4803664 6983072 3078768 39.06 2688 2345676 868632 9.73 279392 2411040 4 03:43:01 AM 4800872 6980588 3081560 39.09 2688 2345976 901476 10.09 279392 2413360 24 03:44:01 AM 4882748 6943924 2999684 38.06 2688 2232832 711548 7.97 371528 2261888 90964 03:45:01 AM 4855028 6919988 3027404 38.41 2688 2237476 744332 8.33 531340 2135284 26268 Average: 4843063 6989888 3039369 38.56 2688 2313203 741612 8.30 275574 2376869 584 12:00:01 AM IFACE rxpck/s txpck/s rxkB/s txkB/s rxcmp/s txcmp/s rxmcst/s %ifutil 12:01:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:01:01 AM eth0 0.83 0.15 0.11 0.02 0.00 0.00 0.00 0.00 12:02:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:02:01 AM eth0 9.71 9.45 1.60 0.88 0.00 0.00 0.00 0.00 12:03:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:03:01 AM eth0 5.80 4.63 0.77 0.36 0.00 0.00 0.00 0.00 12:04:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:04:01 AM eth0 13.18 9.88 2.66 1.43 0.00 0.00 0.00 0.00 12:05:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:05:01 AM eth0 0.95 0.50 0.10 0.09 0.00 0.00 0.00 0.00 12:06:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:06:01 AM eth0 4.17 0.32 0.80 0.26 0.00 0.00 0.00 0.00 12:07:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:07:01 AM eth0 4.65 4.32 0.65 0.30 0.00 0.00 0.00 0.00 12:08:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:08:01 AM eth0 1.03 0.50 0.10 0.09 0.00 0.00 0.00 0.00 12:09:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:09:01 AM eth0 0.53 0.18 0.16 0.07 0.00 0.00 0.00 0.00 12:10:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:10:01 AM eth0 9.32 8.72 1.52 0.88 0.00 0.00 0.00 0.00 12:11:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:11:01 AM eth0 10.13 8.55 2.50 1.66 0.00 0.00 0.00 0.00 12:12:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:12:01 AM eth0 11.35 8.97 2.85 1.79 0.00 0.00 0.00 0.00 12:13:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:13:01 AM eth0 12.46 8.97 3.05 1.78 0.00 0.00 0.00 0.00 12:14:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:14:01 AM eth0 13.33 9.85 4.10 2.68 0.00 0.00 0.00 0.00 12:15:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:15:01 AM eth0 10.75 8.95 2.71 1.78 0.00 0.00 0.00 0.00 12:16:02 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:16:02 AM eth0 11.50 9.08 2.81 1.79 0.00 0.00 0.00 0.00 12:17:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:17:01 AM eth0 18.52 15.56 4.80 2.98 0.00 0.00 0.00 0.00 12:18:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:18:01 AM eth0 0.93 0.52 0.14 0.07 0.00 0.00 0.00 0.00 12:19:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:19:01 AM eth0 0.93 0.15 0.24 0.07 0.00 0.00 0.00 0.00 12:20:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:20:01 AM eth0 1.33 0.65 0.48 0.31 0.00 0.00 0.00 0.00 12:21:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:21:01 AM eth0 0.30 0.05 0.02 0.00 0.00 0.00 0.00 0.00 12:22:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:22:01 AM eth0 0.77 0.42 0.11 0.05 0.00 0.00 0.00 0.00 12:23:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:23:01 AM eth0 1.65 0.05 0.08 0.00 0.00 0.00 0.00 0.00 12:24:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:24:01 AM eth0 7.45 3.50 1.23 0.72 0.00 0.00 0.00 0.00 12:25:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:25:01 AM eth0 12.98 11.71 2.05 1.07 0.00 0.00 0.00 0.00 12:26:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:26:01 AM eth0 6.55 5.30 1.18 0.64 0.00 0.00 0.00 0.00 12:27:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:27:01 AM eth0 1.33 0.62 0.64 0.48 0.00 0.00 0.00 0.00 12:28:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:28:01 AM eth0 1.18 0.05 0.16 0.00 0.00 0.00 0.00 0.00 12:29:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:29:01 AM eth0 5.90 5.25 1.20 0.72 0.00 0.00 0.00 0.00 12:30:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:30:01 AM eth0 0.57 0.05 0.04 0.00 0.00 0.00 0.00 0.00 12:31:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:31:01 AM eth0 8.26 5.26 1.52 0.73 0.00 0.00 0.00 0.00 12:32:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:32:01 AM eth0 1.57 0.85 0.37 0.29 0.00 0.00 0.00 0.00 12:33:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:33:01 AM eth0 3.17 0.35 0.71 0.26 0.00 0.00 0.00 0.00 12:34:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:34:01 AM eth0 6.50 4.68 1.20 0.63 0.00 0.00 0.00 0.00 12:35:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:35:01 AM eth0 6.50 5.20 0.93 0.45 0.00 0.00 0.00 0.00 12:36:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:36:01 AM eth0 0.77 0.23 0.09 0.03 0.00 0.00 0.00 0.00 12:37:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:37:01 AM eth0 0.32 0.05 0.02 0.00 0.00 0.00 0.00 0.00 12:38:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:38:01 AM eth0 7.83 7.03 1.31 0.82 0.00 0.00 0.00 0.00 12:39:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:39:01 AM eth0 13.33 11.20 2.46 1.37 0.00 0.00 0.00 0.00 12:40:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:40:01 AM eth0 13.11 9.40 3.13 2.00 0.00 0.00 0.00 0.00 12:41:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:41:01 AM eth0 12.86 10.03 3.00 1.91 0.00 0.00 0.00 0.00 12:42:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:42:01 AM eth0 12.51 9.42 3.25 2.00 0.00 0.00 0.00 0.00 12:43:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:43:01 AM eth0 12.78 8.96 2.97 1.79 0.00 0.00 0.00 0.00 12:44:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:44:01 AM eth0 13.28 9.32 3.40 2.11 0.00 0.00 0.00 0.00 12:45:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:45:01 AM eth0 16.84 13.98 4.94 3.10 0.00 0.00 0.00 0.00 12:46:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:46:01 AM eth0 8.81 5.66 1.72 0.99 0.00 0.00 0.00 0.00 12:47:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:47:01 AM eth0 0.83 0.05 0.09 0.00 0.00 0.00 0.00 0.00 12:48:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:48:01 AM eth0 1.05 0.40 0.15 0.05 0.00 0.00 0.00 0.00 12:49:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:49:01 AM eth0 0.65 0.15 0.16 0.07 0.00 0.00 0.00 0.00 12:50:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:50:01 AM eth0 0.45 0.02 0.03 0.00 0.00 0.00 0.00 0.00 12:51:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:51:01 AM eth0 0.60 0.35 0.10 0.05 0.00 0.00 0.00 0.00 12:52:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:52:01 AM eth0 0.35 0.00 0.02 0.00 0.00 0.00 0.00 0.00 12:53:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:53:01 AM eth0 10.75 9.90 1.87 1.13 0.00 0.00 0.00 0.00 12:54:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:54:01 AM eth0 9.73 5.68 1.70 0.76 0.00 0.00 0.00 0.00 12:55:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:55:01 AM eth0 0.93 0.38 0.53 0.39 0.00 0.00 0.00 0.00 12:56:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:56:01 AM eth0 9.25 4.57 1.32 0.32 0.00 0.00 0.00 0.00 12:57:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:57:01 AM eth0 3.10 0.57 0.21 0.11 0.00 0.00 0.00 0.00 12:58:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:58:01 AM eth0 2.10 0.05 0.10 0.00 0.00 0.00 0.00 0.00 12:59:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 12:59:01 AM eth0 4.88 4.42 0.77 0.36 0.00 0.00 0.00 0.00 01:00:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:00:01 AM eth0 6.13 5.45 0.90 0.46 0.00 0.00 0.00 0.00 01:01:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:01:01 AM eth0 7.98 7.75 1.24 0.59 0.00 0.00 0.00 0.00 01:02:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:02:01 AM eth0 5.51 4.76 0.81 0.41 0.00 0.00 0.00 0.00 01:03:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:03:01 AM eth0 1.90 0.20 0.15 0.04 0.00 0.00 0.00 0.00 01:04:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:04:01 AM eth0 3.28 0.50 0.97 0.51 0.00 0.00 0.00 0.00 01:05:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:05:01 AM eth0 5.31 4.63 1.20 0.76 0.00 0.00 0.00 0.00 01:06:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:06:01 AM eth0 2.53 0.00 0.32 0.00 0.00 0.00 0.00 0.00 01:07:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:07:01 AM eth0 2.68 2.27 0.58 0.44 0.00 0.00 0.00 0.00 01:08:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:08:01 AM eth0 11.35 10.28 2.04 1.15 0.00 0.00 0.00 0.00 01:09:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:09:01 AM eth0 12.23 9.66 3.33 2.19 0.00 0.00 0.00 0.00 01:10:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:10:01 AM eth0 11.16 8.97 2.73 1.79 0.00 0.00 0.00 0.00 01:11:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:11:01 AM eth0 10.70 8.90 2.69 1.78 0.00 0.00 0.00 0.00 01:12:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:12:01 AM eth0 11.11 8.88 2.78 1.78 0.00 0.00 0.00 0.00 01:13:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:13:01 AM eth0 11.26 9.30 2.85 1.84 0.00 0.00 0.00 0.00 01:14:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:14:01 AM eth0 11.73 9.26 3.07 1.87 0.00 0.00 0.00 0.00 01:15:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:15:01 AM eth0 12.76 11.25 3.34 2.09 0.00 0.00 0.00 0.00 01:16:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:16:01 AM eth0 1.15 0.00 0.17 0.00 0.00 0.00 0.00 0.00 01:17:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:17:01 AM eth0 1.00 0.48 0.43 0.30 0.00 0.00 0.00 0.00 01:18:02 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:18:02 AM eth0 0.98 0.12 0.08 0.01 0.00 0.00 0.00 0.00 01:19:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:19:01 AM eth0 1.14 0.32 0.50 0.26 0.00 0.00 0.00 0.00 01:20:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:20:01 AM eth0 0.57 0.33 0.09 0.05 0.00 0.00 0.00 0.00 01:21:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:21:01 AM eth0 0.83 0.23 0.31 0.20 0.00 0.00 0.00 0.00 01:22:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:22:01 AM eth0 10.51 9.95 1.84 1.10 0.00 0.00 0.00 0.00 01:23:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:23:01 AM eth0 10.03 9.90 1.46 0.78 0.00 0.00 0.00 0.00 01:24:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:24:01 AM eth0 1.10 0.75 0.23 0.18 0.00 0.00 0.00 0.00 01:25:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:25:01 AM eth0 0.25 0.08 0.02 0.00 0.00 0.00 0.00 0.00 01:26:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:26:01 AM eth0 4.77 4.48 0.66 0.31 0.00 0.00 0.00 0.00 01:27:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:27:01 AM eth0 1.38 0.72 0.46 0.34 0.00 0.00 0.00 0.00 01:28:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:28:01 AM eth0 0.22 0.00 0.02 0.00 0.00 0.00 0.00 0.00 01:29:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:29:01 AM eth0 5.43 5.17 0.89 0.46 0.00 0.00 0.00 0.00 01:30:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:30:01 AM eth0 0.52 0.33 0.04 0.05 0.00 0.00 0.00 0.00 01:31:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:31:01 AM eth0 0.60 0.23 0.27 0.21 0.00 0.00 0.00 0.00 01:32:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:32:01 AM eth0 10.15 9.53 1.46 0.71 0.00 0.00 0.00 0.00 01:33:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:33:01 AM eth0 1.32 0.80 0.44 0.37 0.00 0.00 0.00 0.00 01:34:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:34:01 AM eth0 0.65 0.18 0.17 0.07 0.00 0.00 0.00 0.00 01:35:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:35:01 AM eth0 5.25 4.72 1.05 0.57 0.00 0.00 0.00 0.00 01:36:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:36:01 AM eth0 9.00 8.93 1.52 0.94 0.00 0.00 0.00 0.00 01:37:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:37:01 AM eth0 10.90 9.25 2.70 1.80 0.00 0.00 0.00 0.00 01:38:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:38:01 AM eth0 11.00 9.05 2.79 1.79 0.00 0.00 0.00 0.00 01:39:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:39:01 AM eth0 11.03 9.15 2.88 1.85 0.00 0.00 0.00 0.00 01:40:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:40:01 AM eth0 10.85 9.02 2.76 1.79 0.00 0.00 0.00 0.00 01:41:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:41:01 AM eth0 11.30 8.96 2.84 1.79 0.00 0.00 0.00 0.00 01:42:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:42:01 AM eth0 11.40 9.13 3.06 1.97 0.00 0.00 0.00 0.00 01:43:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:43:01 AM eth0 17.71 14.43 4.51 2.74 0.00 0.00 0.00 0.00 01:44:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:44:01 AM eth0 1.30 0.70 0.57 0.37 0.00 0.00 0.00 0.00 01:45:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:45:01 AM eth0 0.17 0.00 0.01 0.00 0.00 0.00 0.00 0.00 01:46:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:46:01 AM eth0 0.57 0.32 0.08 0.04 0.00 0.00 0.00 0.00 01:47:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:47:01 AM eth0 0.22 0.03 0.02 0.00 0.00 0.00 0.00 0.00 01:48:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:48:01 AM eth0 0.87 0.38 0.14 0.05 0.00 0.00 0.00 0.00 01:49:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:49:01 AM eth0 0.38 0.20 0.15 0.07 0.00 0.00 0.00 0.00 01:50:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:50:01 AM eth0 9.95 9.45 1.78 1.06 0.00 0.00 0.00 0.00 01:51:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:51:01 AM eth0 5.76 5.40 0.83 0.40 0.00 0.00 0.00 0.00 01:52:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:52:01 AM eth0 0.80 0.58 0.09 0.10 0.00 0.00 0.00 0.00 01:53:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:53:01 AM eth0 0.42 0.15 0.03 0.01 0.00 0.00 0.00 0.00 01:54:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:54:01 AM eth0 5.56 5.33 0.89 0.47 0.00 0.00 0.00 0.00 01:55:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:55:01 AM eth0 9.13 9.00 1.30 0.67 0.00 0.00 0.00 0.00 01:56:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:56:01 AM eth0 5.37 4.90 0.84 0.42 0.00 0.00 0.00 0.00 01:57:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:57:01 AM eth0 5.46 5.28 0.75 0.43 0.00 0.00 0.00 0.00 01:58:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:58:01 AM eth0 0.25 0.00 0.02 0.00 0.00 0.00 0.00 0.00 01:59:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 01:59:01 AM eth0 2.83 2.47 0.71 0.52 0.00 0.00 0.00 0.00 02:00:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:00:01 AM eth0 11.30 10.70 2.15 1.28 0.00 0.00 0.00 0.00 02:01:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:01:01 AM eth0 10.85 9.03 2.77 1.79 0.00 0.00 0.00 0.00 02:02:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:02:01 AM eth0 10.43 8.85 2.65 1.78 0.00 0.00 0.00 0.00 02:03:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:03:01 AM eth0 10.90 9.03 2.75 1.79 0.00 0.00 0.00 0.00 02:04:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:04:01 AM eth0 11.06 8.96 2.85 1.84 0.00 0.00 0.00 0.00 02:05:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:05:01 AM eth0 11.28 9.23 2.84 1.84 0.00 0.00 0.00 0.00 02:06:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:06:01 AM eth0 15.21 13.00 4.10 2.56 0.00 0.00 0.00 0.00 02:07:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:07:01 AM eth0 8.32 6.63 1.90 1.17 0.00 0.00 0.00 0.00 02:08:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:08:01 AM eth0 0.23 0.00 0.02 0.00 0.00 0.00 0.00 0.00 02:09:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:09:01 AM eth0 0.80 0.42 0.22 0.11 0.00 0.00 0.00 0.00 02:10:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:10:01 AM eth0 0.63 0.05 0.10 0.00 0.00 0.00 0.00 0.00 02:11:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:11:01 AM eth0 0.17 0.02 0.02 0.00 0.00 0.00 0.00 0.00 02:12:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:12:01 AM eth0 0.48 0.28 0.08 0.04 0.00 0.00 0.00 0.00 02:13:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:13:01 AM eth0 0.18 0.05 0.02 0.00 0.00 0.00 0.00 0.00 02:14:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:14:01 AM eth0 10.73 10.43 1.95 7.41 0.00 0.00 0.00 0.00 02:15:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:15:01 AM eth0 5.33 5.28 0.75 0.42 0.00 0.00 0.00 0.00 02:16:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:16:01 AM eth0 0.37 0.00 0.07 0.00 0.00 0.00 0.00 0.00 02:17:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:17:01 AM eth0 4.87 4.68 0.68 0.33 0.00 0.00 0.00 0.00 02:18:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:18:01 AM eth0 4.45 4.37 0.63 0.35 0.00 0.00 0.00 0.00 02:19:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:19:01 AM eth0 9.78 9.48 1.47 6.97 0.00 0.00 0.00 0.00 02:20:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:20:01 AM eth0 5.13 5.00 0.74 0.39 0.00 0.00 0.00 0.00 02:21:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:21:01 AM eth0 0.95 0.58 0.16 0.10 0.00 0.00 0.00 0.00 02:22:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:22:01 AM eth0 0.18 0.05 0.02 0.00 0.00 0.00 0.00 0.00 02:23:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:23:01 AM eth0 8.83 8.62 1.49 0.87 0.00 0.00 0.00 0.00 02:24:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:24:01 AM eth0 11.83 10.13 2.98 8.22 0.00 0.00 0.00 0.00 02:25:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:25:01 AM eth0 11.55 9.00 2.90 1.79 0.00 0.00 0.00 0.00 02:26:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:26:01 AM eth0 12.16 9.45 3.20 2.06 0.00 0.00 0.00 0.00 02:27:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:27:01 AM eth0 11.60 9.03 2.85 1.79 0.00 0.00 0.00 0.00 02:28:02 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:28:02 AM eth0 10.60 8.78 2.67 1.73 0.00 0.00 0.00 0.00 02:29:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:29:01 AM eth0 11.37 7.83 2.65 1.57 0.00 0.00 0.00 0.00 02:30:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:30:01 AM eth0 21.71 15.03 5.52 3.25 0.00 0.00 0.00 0.00 02:31:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:31:01 AM eth0 1.45 1.08 0.77 0.52 0.00 0.00 0.00 0.00 02:32:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:32:01 AM eth0 4.95 0.87 1.04 0.33 0.00 0.00 0.00 0.00 02:33:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:33:01 AM eth0 4.73 0.90 1.04 0.39 0.00 0.00 0.00 0.00 02:34:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:34:01 AM eth0 1.73 0.72 0.50 0.19 0.00 0.00 0.00 0.00 02:35:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:35:01 AM eth0 0.97 0.60 0.27 0.13 0.00 0.00 0.00 0.00 02:36:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:36:01 AM eth0 10.36 9.60 1.95 1.13 0.00 0.00 0.00 0.00 02:37:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:37:01 AM eth0 4.67 3.62 0.65 0.29 0.00 0.00 0.00 0.00 02:38:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:38:01 AM eth0 5.13 4.00 0.71 0.38 0.00 0.00 0.00 0.00 02:39:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:39:01 AM eth0 6.97 4.62 0.92 0.38 0.00 0.00 0.00 0.00 02:40:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:40:01 AM eth0 11.40 9.10 1.64 0.96 0.00 0.00 0.00 0.00 02:41:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:41:01 AM eth0 7.83 6.33 1.90 1.23 0.00 0.00 0.00 0.00 02:42:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:42:01 AM eth0 11.28 8.96 2.81 1.79 0.00 0.00 0.00 0.00 02:43:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:43:01 AM eth0 11.33 9.03 2.81 1.79 0.00 0.00 0.00 0.00 02:44:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:44:01 AM eth0 13.93 9.70 3.72 2.31 0.00 0.00 0.00 0.00 02:45:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:45:01 AM eth0 11.08 9.23 2.98 1.98 0.00 0.00 0.00 0.00 02:46:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:46:01 AM eth0 12.71 9.16 3.02 1.80 0.00 0.00 0.00 0.00 02:47:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:47:01 AM eth0 21.91 16.45 5.31 3.15 0.00 0.00 0.00 0.00 02:48:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:48:01 AM eth0 1.48 0.80 0.51 0.31 0.00 0.00 0.00 0.00 02:49:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:49:01 AM eth0 1.23 0.68 0.37 0.18 0.00 0.00 0.00 0.00 02:50:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:50:01 AM eth0 0.87 0.60 0.26 0.13 0.00 0.00 0.00 0.00 02:51:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:51:01 AM eth0 0.87 0.60 0.26 0.13 0.00 0.00 0.00 0.00 02:52:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:52:01 AM eth0 0.95 0.60 0.27 0.13 0.00 0.00 0.00 0.00 02:53:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:53:01 AM eth0 5.06 3.90 1.18 0.78 0.00 0.00 0.00 0.00 02:54:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:54:01 AM eth0 11.68 9.28 2.50 1.45 0.00 0.00 0.00 0.00 02:55:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:55:01 AM eth0 11.40 10.85 2.06 1.27 0.00 0.00 0.00 0.00 02:56:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:56:01 AM eth0 21.88 9.00 70.33 1.79 0.00 0.00 0.00 0.00 02:57:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:57:01 AM eth0 11.51 9.00 2.80 1.79 0.00 0.00 0.00 0.00 02:58:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:58:01 AM eth0 11.18 9.02 2.76 1.79 0.00 0.00 0.00 0.00 02:59:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 02:59:01 AM eth0 11.11 9.00 2.85 1.84 0.00 0.00 0.00 0.00 03:00:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:00:01 AM eth0 11.56 8.92 2.82 1.78 0.00 0.00 0.00 0.00 03:01:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:01:01 AM eth0 10.96 8.85 2.74 1.78 0.00 0.00 0.00 0.00 03:02:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:02:01 AM eth0 13.28 11.31 3.42 2.13 0.00 0.00 0.00 0.00 03:03:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:03:01 AM eth0 1.12 0.60 0.32 0.13 0.00 0.00 0.00 0.00 03:04:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:04:01 AM eth0 1.33 0.72 0.41 0.20 0.00 0.00 0.00 0.00 03:05:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:05:01 AM eth0 0.85 0.65 0.26 0.13 0.00 0.00 0.00 0.00 03:06:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:06:01 AM eth0 1.13 0.65 0.32 0.13 0.00 0.00 0.00 0.00 03:07:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:07:01 AM eth0 0.65 0.60 0.25 0.13 0.00 0.00 0.00 0.00 03:08:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:08:01 AM eth0 9.03 8.57 1.77 1.03 0.00 0.00 0.00 0.00 03:09:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:09:01 AM eth0 10.25 9.93 2.04 1.25 0.00 0.00 0.00 0.00 03:10:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:10:01 AM eth0 9.85 8.47 2.43 1.65 0.00 0.00 0.00 0.00 03:11:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:11:01 AM eth0 10.88 9.01 2.76 1.79 0.00 0.00 0.00 0.00 03:12:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:12:01 AM eth0 10.75 8.87 2.70 1.78 0.00 0.00 0.00 0.00 03:13:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:13:01 AM eth0 10.96 9.07 2.74 1.79 0.00 0.00 0.00 0.00 03:14:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:14:01 AM eth0 10.81 9.26 2.83 1.86 0.00 0.00 0.00 0.00 03:15:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:15:01 AM eth0 10.88 9.02 2.77 1.79 0.00 0.00 0.00 0.00 03:16:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:16:01 AM eth0 16.94 14.40 4.36 2.73 0.00 0.00 0.00 0.00 03:17:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:17:01 AM eth0 0.75 0.68 0.26 0.14 0.00 0.00 0.00 0.00 03:18:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:18:01 AM eth0 1.27 0.60 0.31 0.13 0.00 0.00 0.00 0.00 03:19:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:19:01 AM eth0 1.42 1.08 0.74 0.46 0.00 0.00 0.00 0.00 03:20:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:20:01 AM eth0 0.78 0.60 0.26 0.13 0.00 0.00 0.00 0.00 03:21:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:21:01 AM eth0 0.65 0.60 0.25 0.13 0.00 0.00 0.00 0.00 03:22:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:22:01 AM eth0 9.57 9.20 1.87 1.10 0.00 0.00 0.00 0.00 03:23:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:23:01 AM eth0 10.36 10.25 1.90 1.20 0.00 0.00 0.00 0.00 03:24:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:24:01 AM eth0 6.77 5.53 1.67 1.09 0.00 0.00 0.00 0.00 03:25:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:25:01 AM eth0 11.45 9.25 3.07 2.04 0.00 0.00 0.00 0.00 03:26:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:26:01 AM eth0 10.88 8.98 2.71 1.77 0.00 0.00 0.00 0.00 03:27:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:27:01 AM eth0 10.80 8.96 2.71 1.76 0.00 0.00 0.00 0.00 03:28:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:28:01 AM eth0 12.25 9.46 3.20 2.02 0.00 0.00 0.00 0.00 03:29:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:29:01 AM eth0 11.61 9.32 2.99 1.89 0.00 0.00 0.00 0.00 03:30:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:30:01 AM eth0 20.84 17.48 5.31 3.36 0.00 0.00 0.00 0.00 03:31:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:31:01 AM eth0 1.07 0.90 0.46 0.31 0.00 0.00 0.00 0.00 03:32:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:32:01 AM eth0 1.17 0.65 0.31 0.13 0.00 0.00 0.00 0.00 03:33:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:33:01 AM eth0 1.00 0.78 0.50 0.32 0.00 0.00 0.00 0.00 03:34:02 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:34:02 AM eth0 1.45 0.93 0.64 0.38 0.00 0.00 0.00 0.00 03:35:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:35:01 AM eth0 0.59 0.61 0.25 0.13 0.00 0.00 0.00 0.00 03:36:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:36:01 AM eth0 3.07 0.65 0.54 0.13 0.00 0.00 0.00 0.00 03:37:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:37:01 AM eth0 17.54 12.13 4.58 2.41 0.00 0.00 0.00 0.00 03:38:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:38:01 AM eth0 10.63 9.07 3.31 1.87 0.00 0.00 0.00 0.00 03:39:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:39:01 AM eth0 15.25 13.41 3.56 1.51 0.00 0.00 0.00 0.00 03:40:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:40:01 AM eth0 10.11 6.83 2.42 0.86 0.00 0.00 0.00 0.00 03:41:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:41:01 AM eth0 13.75 9.70 3.09 1.48 0.00 0.00 0.00 0.00 03:42:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:42:01 AM eth0 8.87 8.13 2.22 0.95 0.00 0.00 0.00 0.00 03:43:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:43:01 AM eth0 7.83 7.36 2.15 0.98 0.00 0.00 0.00 0.00 03:44:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:44:01 AM eth0 132.68 85.70 203.92 252.62 0.00 0.00 0.00 0.00 03:45:01 AM lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 03:45:01 AM eth0 20.54 16.98 15.59 8.63 0.00 0.00 0.00 0.00 Average: lo 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Average: eth0 7.34 5.70 2.78 2.14 0.00 0.00 0.00 0.00 ---> sar -P ALL: Linux 4.18.0-553.5.1.el8.x86_64 (prd-centos8-robot-2c-8g-11129.novalocal) 11/17/2025 _x86_64_ (2 CPU) 12:00:01 AM CPU %user %nice %system %iowait %steal %idle 12:01:01 AM all 0.33 0.03 0.25 0.03 0.03 99.33 12:01:01 AM 0 0.13 0.03 0.30 0.07 0.03 99.43 12:01:01 AM 1 0.53 0.02 0.20 0.00 0.02 99.23 12:02:01 AM all 4.57 0.00 2.00 0.00 0.06 93.37 12:02:01 AM 0 7.36 0.00 3.85 0.00 0.07 88.71 12:02:01 AM 1 1.92 0.00 0.24 0.00 0.05 97.79 12:03:01 AM all 5.71 0.00 2.88 0.00 0.06 91.35 12:03:01 AM 0 0.68 0.00 0.15 0.00 0.07 99.10 12:03:01 AM 1 11.29 0.00 5.90 0.00 0.06 82.76 12:04:01 AM all 5.30 0.00 1.67 0.01 0.07 92.95 12:04:01 AM 0 7.20 0.00 2.59 0.00 0.07 90.14 12:04:01 AM 1 3.48 0.00 0.80 0.02 0.07 95.64 12:05:01 AM all 3.40 0.00 1.74 0.00 0.03 94.82 12:05:01 AM 0 0.62 0.00 0.10 0.00 0.03 99.25 12:05:01 AM 1 6.35 0.00 3.47 0.00 0.04 90.14 12:06:01 AM all 0.36 0.00 0.08 0.00 0.05 99.51 12:06:01 AM 0 0.55 0.00 0.10 0.00 0.05 99.30 12:06:01 AM 1 0.17 0.00 0.07 0.00 0.05 99.72 12:07:01 AM all 2.21 0.00 0.41 0.00 0.04 97.34 12:07:01 AM 0 0.23 0.00 0.10 0.00 0.03 99.63 12:07:01 AM 1 4.27 0.00 0.73 0.00 0.05 94.95 12:08:01 AM all 2.02 0.00 1.02 0.00 0.04 96.91 12:08:01 AM 0 0.48 0.00 0.15 0.00 0.05 99.32 12:08:01 AM 1 3.62 0.00 1.92 0.00 0.03 94.42 12:09:01 AM all 0.36 0.00 0.08 0.00 0.05 99.52 12:09:01 AM 0 0.58 0.00 0.07 0.00 0.05 99.30 12:09:01 AM 1 0.13 0.00 0.08 0.00 0.05 99.73 12:10:01 AM all 3.49 0.00 0.89 0.00 0.06 95.56 12:10:01 AM 0 2.27 0.00 0.22 0.00 0.05 97.46 12:10:01 AM 1 4.74 0.00 1.57 0.00 0.07 93.62 12:11:01 AM all 5.25 0.00 1.87 0.00 0.08 92.81 12:11:01 AM 0 2.94 0.00 0.32 0.00 0.07 96.67 12:11:01 AM 1 7.63 0.00 3.46 0.00 0.09 88.82 12:11:01 AM CPU %user %nice %system %iowait %steal %idle 12:12:01 AM all 4.27 0.00 0.33 0.00 0.08 95.33 12:12:01 AM 0 2.74 0.00 0.23 0.00 0.08 96.94 12:12:01 AM 1 5.80 0.00 0.42 0.00 0.07 93.72 12:13:01 AM all 4.24 0.00 0.36 0.00 0.08 95.32 12:13:01 AM 0 4.23 0.00 0.33 0.00 0.07 95.37 12:13:01 AM 1 4.26 0.00 0.38 0.00 0.08 95.27 12:14:01 AM all 4.41 0.00 0.33 0.00 0.08 95.19 12:14:01 AM 0 4.95 0.00 0.37 0.00 0.08 94.60 12:14:01 AM 1 3.86 0.00 0.28 0.00 0.07 95.79 12:15:01 AM all 4.33 0.00 0.33 0.00 0.08 95.27 12:15:01 AM 0 3.44 0.00 0.27 0.00 0.07 96.22 12:15:01 AM 1 5.21 0.00 0.38 0.00 0.08 94.32 12:16:02 AM all 4.21 0.00 0.31 0.00 0.08 95.40 12:16:02 AM 0 1.92 0.00 0.23 0.00 0.07 97.78 12:16:02 AM 1 6.49 0.00 0.38 0.00 0.10 93.02 12:17:01 AM all 6.31 0.00 0.34 0.00 0.08 93.27 12:17:01 AM 0 8.29 0.00 0.41 0.00 0.09 91.22 12:17:01 AM 1 4.33 0.00 0.27 0.00 0.07 95.33 12:18:01 AM all 0.57 0.00 0.08 0.00 0.06 99.29 12:18:01 AM 0 0.80 0.00 0.08 0.00 0.07 99.05 12:18:01 AM 1 0.33 0.00 0.08 0.00 0.05 99.53 12:19:01 AM all 0.33 0.00 0.06 0.00 0.03 99.59 12:19:01 AM 0 0.55 0.00 0.05 0.00 0.03 99.37 12:19:01 AM 1 0.10 0.00 0.07 0.00 0.02 99.82 12:20:01 AM all 0.48 0.00 0.07 0.00 0.04 99.42 12:20:01 AM 0 0.58 0.00 0.05 0.00 0.03 99.33 12:20:01 AM 1 0.37 0.00 0.08 0.00 0.05 99.50 12:21:01 AM all 0.23 0.00 0.07 0.00 0.04 99.67 12:21:01 AM 0 0.35 0.00 0.05 0.00 0.03 99.57 12:21:01 AM 1 0.10 0.00 0.08 0.00 0.05 99.77 12:22:01 AM all 0.36 0.00 0.09 0.00 0.05 99.50 12:22:01 AM 0 0.57 0.00 0.08 0.00 0.03 99.32 12:22:01 AM 1 0.15 0.00 0.10 0.00 0.07 99.68 12:22:01 AM CPU %user %nice %system %iowait %steal %idle 12:23:01 AM all 0.26 0.00 0.09 0.00 0.03 99.62 12:23:01 AM 0 0.08 0.00 0.07 0.00 0.03 99.82 12:23:01 AM 1 0.43 0.00 0.12 0.00 0.03 99.42 12:24:01 AM all 2.05 0.00 0.18 0.00 0.03 97.74 12:24:01 AM 0 2.68 0.00 0.25 0.00 0.03 97.03 12:24:01 AM 1 1.42 0.00 0.10 0.00 0.03 98.45 12:25:01 AM all 5.39 0.00 3.74 0.00 0.07 90.80 12:25:01 AM 0 0.70 0.00 0.17 0.00 0.07 99.06 12:25:01 AM 1 10.57 0.00 7.69 0.00 0.07 81.66 12:26:01 AM all 5.66 0.00 3.06 0.00 0.06 91.21 12:26:01 AM 0 0.68 0.00 0.17 0.00 0.07 99.08 12:26:01 AM 1 11.19 0.00 6.28 0.00 0.06 82.47 12:27:01 AM all 1.20 0.00 0.22 0.00 0.05 98.53 12:27:01 AM 0 0.68 0.00 0.08 0.00 0.07 99.16 12:27:01 AM 1 1.72 0.00 0.35 0.00 0.03 97.90 12:28:01 AM all 0.29 0.00 0.08 0.00 0.04 99.58 12:28:01 AM 0 0.23 0.00 0.08 0.00 0.05 99.63 12:28:01 AM 1 0.35 0.00 0.08 0.00 0.03 99.53 12:29:01 AM all 3.74 0.00 1.33 0.00 0.04 94.88 12:29:01 AM 0 0.45 0.00 0.10 0.00 0.03 99.42 12:29:01 AM 1 7.29 0.00 2.67 0.00 0.05 89.99 12:30:01 AM all 0.36 0.00 0.08 0.00 0.05 99.52 12:30:01 AM 0 0.60 0.00 0.08 0.00 0.05 99.27 12:30:01 AM 1 0.12 0.00 0.07 0.00 0.05 99.77 12:31:01 AM all 0.70 0.00 0.13 0.00 0.04 99.14 12:31:01 AM 0 0.58 0.00 0.08 0.00 0.02 99.32 12:31:01 AM 1 0.81 0.00 0.17 0.00 0.07 98.96 12:32:01 AM all 3.70 0.00 1.74 0.00 0.06 94.50 12:32:01 AM 0 0.65 0.00 0.08 0.00 0.07 99.20 12:32:01 AM 1 6.94 0.00 3.51 0.00 0.05 89.50 12:33:01 AM all 0.35 0.00 0.08 0.00 0.04 99.53 12:33:01 AM 0 0.62 0.00 0.08 0.00 0.05 99.25 12:33:01 AM 1 0.08 0.00 0.07 0.00 0.03 99.82 12:33:01 AM CPU %user %nice %system %iowait %steal %idle 12:34:01 AM all 1.78 0.01 0.23 0.02 0.03 97.93 12:34:01 AM 0 0.60 0.00 0.12 0.02 0.03 99.23 12:34:01 AM 1 3.00 0.02 0.35 0.02 0.03 96.58 12:35:01 AM all 4.89 0.00 3.19 0.00 0.05 91.87 12:35:01 AM 0 2.19 0.00 0.26 0.00 0.05 97.50 12:35:01 AM 1 7.78 0.00 6.34 0.00 0.05 85.83 12:36:01 AM all 0.89 0.00 0.10 0.00 0.06 98.95 12:36:01 AM 0 0.30 0.00 0.10 0.00 0.08 99.52 12:36:01 AM 1 1.48 0.00 0.10 0.00 0.03 98.39 12:37:01 AM all 0.33 0.00 0.07 0.00 0.03 99.57 12:37:01 AM 0 0.17 0.00 0.10 0.00 0.03 99.70 12:37:01 AM 1 0.48 0.00 0.03 0.00 0.03 99.45 12:38:01 AM all 4.96 0.00 1.37 0.00 0.05 93.62 12:38:01 AM 0 9.59 0.00 2.76 0.00 0.06 87.59 12:38:01 AM 1 0.77 0.00 0.10 0.00 0.05 99.08 12:39:01 AM all 5.55 0.00 2.52 0.00 0.08 91.84 12:39:01 AM 0 9.57 0.00 5.02 0.00 0.07 85.34 12:39:01 AM 1 1.84 0.00 0.22 0.00 0.08 97.86 12:40:01 AM all 4.35 0.00 0.34 0.00 0.08 95.22 12:40:01 AM 0 3.31 0.00 0.30 0.00 0.08 96.31 12:40:01 AM 1 5.40 0.00 0.38 0.00 0.08 94.14 12:41:01 AM all 4.54 0.23 0.43 0.13 0.08 94.61 12:41:01 AM 0 4.09 0.00 0.30 0.02 0.07 95.52 12:41:01 AM 1 4.98 0.45 0.55 0.23 0.08 93.70 12:42:01 AM all 4.32 0.00 0.31 0.00 0.07 95.31 12:42:01 AM 0 2.89 0.00 0.22 0.00 0.07 96.82 12:42:01 AM 1 5.74 0.00 0.40 0.00 0.07 93.79 12:43:01 AM all 4.27 0.00 0.33 0.00 0.08 95.31 12:43:01 AM 0 1.54 0.00 0.15 0.00 0.07 98.24 12:43:01 AM 1 7.00 0.00 0.52 0.00 0.10 92.38 12:44:01 AM all 4.33 0.00 0.34 0.00 0.08 95.25 12:44:01 AM 0 3.90 0.00 0.33 0.00 0.05 95.72 12:44:01 AM 1 4.76 0.00 0.35 0.00 0.10 94.79 12:44:01 AM CPU %user %nice %system %iowait %steal %idle 12:45:01 AM all 6.27 0.00 0.39 0.00 0.07 93.27 12:45:01 AM 0 5.03 0.00 0.37 0.00 0.07 94.53 12:45:01 AM 1 7.51 0.00 0.42 0.00 0.07 92.00 12:46:01 AM all 2.25 0.00 0.20 0.00 0.08 97.48 12:46:01 AM 0 0.94 0.00 0.13 0.00 0.07 98.86 12:46:01 AM 1 3.56 0.00 0.27 0.00 0.08 96.09 12:47:01 AM all 0.22 0.00 0.08 0.00 0.04 99.67 12:47:01 AM 0 0.28 0.00 0.07 0.00 0.02 99.63 12:47:01 AM 1 0.15 0.00 0.08 0.00 0.07 99.70 12:48:01 AM all 0.48 0.00 0.08 0.00 0.05 99.40 12:48:01 AM 0 0.42 0.00 0.08 0.00 0.05 99.45 12:48:01 AM 1 0.53 0.00 0.07 0.00 0.05 99.35 12:49:01 AM all 0.33 0.00 0.06 0.00 0.03 99.57 12:49:01 AM 0 0.13 0.00 0.08 0.00 0.05 99.73 12:49:01 AM 1 0.53 0.00 0.03 0.00 0.02 99.42 12:50:01 AM all 0.36 0.00 0.09 0.00 0.03 99.52 12:50:01 AM 0 0.23 0.00 0.10 0.00 0.03 99.63 12:50:01 AM 1 0.48 0.00 0.08 0.00 0.03 99.40 12:51:01 AM all 0.30 0.00 0.09 0.00 0.03 99.57 12:51:01 AM 0 0.13 0.00 0.10 0.00 0.05 99.72 12:51:01 AM 1 0.47 0.00 0.08 0.00 0.02 99.43 12:52:01 AM all 0.33 0.00 0.06 0.00 0.03 99.58 12:52:01 AM 0 0.15 0.00 0.10 0.00 0.03 99.72 12:52:01 AM 1 0.52 0.00 0.02 0.00 0.02 99.45 12:53:01 AM all 5.24 0.00 3.73 0.00 0.06 90.98 12:53:01 AM 0 1.12 0.00 0.13 0.00 0.05 98.70 12:53:01 AM 1 9.64 0.00 7.57 0.00 0.07 82.72 12:54:01 AM all 4.64 0.00 1.93 0.00 0.07 93.36 12:54:01 AM 0 0.25 0.00 0.15 0.00 0.07 99.53 12:54:01 AM 1 9.41 0.00 3.86 0.00 0.07 86.65 12:55:01 AM all 0.38 0.00 0.08 0.00 0.05 99.50 12:55:01 AM 0 0.62 0.00 0.07 0.00 0.05 99.27 12:55:01 AM 1 0.13 0.00 0.08 0.00 0.05 99.73 12:55:01 AM CPU %user %nice %system %iowait %steal %idle 12:56:01 AM all 0.82 0.00 0.10 0.00 0.06 99.02 12:56:01 AM 0 0.65 0.00 0.07 0.00 0.05 99.23 12:56:01 AM 1 1.00 0.00 0.14 0.00 0.07 98.80 12:57:01 AM all 3.73 0.00 1.25 0.00 0.04 94.98 12:57:01 AM 0 2.99 0.00 0.20 0.00 0.03 96.77 12:57:01 AM 1 4.49 0.00 2.33 0.00 0.05 93.13 12:58:01 AM all 0.33 0.00 0.12 0.00 0.04 99.51 12:58:01 AM 0 0.40 0.00 0.10 0.00 0.03 99.47 12:58:01 AM 1 0.27 0.00 0.13 0.00 0.05 99.55 12:59:01 AM all 2.11 0.00 0.25 0.00 0.04 97.60 12:59:01 AM 0 0.82 0.00 0.15 0.00 0.03 99.00 12:59:01 AM 1 3.44 0.00 0.35 0.00 0.05 96.16 01:00:01 AM all 5.68 0.00 2.94 0.00 0.07 91.31 01:00:01 AM 0 0.70 0.00 0.15 0.00 0.05 99.10 01:00:01 AM 1 11.24 0.00 6.06 0.00 0.09 82.61 01:01:01 AM all 5.76 0.00 2.05 0.00 0.08 92.11 01:01:01 AM 0 0.70 0.00 0.23 0.00 0.08 98.98 01:01:01 AM 1 11.37 0.00 4.05 0.00 0.07 84.51 01:02:01 AM all 5.46 0.00 2.97 0.00 0.08 91.49 01:02:01 AM 0 6.66 0.00 3.59 0.00 0.09 89.67 01:02:01 AM 1 4.29 0.00 2.37 0.00 0.07 93.27 01:03:01 AM all 0.90 0.00 0.11 0.00 0.03 98.96 01:03:01 AM 0 1.58 0.00 0.12 0.00 0.02 98.29 01:03:01 AM 1 0.22 0.00 0.10 0.00 0.05 99.63 01:04:01 AM all 0.38 0.00 0.13 0.00 0.05 99.45 01:04:01 AM 0 0.50 0.00 0.13 0.00 0.05 99.31 01:04:01 AM 1 0.25 0.00 0.12 0.00 0.05 99.58 01:05:01 AM all 4.15 0.00 1.36 0.00 0.05 94.43 01:05:01 AM 0 7.01 0.00 2.61 0.00 0.05 90.33 01:05:01 AM 1 1.47 0.00 0.19 0.00 0.05 98.30 01:06:01 AM all 0.33 0.00 0.10 0.00 0.06 99.51 01:06:01 AM 0 0.15 0.00 0.07 0.00 0.07 99.72 01:06:01 AM 1 0.52 0.00 0.13 0.00 0.05 99.30 01:06:01 AM CPU %user %nice %system %iowait %steal %idle 01:07:01 AM all 1.33 0.00 0.08 0.00 0.04 98.54 01:07:01 AM 0 1.99 0.00 0.13 0.00 0.05 97.83 01:07:01 AM 1 0.68 0.00 0.03 0.00 0.03 99.25 01:08:01 AM all 5.80 0.00 2.48 0.00 0.07 91.66 01:08:01 AM 0 0.73 0.00 0.18 0.00 0.07 99.01 01:08:01 AM 1 11.32 0.00 4.98 0.00 0.07 83.63 01:09:01 AM all 4.31 0.00 0.33 1.58 0.08 93.69 01:09:01 AM 0 3.89 0.00 0.28 0.00 0.07 95.76 01:09:01 AM 1 4.73 0.00 0.38 3.16 0.10 91.63 01:10:01 AM all 4.26 0.00 0.31 0.00 0.07 95.37 01:10:01 AM 0 1.55 0.00 0.17 0.00 0.05 98.23 01:10:01 AM 1 6.96 0.00 0.45 0.00 0.08 92.51 01:11:01 AM all 4.30 0.00 0.32 0.00 0.08 95.30 01:11:01 AM 0 1.07 0.00 0.15 0.00 0.07 98.71 01:11:01 AM 1 7.54 0.00 0.48 0.00 0.08 91.89 01:12:01 AM all 4.30 0.00 0.32 0.00 0.07 95.32 01:12:01 AM 0 5.64 0.00 0.38 0.00 0.07 93.91 01:12:01 AM 1 2.96 0.00 0.25 0.00 0.07 96.72 01:13:01 AM all 4.40 0.01 0.32 0.00 0.09 95.19 01:13:01 AM 0 2.79 0.00 0.17 0.00 0.08 96.96 01:13:01 AM 1 6.00 0.02 0.47 0.00 0.10 93.42 01:14:01 AM all 4.36 0.00 0.34 0.00 0.08 95.21 01:14:01 AM 0 4.90 0.00 0.37 0.00 0.08 94.65 01:14:01 AM 1 3.83 0.00 0.32 0.00 0.08 95.77 01:15:01 AM all 4.25 0.00 0.25 0.00 0.05 95.45 01:15:01 AM 0 4.85 0.00 0.27 0.00 0.05 94.83 01:15:01 AM 1 3.66 0.00 0.23 0.00 0.05 96.06 01:16:01 AM all 0.31 0.00 0.05 0.00 0.04 99.60 01:16:01 AM 0 0.05 0.00 0.03 0.00 0.02 99.90 01:16:01 AM 1 0.57 0.00 0.07 0.00 0.07 99.30 01:17:01 AM all 0.46 0.00 0.08 0.00 0.05 99.42 01:17:01 AM 0 0.35 0.00 0.08 0.00 0.05 99.52 01:17:01 AM 1 0.57 0.00 0.07 0.00 0.05 99.32 01:17:01 AM CPU %user %nice %system %iowait %steal %idle 01:18:02 AM all 0.33 0.00 0.05 0.00 0.05 99.57 01:18:02 AM 0 0.07 0.00 0.07 0.00 0.03 99.83 01:18:02 AM 1 0.60 0.00 0.03 0.00 0.07 99.30 01:19:01 AM all 0.33 0.00 0.07 0.00 0.05 99.55 01:19:01 AM 0 0.10 0.00 0.08 0.00 0.05 99.76 01:19:01 AM 1 0.56 0.00 0.05 0.00 0.05 99.34 01:20:01 AM all 0.48 0.00 0.07 0.00 0.04 99.42 01:20:01 AM 0 0.38 0.00 0.08 0.00 0.03 99.50 01:20:01 AM 1 0.57 0.00 0.05 0.00 0.05 99.33 01:21:01 AM all 0.28 0.00 0.05 0.00 0.04 99.63 01:21:01 AM 0 0.38 0.00 0.03 0.00 0.03 99.55 01:21:01 AM 1 0.17 0.00 0.07 0.00 0.05 99.72 01:22:01 AM all 4.48 0.00 3.51 0.00 0.05 91.96 01:22:01 AM 0 1.24 0.00 0.18 0.00 0.05 98.53 01:22:01 AM 1 7.90 0.00 7.04 0.00 0.05 85.00 01:23:01 AM all 5.50 0.00 2.02 0.00 0.06 92.41 01:23:01 AM 0 0.23 0.00 0.12 0.00 0.07 99.58 01:23:01 AM 1 11.33 0.00 4.13 0.00 0.06 84.48 01:24:01 AM all 3.88 0.00 1.53 0.00 0.05 94.54 01:24:01 AM 0 0.67 0.00 0.08 0.00 0.07 99.18 01:24:01 AM 1 7.31 0.00 3.08 0.00 0.04 89.58 01:25:01 AM all 0.31 0.00 0.08 0.00 0.05 99.57 01:25:01 AM 0 0.47 0.00 0.07 0.00 0.03 99.43 01:25:01 AM 1 0.15 0.00 0.08 0.00 0.07 99.70 01:26:01 AM all 1.71 0.00 0.17 0.00 0.06 98.06 01:26:01 AM 0 0.15 0.00 0.10 0.00 0.07 99.68 01:26:01 AM 1 3.33 0.00 0.24 0.00 0.05 96.38 01:27:01 AM all 2.20 0.00 1.78 0.00 0.04 95.98 01:27:01 AM 0 0.15 0.00 0.13 0.00 0.05 99.67 01:27:01 AM 1 4.31 0.00 3.48 0.00 0.03 92.17 01:28:01 AM all 0.34 0.00 0.07 0.00 0.04 99.55 01:28:01 AM 0 0.12 0.00 0.08 0.00 0.05 99.75 01:28:01 AM 1 0.57 0.00 0.05 0.00 0.03 99.35 01:28:01 AM CPU %user %nice %system %iowait %steal %idle 01:29:01 AM all 2.98 0.00 1.20 0.00 0.05 95.77 01:29:01 AM 0 0.20 0.00 0.15 0.00 0.07 99.58 01:29:01 AM 1 5.94 0.00 2.30 0.00 0.04 91.72 01:30:01 AM all 1.42 0.00 0.14 0.00 0.06 98.38 01:30:01 AM 0 0.47 0.00 0.05 0.00 0.05 99.43 01:30:01 AM 1 2.39 0.00 0.24 0.00 0.07 97.31 01:31:01 AM all 0.37 0.00 0.06 0.00 0.04 99.53 01:31:01 AM 0 0.60 0.00 0.03 0.00 0.03 99.33 01:31:01 AM 1 0.13 0.00 0.08 0.00 0.05 99.73 01:32:01 AM all 4.17 0.00 1.87 0.00 0.06 93.90 01:32:01 AM 0 0.55 0.00 0.12 0.00 0.05 99.28 01:32:01 AM 1 8.07 0.00 3.77 0.00 0.07 88.09 01:33:01 AM all 3.27 0.00 1.29 0.00 0.04 95.40 01:33:01 AM 0 1.24 0.00 0.15 0.00 0.05 98.55 01:33:01 AM 1 5.41 0.00 2.48 0.00 0.04 92.07 01:34:01 AM all 0.31 0.00 0.08 0.00 0.04 99.57 01:34:01 AM 0 0.55 0.00 0.07 0.00 0.05 99.33 01:34:01 AM 1 0.07 0.00 0.10 0.00 0.03 99.80 01:35:01 AM all 1.93 0.00 0.20 0.00 0.05 97.81 01:35:01 AM 0 0.62 0.00 0.08 0.00 0.05 99.25 01:35:01 AM 1 3.29 0.00 0.33 0.00 0.05 96.33 01:36:01 AM all 5.59 0.00 2.91 0.00 0.05 91.45 01:36:01 AM 0 1.93 0.00 0.24 0.00 0.05 97.78 01:36:01 AM 1 9.52 0.00 5.78 0.00 0.05 84.64 01:37:01 AM all 4.88 0.00 1.06 0.00 0.08 93.98 01:37:01 AM 0 2.11 0.00 0.22 0.00 0.08 97.59 01:37:01 AM 1 7.73 0.00 1.93 0.00 0.09 90.26 01:38:01 AM all 4.18 0.00 0.31 0.00 0.08 95.43 01:38:01 AM 0 3.85 0.00 0.30 0.00 0.07 95.78 01:38:01 AM 1 4.51 0.00 0.32 0.00 0.08 95.09 01:39:01 AM all 4.26 0.00 0.30 0.00 0.08 95.36 01:39:01 AM 0 5.00 0.00 0.37 0.00 0.08 94.55 01:39:01 AM 1 3.53 0.00 0.23 0.00 0.07 96.17 01:39:01 AM CPU %user %nice %system %iowait %steal %idle 01:40:01 AM all 4.28 0.00 0.29 0.01 0.08 95.34 01:40:01 AM 0 3.74 0.00 0.25 0.02 0.08 95.90 01:40:01 AM 1 4.82 0.00 0.33 0.00 0.08 94.77 01:41:01 AM all 4.33 0.00 0.33 0.00 0.08 95.26 01:41:01 AM 0 2.02 0.00 0.20 0.00 0.07 97.71 01:41:01 AM 1 6.64 0.00 0.47 0.00 0.08 92.81 01:42:01 AM all 4.30 0.00 0.33 0.00 0.08 95.28 01:42:01 AM 0 3.03 0.00 0.25 0.00 0.07 96.66 01:42:01 AM 1 5.58 0.00 0.42 0.00 0.10 93.90 01:43:01 AM all 5.74 0.00 0.33 0.00 0.06 93.87 01:43:01 AM 0 5.72 0.00 0.28 0.00 0.05 93.95 01:43:01 AM 1 5.77 0.00 0.37 0.00 0.07 93.80 01:44:01 AM all 0.42 0.00 0.10 0.00 0.04 99.44 01:44:01 AM 0 0.17 0.00 0.12 0.00 0.05 99.67 01:44:01 AM 1 0.67 0.00 0.08 0.00 0.03 99.22 01:45:01 AM all 0.13 0.00 0.08 0.00 0.04 99.76 01:45:01 AM 0 0.12 0.00 0.10 0.00 0.05 99.73 01:45:01 AM 1 0.13 0.00 0.05 0.00 0.03 99.78 01:46:01 AM all 0.49 0.17 0.10 0.00 0.03 99.21 01:46:01 AM 0 0.43 0.33 0.17 0.00 0.03 99.03 01:46:01 AM 1 0.55 0.00 0.03 0.00 0.03 99.38 01:47:01 AM all 0.33 0.00 0.05 0.00 0.04 99.58 01:47:01 AM 0 0.12 0.00 0.08 0.00 0.05 99.75 01:47:01 AM 1 0.53 0.00 0.02 0.00 0.03 99.42 01:48:01 AM all 0.48 0.00 0.06 0.00 0.05 99.42 01:48:01 AM 0 0.40 0.00 0.07 0.00 0.05 99.48 01:48:01 AM 1 0.55 0.00 0.05 0.00 0.05 99.35 01:49:01 AM all 0.30 0.00 0.05 0.00 0.03 99.62 01:49:01 AM 0 0.15 0.00 0.07 0.00 0.05 99.73 01:49:01 AM 1 0.45 0.00 0.03 0.00 0.02 99.50 01:50:01 AM all 2.64 0.00 0.67 0.00 0.05 96.64 01:50:01 AM 0 4.66 0.00 1.25 0.00 0.07 94.02 01:50:01 AM 1 0.67 0.00 0.10 0.00 0.03 99.20 01:50:01 AM CPU %user %nice %system %iowait %steal %idle 01:51:01 AM all 5.22 0.00 4.38 0.01 0.05 90.35 01:51:01 AM 0 6.39 0.00 7.55 0.00 0.05 86.01 01:51:01 AM 1 4.04 0.00 1.18 0.02 0.05 94.71 01:52:01 AM all 2.10 0.00 1.26 0.00 0.05 96.59 01:52:01 AM 0 0.47 0.00 0.08 0.00 0.05 99.40 01:52:01 AM 1 3.78 0.00 2.46 0.00 0.05 93.71 01:53:01 AM all 0.33 0.00 0.06 0.00 0.03 99.58 01:53:01 AM 0 0.18 0.00 0.03 0.00 0.02 99.77 01:53:01 AM 1 0.47 0.00 0.08 0.00 0.05 99.40 01:54:01 AM all 3.30 0.00 1.36 0.00 0.05 95.28 01:54:01 AM 0 0.15 0.00 0.13 0.00 0.05 99.67 01:54:01 AM 1 6.67 0.00 2.68 0.00 0.05 90.60 01:55:01 AM all 5.40 0.00 2.05 0.00 0.08 92.47 01:55:01 AM 0 0.17 0.00 0.17 0.00 0.08 99.58 01:55:01 AM 1 11.20 0.00 4.15 0.00 0.07 84.59 01:56:01 AM all 5.58 0.00 2.92 0.00 0.06 91.44 01:56:01 AM 0 11.28 0.00 6.05 0.00 0.06 82.62 01:56:01 AM 1 0.42 0.00 0.10 0.00 0.07 99.41 01:57:01 AM all 4.27 0.00 1.31 0.00 0.06 94.36 01:57:01 AM 0 8.25 0.00 2.61 0.00 0.04 89.10 01:57:01 AM 1 0.63 0.00 0.12 0.00 0.08 99.17 01:58:01 AM all 0.34 0.00 0.08 0.00 0.06 99.52 01:58:01 AM 0 0.55 0.00 0.07 0.00 0.05 99.33 01:58:01 AM 1 0.13 0.00 0.10 0.00 0.07 99.70 01:59:01 AM all 1.46 0.00 0.16 0.00 0.03 98.35 01:59:01 AM 0 1.80 0.00 0.10 0.00 0.03 98.06 01:59:01 AM 1 1.11 0.00 0.22 0.00 0.03 98.63 02:00:01 AM all 5.56 0.00 2.48 0.00 0.08 91.88 02:00:01 AM 0 0.80 0.00 0.17 0.00 0.08 98.95 02:00:01 AM 1 10.76 0.00 5.00 0.00 0.07 84.17 02:01:01 AM all 4.35 0.00 0.30 0.00 0.07 95.28 02:01:01 AM 0 3.18 0.00 0.25 0.00 0.05 96.52 02:01:01 AM 1 5.53 0.00 0.35 0.00 0.08 94.03 02:01:01 AM CPU %user %nice %system %iowait %steal %idle 02:02:01 AM all 4.31 0.00 0.32 0.00 0.08 95.29 02:02:01 AM 0 1.74 0.00 0.18 0.00 0.08 97.99 02:02:01 AM 1 6.89 0.00 0.45 0.00 0.08 92.58 02:03:01 AM all 4.31 0.00 0.33 0.00 0.08 95.29 02:03:01 AM 0 3.01 0.00 0.30 0.00 0.07 96.63 02:03:01 AM 1 5.62 0.00 0.35 0.00 0.08 93.95 02:04:01 AM all 4.30 0.00 0.33 0.00 0.08 95.29 02:04:01 AM 0 1.92 0.00 0.18 0.00 0.07 97.83 02:04:01 AM 1 6.69 0.00 0.47 0.00 0.10 92.74 02:05:01 AM all 4.34 0.00 0.33 0.00 0.09 95.24 02:05:01 AM 0 5.12 0.00 0.40 0.00 0.08 94.40 02:05:01 AM 1 3.56 0.00 0.25 0.00 0.10 96.09 02:06:01 AM all 5.87 0.00 0.36 0.00 0.08 93.69 02:06:01 AM 0 6.66 0.00 0.37 0.00 0.08 92.89 02:06:01 AM 1 5.08 0.00 0.35 0.00 0.08 94.48 02:07:01 AM all 2.50 0.00 0.18 0.00 0.05 97.27 02:07:01 AM 0 1.12 0.00 0.15 0.00 0.05 98.68 02:07:01 AM 1 3.88 0.00 0.22 0.00 0.05 95.86 02:08:01 AM all 0.29 0.00 0.05 0.00 0.05 99.61 02:08:01 AM 0 0.48 0.00 0.03 0.00 0.03 99.45 02:08:01 AM 1 0.10 0.00 0.07 0.00 0.07 99.77 02:09:01 AM all 0.38 0.00 0.07 0.00 0.03 99.52 02:09:01 AM 0 0.60 0.00 0.05 0.00 0.02 99.33 02:09:01 AM 1 0.15 0.00 0.08 0.00 0.05 99.72 02:10:01 AM all 0.25 0.00 0.07 0.00 0.04 99.64 02:10:01 AM 0 0.37 0.00 0.05 0.00 0.02 99.57 02:10:01 AM 1 0.13 0.00 0.08 0.00 0.07 99.72 02:11:01 AM all 0.33 0.00 0.04 0.00 0.04 99.58 02:11:01 AM 0 0.58 0.00 0.03 0.00 0.05 99.33 02:11:01 AM 1 0.08 0.00 0.05 0.00 0.03 99.83 02:12:01 AM all 0.48 0.00 0.06 0.00 0.04 99.42 02:12:01 AM 0 0.58 0.00 0.07 0.00 0.03 99.32 02:12:01 AM 1 0.38 0.00 0.05 0.00 0.05 99.52 02:12:01 AM CPU %user %nice %system %iowait %steal %idle 02:13:01 AM all 0.29 0.00 0.06 0.00 0.03 99.62 02:13:01 AM 0 0.08 0.00 0.05 0.00 0.02 99.85 02:13:01 AM 1 0.50 0.00 0.07 0.00 0.03 99.40 02:14:01 AM all 4.92 0.00 3.73 0.03 0.06 91.26 02:14:01 AM 0 3.10 0.00 0.38 0.05 0.05 96.42 02:14:01 AM 1 6.77 0.00 7.17 0.00 0.07 85.99 02:15:01 AM all 4.63 0.00 1.90 0.00 0.07 93.40 02:15:01 AM 0 0.33 0.00 0.08 0.00 0.05 99.53 02:15:01 AM 1 9.29 0.00 3.88 0.00 0.09 86.74 02:16:01 AM all 0.11 0.00 0.08 0.00 0.03 99.78 02:16:01 AM 0 0.10 0.00 0.07 0.00 0.02 99.82 02:16:01 AM 1 0.12 0.00 0.10 0.00 0.03 99.75 02:17:01 AM all 0.69 0.00 0.13 0.00 0.04 99.15 02:17:01 AM 0 1.19 0.00 0.15 0.00 0.05 98.60 02:17:01 AM 1 0.18 0.00 0.10 0.00 0.03 99.68 02:18:01 AM all 5.02 0.00 2.13 0.00 0.07 92.78 02:18:01 AM 0 10.46 0.00 4.32 0.00 0.07 85.15 02:18:01 AM 1 0.17 0.00 0.17 0.00 0.07 99.60 02:19:01 AM all 5.14 0.00 2.51 0.00 0.09 92.27 02:19:01 AM 0 10.00 0.00 5.05 0.00 0.13 84.82 02:19:01 AM 1 0.72 0.00 0.20 0.00 0.05 99.03 02:20:01 AM all 5.16 0.00 2.35 0.00 0.06 92.43 02:20:01 AM 0 3.87 0.00 1.03 0.00 0.05 95.05 02:20:01 AM 1 6.47 0.00 3.69 0.00 0.07 89.77 02:21:01 AM all 2.18 0.00 1.02 0.00 0.04 96.76 02:21:01 AM 0 0.22 0.00 0.13 0.00 0.05 99.60 02:21:01 AM 1 4.20 0.00 1.92 0.00 0.03 93.85 02:22:01 AM all 0.12 0.00 0.07 0.00 0.03 99.79 02:22:01 AM 0 0.15 0.00 0.07 0.00 0.03 99.75 02:22:01 AM 1 0.08 0.00 0.07 0.00 0.02 99.83 02:23:01 AM all 3.70 0.00 1.22 0.00 0.05 95.03 02:23:01 AM 0 4.37 0.00 2.22 0.00 0.05 93.36 02:23:01 AM 1 3.05 0.00 0.24 0.00 0.05 96.66 02:23:01 AM CPU %user %nice %system %iowait %steal %idle 02:24:01 AM all 4.98 0.00 1.53 0.00 0.06 93.43 02:24:01 AM 0 7.48 0.00 2.84 0.00 0.07 89.61 02:24:01 AM 1 2.54 0.00 0.25 0.00 0.05 97.15 02:25:01 AM all 4.43 0.00 0.36 0.00 0.07 95.14 02:25:01 AM 0 5.62 0.00 0.43 0.00 0.07 93.88 02:25:01 AM 1 3.24 0.00 0.28 0.00 0.07 96.41 02:26:01 AM all 4.43 0.00 0.35 0.00 0.08 95.14 02:26:01 AM 0 3.86 0.00 0.32 0.00 0.08 95.74 02:26:01 AM 1 5.00 0.00 0.38 0.00 0.08 94.54 02:27:01 AM all 4.41 0.00 0.31 0.00 0.08 95.20 02:27:01 AM 0 2.61 0.00 0.20 0.00 0.07 97.12 02:27:01 AM 1 6.22 0.00 0.42 0.00 0.08 93.28 02:28:02 AM all 4.28 0.00 0.35 0.00 0.07 95.30 02:28:02 AM 0 2.31 0.00 0.23 0.00 0.05 97.41 02:28:02 AM 1 6.25 0.00 0.47 0.00 0.08 93.20 02:29:01 AM all 3.69 0.00 0.29 0.00 0.07 95.95 02:29:01 AM 0 2.97 0.00 0.22 0.00 0.07 96.74 02:29:01 AM 1 4.40 0.00 0.36 0.00 0.07 95.17 02:30:01 AM all 5.96 0.00 0.38 0.00 0.08 93.58 02:30:01 AM 0 3.40 0.00 0.28 0.00 0.07 96.25 02:30:01 AM 1 8.51 0.00 0.49 0.00 0.08 90.92 02:31:01 AM all 0.94 0.00 0.13 0.00 0.07 98.87 02:31:01 AM 0 0.82 0.00 0.08 0.00 0.05 99.05 02:31:01 AM 1 1.05 0.00 0.17 0.00 0.08 98.70 02:32:01 AM all 0.96 0.00 0.13 0.00 0.08 98.84 02:32:01 AM 0 0.84 0.00 0.10 0.00 0.07 99.00 02:32:01 AM 1 1.09 0.00 0.15 0.00 0.08 98.68 02:33:01 AM all 0.95 0.00 0.11 0.00 0.08 98.86 02:33:01 AM 0 1.19 0.00 0.08 0.00 0.08 98.65 02:33:01 AM 1 0.72 0.00 0.13 0.00 0.07 99.08 02:34:01 AM all 0.96 0.00 0.13 0.00 0.06 98.86 02:34:01 AM 0 0.85 0.00 0.12 0.00 0.05 98.98 02:34:01 AM 1 1.07 0.00 0.13 0.00 0.07 98.73 02:34:01 AM CPU %user %nice %system %iowait %steal %idle 02:35:01 AM all 0.89 0.00 0.13 0.00 0.05 98.92 02:35:01 AM 0 0.55 0.00 0.10 0.00 0.03 99.32 02:35:01 AM 1 1.24 0.00 0.17 0.00 0.07 98.53 02:36:01 AM all 3.37 0.00 0.93 0.00 0.07 95.63 02:36:01 AM 0 4.32 0.00 0.37 0.00 0.05 95.25 02:36:01 AM 1 2.42 0.00 1.48 0.00 0.08 96.01 02:37:01 AM all 5.41 0.00 3.30 0.00 0.08 91.21 02:37:01 AM 0 0.20 0.00 0.15 0.00 0.07 99.58 02:37:01 AM 1 11.22 0.00 6.81 0.00 0.09 81.88 02:38:01 AM all 5.60 0.00 2.26 0.00 0.06 92.08 02:38:01 AM 0 8.11 0.00 2.61 0.00 0.07 89.20 02:38:01 AM 1 3.20 0.00 1.93 0.00 0.05 94.82 02:39:01 AM all 0.84 0.00 0.13 0.01 0.05 98.97 02:39:01 AM 0 0.99 0.00 0.13 0.00 0.03 98.84 02:39:01 AM 1 0.69 0.00 0.13 0.02 0.07 99.10 02:40:01 AM all 6.72 0.00 2.11 0.00 0.07 91.11 02:40:01 AM 0 10.31 0.00 4.09 0.00 0.09 85.51 02:40:01 AM 1 3.35 0.00 0.25 0.00 0.05 96.34 02:41:01 AM all 5.67 0.00 2.51 0.00 0.08 91.74 02:41:01 AM 0 9.60 0.00 4.90 0.00 0.11 85.40 02:41:01 AM 1 1.95 0.00 0.25 0.00 0.05 97.75 02:42:01 AM all 4.45 0.00 0.34 0.00 0.06 95.15 02:42:01 AM 0 4.62 0.00 0.32 0.00 0.05 95.02 02:42:01 AM 1 4.28 0.00 0.37 0.00 0.07 95.29 02:43:01 AM all 4.45 0.00 0.33 0.00 0.07 95.16 02:43:01 AM 0 4.08 0.00 0.28 0.00 0.05 95.58 02:43:01 AM 1 4.82 0.00 0.37 0.00 0.08 94.73 02:44:01 AM all 4.57 0.00 0.33 0.00 0.06 95.03 02:44:01 AM 0 4.90 0.00 0.37 0.00 0.07 94.66 02:44:01 AM 1 4.25 0.00 0.30 0.00 0.05 95.40 02:45:01 AM all 4.47 0.00 0.34 0.00 0.08 95.11 02:45:01 AM 0 3.38 0.00 0.30 0.00 0.07 96.26 02:45:01 AM 1 5.57 0.00 0.38 0.00 0.08 93.96 02:45:01 AM CPU %user %nice %system %iowait %steal %idle 02:46:01 AM all 4.54 0.00 0.35 0.00 0.07 95.04 02:46:01 AM 0 4.92 0.00 0.38 0.00 0.05 94.65 02:46:01 AM 1 4.16 0.00 0.32 0.00 0.08 95.44 02:47:01 AM all 6.83 0.00 0.43 0.00 0.08 92.67 02:47:01 AM 0 4.84 0.00 0.39 0.00 0.07 94.71 02:47:01 AM 1 8.82 0.00 0.47 0.00 0.08 90.63 02:48:01 AM all 0.89 0.00 0.13 0.00 0.05 98.93 02:48:01 AM 0 0.52 0.00 0.10 0.00 0.05 99.33 02:48:01 AM 1 1.27 0.00 0.15 0.00 0.05 98.53 02:49:01 AM all 0.91 0.00 0.13 0.00 0.05 98.91 02:49:01 AM 0 0.95 0.00 0.13 0.00 0.05 98.86 02:49:01 AM 1 0.87 0.00 0.13 0.00 0.05 98.95 02:50:01 AM all 1.04 0.00 0.13 0.00 0.05 98.77 02:50:01 AM 0 0.70 0.00 0.15 0.00 0.03 99.11 02:50:01 AM 1 1.39 0.00 0.12 0.00 0.07 98.43 02:51:01 AM all 0.93 0.00 0.13 0.00 0.04 98.90 02:51:01 AM 0 0.62 0.00 0.12 0.00 0.05 99.21 02:51:01 AM 1 1.24 0.00 0.15 0.00 0.03 98.58 02:52:01 AM all 0.95 0.00 0.13 0.00 0.04 98.88 02:52:01 AM 0 0.77 0.00 0.15 0.00 0.05 99.03 02:52:01 AM 1 1.14 0.00 0.10 0.00 0.03 98.73 02:53:01 AM all 2.22 0.00 0.18 0.00 0.06 97.54 02:53:01 AM 0 0.64 0.00 0.15 0.00 0.05 99.16 02:53:01 AM 1 3.81 0.00 0.20 0.00 0.07 95.92 02:54:01 AM all 6.02 0.00 3.52 0.00 0.07 90.39 02:54:01 AM 0 0.75 0.00 0.13 0.00 0.05 99.06 02:54:01 AM 1 11.86 0.00 7.26 0.00 0.09 80.79 02:55:01 AM all 6.57 0.00 2.73 0.00 0.11 90.58 02:55:01 AM 0 11.94 0.00 5.44 0.00 0.15 82.47 02:55:01 AM 1 1.65 0.00 0.25 0.00 0.08 98.01 02:56:01 AM all 4.47 0.00 0.32 0.00 0.08 95.14 02:56:01 AM 0 3.55 0.00 0.27 0.00 0.07 96.12 02:56:01 AM 1 5.38 0.00 0.37 0.00 0.08 94.16 02:56:01 AM CPU %user %nice %system %iowait %steal %idle 02:57:01 AM all 4.52 0.00 0.36 0.00 0.08 95.04 02:57:01 AM 0 3.80 0.00 0.33 0.00 0.07 95.80 02:57:01 AM 1 5.25 0.00 0.38 0.00 0.08 94.28 02:58:01 AM all 4.48 0.00 0.34 0.00 0.07 95.11 02:58:01 AM 0 3.98 0.00 0.28 0.00 0.07 95.67 02:58:01 AM 1 4.98 0.00 0.40 0.00 0.07 94.55 02:59:01 AM all 4.50 0.00 0.36 0.00 0.05 95.09 02:59:01 AM 0 3.55 0.00 0.30 0.00 0.05 96.10 02:59:01 AM 1 5.46 0.00 0.42 0.00 0.05 94.08 03:00:01 AM all 4.48 0.00 0.33 0.00 0.08 95.11 03:00:01 AM 0 3.58 0.00 0.27 0.00 0.05 96.10 03:00:01 AM 1 5.38 0.00 0.40 0.00 0.10 94.12 03:01:01 AM all 4.45 0.00 0.33 0.00 0.04 95.18 03:01:01 AM 0 5.32 0.00 0.42 0.00 0.05 94.21 03:01:01 AM 1 3.58 0.00 0.25 0.00 0.03 96.14 03:02:01 AM all 4.61 0.00 0.28 0.00 0.08 95.04 03:02:01 AM 0 3.42 0.00 0.22 0.00 0.07 96.29 03:02:01 AM 1 5.79 0.00 0.33 0.00 0.08 93.79 03:03:01 AM all 1.02 0.00 0.13 0.01 0.04 98.81 03:03:01 AM 0 1.27 0.00 0.13 0.02 0.05 98.53 03:03:01 AM 1 0.77 0.00 0.12 0.00 0.03 99.08 03:04:01 AM all 0.80 0.00 0.14 0.00 0.06 99.00 03:04:01 AM 0 0.40 0.00 0.13 0.00 0.05 99.41 03:04:01 AM 1 1.20 0.00 0.15 0.00 0.07 98.58 03:05:01 AM all 0.97 0.00 0.10 0.00 0.05 98.88 03:05:01 AM 0 0.95 0.00 0.10 0.00 0.03 98.91 03:05:01 AM 1 0.99 0.00 0.10 0.00 0.07 98.85 03:06:01 AM all 1.07 0.00 0.12 0.00 0.06 98.75 03:06:01 AM 0 0.74 0.00 0.10 0.00 0.05 99.11 03:06:01 AM 1 1.40 0.00 0.13 0.00 0.07 98.40 03:07:01 AM all 0.96 0.00 0.12 0.00 0.06 98.86 03:07:01 AM 0 0.79 0.00 0.08 0.00 0.05 99.08 03:07:01 AM 1 1.14 0.00 0.15 0.00 0.07 98.65 03:07:01 AM CPU %user %nice %system %iowait %steal %idle 03:08:01 AM all 5.32 0.00 3.43 0.00 0.06 91.19 03:08:01 AM 0 8.44 0.00 6.88 0.00 0.07 84.61 03:08:01 AM 1 2.41 0.00 0.22 0.00 0.05 97.33 03:09:01 AM all 6.20 0.00 1.81 0.00 0.09 91.90 03:09:01 AM 0 12.11 0.00 3.64 0.00 0.11 84.14 03:09:01 AM 1 0.87 0.00 0.17 0.00 0.07 98.90 03:10:01 AM all 5.15 0.00 1.29 0.00 0.07 93.49 03:10:01 AM 0 4.28 0.00 2.16 0.00 0.07 93.49 03:10:01 AM 1 6.00 0.00 0.43 0.00 0.07 93.49 03:11:01 AM all 4.43 0.00 0.33 0.00 0.06 95.17 03:11:01 AM 0 1.61 0.00 0.23 0.00 0.05 98.11 03:11:01 AM 1 7.26 0.00 0.44 0.00 0.07 92.24 03:12:01 AM all 4.48 0.00 0.32 0.00 0.08 95.13 03:12:01 AM 0 2.12 0.00 0.18 0.00 0.07 97.63 03:12:01 AM 1 6.84 0.00 0.45 0.00 0.08 92.63 03:13:01 AM all 4.49 0.00 0.33 0.00 0.06 95.12 03:13:01 AM 0 2.49 0.00 0.23 0.00 0.05 97.22 03:13:01 AM 1 6.49 0.00 0.42 0.00 0.07 93.02 03:14:01 AM all 4.78 0.00 0.38 0.00 0.06 94.77 03:14:01 AM 0 1.89 0.00 0.20 0.00 0.05 97.86 03:14:01 AM 1 7.68 0.00 0.57 0.00 0.07 91.69 03:15:01 AM all 4.56 0.00 0.35 0.00 0.07 95.02 03:15:01 AM 0 3.20 0.00 0.27 0.00 0.05 96.49 03:15:01 AM 1 5.92 0.00 0.43 0.00 0.08 93.56 03:16:01 AM all 5.96 0.00 0.32 0.00 0.05 93.67 03:16:01 AM 0 4.01 0.00 0.27 0.00 0.05 95.67 03:16:01 AM 1 7.91 0.00 0.37 0.00 0.05 91.67 03:17:01 AM all 0.89 0.00 0.12 0.00 0.06 98.93 03:17:01 AM 0 0.53 0.00 0.10 0.00 0.03 99.33 03:17:01 AM 1 1.25 0.00 0.13 0.00 0.08 98.53 03:18:01 AM all 0.95 0.00 0.10 0.00 0.07 98.88 03:18:01 AM 0 0.63 0.00 0.08 0.00 0.05 99.23 03:18:01 AM 1 1.27 0.00 0.12 0.00 0.08 98.53 03:18:01 AM CPU %user %nice %system %iowait %steal %idle 03:19:01 AM all 0.94 0.00 0.13 0.00 0.05 98.89 03:19:01 AM 0 1.00 0.00 0.10 0.00 0.05 98.85 03:19:01 AM 1 0.87 0.00 0.15 0.00 0.05 98.93 03:20:01 AM all 0.97 0.00 0.12 0.00 0.08 98.84 03:20:01 AM 0 1.02 0.00 0.13 0.00 0.08 98.76 03:20:01 AM 1 0.92 0.00 0.10 0.00 0.07 98.91 03:21:01 AM all 0.97 0.00 0.13 0.00 0.06 98.84 03:21:01 AM 0 0.74 0.00 0.12 0.00 0.05 99.10 03:21:01 AM 1 1.20 0.00 0.15 0.00 0.07 98.58 03:22:01 AM all 3.54 0.00 1.13 0.00 0.08 95.26 03:22:01 AM 0 1.62 0.00 0.22 0.00 0.07 98.09 03:22:01 AM 1 5.52 0.00 2.07 0.00 0.09 92.33 03:23:01 AM all 5.54 0.00 2.56 0.00 0.07 91.82 03:23:01 AM 0 0.77 0.00 0.17 0.00 0.07 99.00 03:23:01 AM 1 10.88 0.00 5.24 0.00 0.07 83.81 03:24:01 AM all 6.20 0.17 2.66 0.00 0.11 90.85 03:24:01 AM 0 2.21 0.33 0.30 0.00 0.10 97.05 03:24:01 AM 1 10.45 0.00 5.18 0.00 0.12 84.25 03:25:01 AM all 4.97 0.00 0.37 0.00 0.07 94.59 03:25:01 AM 0 5.84 0.00 0.37 0.00 0.05 93.74 03:25:01 AM 1 4.10 0.00 0.37 0.00 0.08 95.45 03:26:01 AM all 4.49 0.00 0.38 0.00 0.07 95.06 03:26:01 AM 0 5.54 0.00 0.45 0.00 0.08 93.92 03:26:01 AM 1 3.44 0.00 0.32 0.00 0.05 96.19 03:27:01 AM all 4.43 0.00 0.36 0.00 0.07 95.14 03:27:01 AM 0 7.09 0.00 0.47 0.00 0.07 92.37 03:27:01 AM 1 1.77 0.00 0.25 0.00 0.07 97.91 03:28:01 AM all 4.52 0.00 0.33 0.00 0.06 95.09 03:28:01 AM 0 5.65 0.00 0.38 0.00 0.07 93.90 03:28:01 AM 1 3.39 0.00 0.27 0.00 0.05 96.29 03:29:01 AM all 4.42 0.00 0.33 0.00 0.07 95.18 03:29:01 AM 0 3.83 0.00 0.32 0.00 0.07 95.79 03:29:01 AM 1 5.02 0.00 0.35 0.00 0.07 94.57 03:29:01 AM CPU %user %nice %system %iowait %steal %idle 03:30:01 AM all 7.23 0.00 0.45 0.00 0.08 92.24 03:30:01 AM 0 4.72 0.00 0.38 0.00 0.07 94.83 03:30:01 AM 1 9.74 0.00 0.52 0.00 0.10 89.64 03:31:01 AM all 0.85 0.00 0.12 0.00 0.05 98.98 03:31:01 AM 0 1.00 0.00 0.15 0.00 0.07 98.78 03:31:01 AM 1 0.70 0.00 0.08 0.00 0.03 99.18 03:32:01 AM all 0.94 0.00 0.12 0.00 0.05 98.90 03:32:01 AM 0 0.82 0.00 0.08 0.00 0.05 99.05 03:32:01 AM 1 1.05 0.00 0.15 0.00 0.05 98.75 03:33:01 AM all 0.91 0.00 0.13 0.00 0.06 98.90 03:33:01 AM 0 0.60 0.00 0.12 0.00 0.03 99.25 03:33:01 AM 1 1.22 0.00 0.15 0.00 0.08 98.55 03:34:02 AM all 0.90 0.00 0.13 0.00 0.06 98.91 03:34:02 AM 0 0.82 0.00 0.17 0.00 0.07 98.95 03:34:02 AM 1 0.98 0.00 0.08 0.00 0.05 98.88 03:35:01 AM all 1.00 0.00 0.12 0.00 0.08 98.80 03:35:01 AM 0 1.14 0.00 0.10 0.00 0.07 98.69 03:35:01 AM 1 0.87 0.00 0.14 0.00 0.08 98.91 03:36:01 AM all 0.82 0.00 0.14 0.00 0.08 98.96 03:36:01 AM 0 0.45 0.00 0.13 0.00 0.07 99.35 03:36:01 AM 1 1.19 0.00 0.15 0.00 0.08 98.58 03:37:01 AM all 6.95 0.00 3.98 0.00 0.08 88.99 03:37:01 AM 0 1.44 0.00 0.22 0.00 0.08 98.26 03:37:01 AM 1 12.91 0.00 8.05 0.00 0.07 78.97 03:38:01 AM all 6.04 0.00 3.49 0.00 0.07 90.40 03:38:01 AM 0 3.03 0.00 0.29 0.00 0.07 96.62 03:38:01 AM 1 9.30 0.00 6.95 0.00 0.07 83.67 03:39:01 AM all 6.09 0.00 4.03 0.00 0.09 89.79 03:39:01 AM 0 11.18 0.00 7.21 0.00 0.07 81.53 03:39:01 AM 1 1.43 0.00 1.11 0.00 0.10 97.35 03:40:01 AM all 5.91 0.00 4.50 0.00 0.11 89.48 03:40:01 AM 0 7.76 0.00 3.90 0.00 0.13 88.22 03:40:01 AM 1 4.12 0.00 5.09 0.00 0.09 90.70 03:40:01 AM CPU %user %nice %system %iowait %steal %idle 03:41:01 AM all 7.10 0.00 4.19 0.00 0.12 88.60 03:41:01 AM 0 7.72 0.00 3.12 0.00 0.14 89.01 03:41:01 AM 1 6.47 0.00 5.24 0.00 0.09 88.20 03:42:01 AM all 6.70 0.00 4.51 0.00 0.10 88.69 03:42:01 AM 0 0.70 0.00 0.30 0.00 0.08 98.91 03:42:01 AM 1 13.30 0.00 9.15 0.00 0.11 77.44 03:43:01 AM all 5.93 0.00 3.64 0.00 0.08 90.36 03:43:01 AM 0 0.67 0.00 0.23 0.00 0.07 99.03 03:43:01 AM 1 11.76 0.00 7.41 0.00 0.09 80.73 03:44:01 AM all 26.97 0.00 2.80 0.07 0.11 70.06 03:44:01 AM 0 18.74 0.00 2.69 0.03 0.10 78.44 03:44:01 AM 1 35.30 0.00 2.91 0.10 0.12 61.57 03:45:01 AM all 31.61 0.00 4.16 0.32 0.09 63.83 03:45:01 AM 0 14.93 0.00 3.47 0.28 0.08 81.23 03:45:01 AM 1 48.36 0.00 4.84 0.35 0.10 46.35 Average: all 3.28 0.00 0.85 0.01 0.06 95.80 Average: 0 2.53 0.00 0.59 0.00 0.06 96.81 Average: 1 4.03 0.00 1.10 0.02 0.06 94.78