Draft, outdated: Carbon SR1 test report ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Test Case Summary ----------------- RelEng stability summary. + tba: Recent failures to be analyzed yet: 0. + test: Recent failures caused by wrong assumptions in test: 0. + akka: Recent failures related to pure UnreachableMember: 5. + tell: Recent failures not clearly caused by UnreachableMember: 9. + few: Tests passing unless low frequency failure happens: 22 (21 without duplication). (Low frequency means UnreachableMemeber or "Message was not delivered, dead letters encountered", both are related to Akka where Controller code has not real control.) + pass: Tests passing consistently: 17 (15 without duplication). + Total: 53 (50 without duplication). + Total minus akka: 48 (45 without duplication). + Total minus akka passing always or mostly: 39 (36 without duplication). + Acceptance rate: 39/48=81.25% (36/45=80.00% without duplication). Table ----- S017 instead of 2017 means Sandbox run (includes changes not merged to stable/carbon yet). Last fail is date of last failure not caused by infra (or by a typo in test or by netconf/bgp failing to initialize properly). "S 17" or "2 17" in Last run means the documented run was superseded by a newer one, but not analyzed yet. "long ago" means the last real test failue happened before around 2017-05-19, or never. .. table:: Releng stability results (pre-SR1) :widths: 30,10,20,20,10,10 ================== ===== ========== ========== ============================================================= ========== Scenario name Type Last fail Last run Bugs Robot link ================== ===== ========== ========== ============================================================= ========== bgp-1n-1m-a pass long ago 2017-07-14 `link <https://jenkins.opendaylight.org/releng/view/bgpcep/job/bgpcep-csit-1node-periodic-bgp-ingest-only-carbon/lastSuccessfulBuild/robot/bgpcep-bgp-ingest.txt/Singlepeer%20Prefixcount/>`__ bgp-1n-300k-t pass long ago 2017-07-14 `link <https://jenkins.opendaylight.org/releng/view/bgpcep/job/bgpcep-csit-1node-periodic-bgp-ingest-only-carbon/lastSuccessfulBuild/robot/bgpcep-bgp-ingest.txt/Singlepeer%20Pc%20Shm%20300Kroutes_1/>`__ bgp-3n-300k-ll-t akka 2017-07-14 2017-07-14 `8318 <https://bugs.opendaylight.org/show_bug.cgi?id=8318>`__ `link <https://logs.opendaylight.org/releng/jenkins092/bgpcep-csit-3node-periodic-bgpclustering-only-carbon/343/log.html.gz#s1-s2-t8-k2-k3-k7-k5-k1-k6-k1-k1-k1-k1-k1-k2-k1-k4>`__ bgp-3n-300k-lr-t akka 2017-07-13 2017-07-14 `8318 <https://bugs.opendaylight.org/show_bug.cgi?id=8318>`__ `link <https://logs.opendaylight.org/releng/jenkins092/bgpcep-csit-3node-periodic-bgpclustering-only-carbon/343/log.html.gz#s1-s4-t8-k2-k3-k7-k7-k1-k6-k1-k1-k1-k1-k1-k2-k1-k2-k4>`__ ddb-cls-ms-ll-t few 2017-07-04 2017-07-15 `8794 <https://bugs.opendaylight.org/show_bug.cgi?id=8794>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/771/log.html.gz#s1-s20-t1-k2-k8>`__ ddb-cls-ms-lr-t few 2017-07-08 2017-07-15 `8618 <https://bugs.opendaylight.org/show_bug.cgi?id=8618>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/776/log.html.gz#s1-s20-t3-k2-k8>`__ ddb-cls-ps-ll-t few 2017-07-09 2017-07-15 `8794 <https://bugs.opendaylight.org/show_bug.cgi?id=8794>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/777/log.html.gz#s1-s22-t1-k2-k8>`__ ddb-cls-ps-lr-t pass long ago 2017-07-15 `link <https://jenkins.opendaylight.org/releng/view/controller/job/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/robot/controller-clustering.txt/Clean%20Leader%20Shutdown%20Prefbasedshard/Remote_Leader_Shutdown/>`__ ddb-elm-ms-lr-t few 2017-06-13 2017-07-15 `8618 <https://bugs.opendaylight.org/show_bug.cgi?id=8618>`__ `link <https://logs.opendaylight.org/sandbox/jenkins091/controller-csit-3node-clustering-only-carbon/31/log.html.gz#s1-s24-t1-k2-k10>`__ ddb-elm-ms-rr-t few 2017-06-10 2017-07-15 `8618 <https://bugs.opendaylight.org/show_bug.cgi?id=8618>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/738/log.html.gz#s1-s24-t3-k2-k10>`__ ddb-elm-ms-rl-t few 2017-06-27 2017-07-15 `8749 <https://bugs.opendaylight.org/show_bug.cgi?id=8749>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/763/log.html.gz#s1-s24-t5-k2-k10>`__ ddb-elm-ps-lr-t few 2017-06-11 2017-07-15 `8664 <https://bugs.opendaylight.org/show_bug.cgi?id=8664>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/736/log.html.gz#s1-s26-t1-k2-k6-k3-k1-k4-k7-k1>`__ ddb-elm-ps-rr-t pass long ago 2017-07-15 `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/log.html.gz#s1-s26-t3>`__ ddb-elm-ps-rl-t few 2017-06-07 2017-07-15 `8403 <https://bugs.opendaylight.org/show_bug.cgi?id=8403>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/735/log.html.gz#s1-s26-t5-k2-k9>`__ ddb-li-ms-st-t tell 2017-07-15 2017-07-15 `8792 <https://bugs.opendaylight.org/show_bug.cgi?id=8792>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/783/log.html.gz#s1-s28-t1-k2-k25-k1-k1>`__ ddb-li-ms-dt-t tell 2017-07-15 2017-07-15 `8619 <https://bugs.opendaylight.org/show_bug.cgi?id=8619>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/783/log.html.gz#s1-s28-t3-k2-k25-k1-k8>`__ ddb-li-ps-st-t few 2017-06-08 2017-07-15 `8371 <https://bugs.opendaylight.org/show_bug.cgi?id=8371>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/736/log.html.gz#s1-s30-t1-k2-k25-k1-k1>`__ ddb-li-ps-dt-t tell 2017-07-15 2017-07-15 `8845 <https://bugs.opendaylight.org/show_bug.cgi?id=8845>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/783/log.html.gz#s1-s30-t3-k2-k25-k1-k8>`__ ddb-ci-ms-ll-ct-t few 2017-06-07 2017-07-15 `8494 <https://bugs.opendaylight.org/show_bug.cgi?id=8494>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/735/log.html.gz#s1-s32-t1-k2-k16-k1-k1>`__ ddb-ci-ms-ll-st-t tell 2017-07-15 2017-07-15 `8494 <https://bugs.opendaylight.org/show_bug.cgi?id=8494>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/783/log.html.gz#s1-s32-t3-k2-k16-k1-k1>`__ ddb-ci-ms-lr-ct-t few 2017-06-08 2017-07-15 `8636 <https://bugs.opendaylight.org/show_bug.cgi?id=8636>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/736/log.html.gz#s1-s32-t5-k2-k15-k1-k1-k1-k1-k1-k1-k2-k1-k1-k1>`__ ddb-ci-ms-lr-st-t tell 2017-07-15 2017-07-15 `8494 <https://bugs.opendaylight.org/show_bug.cgi?id=8494>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/783/log.html.gz#s1-s32-t7-k2-k16-k1-k1>`__ ddb-ci-ps-ll-ct-t few 2017-06-28 2017-07-15 `8494 <https://bugs.opendaylight.org/show_bug.cgi?id=8494>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/764/log.html.gz#s1-s34-t1-k2-k16-k1-k1>`__ ddb-ci-ps-ll-st-t few 2017-06-28 2017-07-15 `8494 <https://bugs.opendaylight.org/show_bug.cgi?id=8494>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/764/log.html.gz#s1-s34-t3-k2-k16-k1-k1>`__ ddb-ci-ps-lr-ct-t few 2017-06-28 2017-07-15 `8494 <https://bugs.opendaylight.org/show_bug.cgi?id=8494>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/764/log.html.gz#s1-s34-t5-k2-k16-k1-k1>`__ ddb-ci-ps-lr-st-t few 2017-06-28 2017-07-15 `8494 <https://bugs.opendaylight.org/show_bug.cgi?id=8494>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/764/log.html.gz#s1-s34-t7-k2-k16-k1-k1>`__ ddb-ls-ms-lr-t tell 2017-07-15 2017-07-15 `8792 <https://bugs.opendaylight.org/show_bug.cgi?id=8792>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/783/log.html.gz#s1-s36-t1-k2-k12>`__ ddb-ls-ms-rr-t tell 2017-07-14 2017-07-15 `8792 <https://bugs.opendaylight.org/show_bug.cgi?id=8792>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/782/log.html.gz#s1-s36-t3-k2-k12>`__ ddb-ls-ms-rl-t tell 2017-07-12 2017-07-15 `8792 <https://bugs.opendaylight.org/show_bug.cgi?id=8792>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/779/log.html.gz#s1-s36-t5-k2-k12>`__ ddb-ls-ps-lr-t pass long ago 2017-07-15 `link <https://jenkins.opendaylight.org/releng/job/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/robot/controller-clustering.txt/Listener%20Stability%20Prefbasedshard/Move_Leader_From_Listener_Local_To_Remote/>`__ ddb-ls-ps-rr-t few 2017-06-26 2017-07-15 `8733 <https://bugs.opendaylight.org/show_bug.cgi?id=8733>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/762/log.html.gz#s1-s38-t3-k2-k14-k2-k1-k4-k7-k1>`__ ddb-ls-ps-rl-t pass long ago 2017-07-15 `link <https://jenkins.opendaylight.org/releng/job/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/robot/controller-clustering.txt/Listener%20Stability%20Prefbasedshard/Move_Leader_From_Listener_Remote_To_Local/>`__ drb-rpp-ms-a pass long ago 2017-07-15 `link <https://jenkins.opendaylight.org/releng/view/controller/job/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/robot/controller-clustering.txt/Rpc%20Provider%20Precedence/>`__ drb-rph-ms-a few 2017-06-28 2017-07-15 `8430 <https://bugs.opendaylight.org/show_bug.cgi?id=8430>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/764/log.html.gz#s1-s4-t6-k2-k1-k1-k1-k1-k1-k1-k2-k1-k1-k1-k3-k1-k1-k1-k2-k1-k4-k7-k1>`__ drb-app-ms-a pass long ago 2017-07-15 `link <https://jenkins.opendaylight.org/releng/view/controller/job/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/robot/controller-clustering.txt/Action%20Provider%20Precedence/>`__ drb-aph-ms-a few 2017-07-02 2017-07-15 `8430 <https://bugs.opendaylight.org/show_bug.cgi?id=8430>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/769/log.html.gz#s1-s8-t6-k2-k1-k1-k1-k1-k1-k1-k1-k1-k1-k1-k3-k1-k1-k1-k3-k1-k4-k7-k1>`__ dnb-1n-60k-a pass long ago 2017-07-15 `link <https://jenkins.opendaylight.org/releng/view/controller/job/controller-csit-1node-rest-cars-perf-only-carbon/lastSuccessfulBuild/robot/controller-rest-cars-perf.txt/Noloss%20Rate%201Node/>`__ ss-ms-ms-a pass long ago 2017-07-15 `link <https://jenkins.opendaylight.org/releng/view/controller/job/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/robot/controller-clustering.txt/Master%20Stability/>`__ ss-ph-ms-a few 2017-06-29 2017-07-15 `8420 <https://bugs.opendaylight.org/show_bug.cgi?id=8420>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/765/log.html.gz#s1-s12-t5-k2-k3-k1-k2>`__ ss-cl-ms-a pass long ago 2017-07-15 `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/log.html.gz#s1-s14>`__ ss-ms-ms-t pass long ago 2017-07-15 `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/log.html.gz#s1-s40>`__ ss-ph-ms-t few 2017-07-15 2017-07-15 `8420 <https://bugs.opendaylight.org/show_bug.cgi?id=8420>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/783/log.html.gz#s1-s42-t5-k2-k3-k1-k2>`__ ss-cl-ms-t pass long ago 2017-07-15 `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-clustering-only-carbon/lastSuccessfulBuild/log.html.gz#s1-s44>`__ netconf-ba-ms-a pass long ago 2017-07-14 `link <https://jenkins.opendaylight.org/releng/view/netconf/job/netconf-csit-3node-clustering-only-carbon/lastSuccessfulBuild/robot/netconf-clustering.txt/CRUD>`__ netconf-ok-ms-a few 2017-06-18 2017-07-14 `8596 <https://bugs.opendaylight.org/show_bug.cgi?id=8596>`__ `link <https://logs.opendaylight.org/releng/jenkins092/netconf-csit-3node-clustering-only-carbon/568/log.html.gz#s1-s5-t17-k2-k3-k2-k2-k1>`__ netconf-rr-ms-a pass long ago 2017-07-14 `link <https://jenkins.opendaylight.org/releng/view/netconf/job/netconf-csit-3node-clustering-only-carbon/lastSuccessfulBuild/robot/netconf-clustering.txt/Outages>`__ bgp-3n-300k-t-long akka 2017-07-08 2017-07-08 `8318 <https://bugs.opendaylight.org/show_bug.cgi?id=8318>`__ `link <https://logs.opendaylight.org/releng/jenkins092/bgpcep-csit-3node-bgpclustering-longevity-only-carbon/11/log.html.gz#s1-s2-t1-k10-k1-k1-k1-k1-k1-k1-k1-k1-k1-k2-k1-k3-k7-k5-k1-k6-k1-k1-k1-k1-k1-k2-k1-k1-k2-k2-k2-k1-k6-k2-k2-k1>`__ ddb-elm-mc-t-long tell 2017-07-08 2017-07-08 `8618 <https://bugs.opendaylight.org/show_bug.cgi?id=8618>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-ddb-expl-lead-movement-longevity-only-carbon/14/log.html.gz#s1-s2-t1-k2-k1-k1-k1-k1-k1-k1-k2-k1-k1-k2-k10>`__ drb-rpp-ms-a-long few 2017-05-07 2017-07-08 `8430 <https://bugs.opendaylight.org/show_bug.cgi?id=8430>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-drb-partnheal-longevity-only-carbon/13/console.log.gz>`__ drb-rph-ms-a-long akka 2017-07-08 2017-07-08 `8430 <https://bugs.opendaylight.org/show_bug.cgi?id=8430>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-drb-partnheal-longevity-only-carbon/18/log.html.gz#s1-t1-k3-k1-k1-k1-k1-k1-k1-k2-k1-k1-k6-k1-k1-k1-k1-k1-k1-k2-k1-k1-k1-k3-k1-k1-k1-k2-k1-k4-k7-k1>`__ dnb-1n-60k-a-long pass long ago 2017-07-08 `link <https://jenkins.opendaylight.org/releng/view/controller/job/controller-csit-1node-rest-cars-perf-only-carbon/620/robot/controller-rest-cars-perf.txt/Noloss%20Rate%201Node/>`__ ss-ph-ms-a-long akka 2017-07-08 2017-07-08 `8420 <https://bugs.opendaylight.org/show_bug.cgi?id=8420>`__ `link <https://logs.opendaylight.org/releng/jenkins092/controller-csit-3node-cs-partnheal-longevity-only-carbon/15/log.html.gz#s1-s2-t1-k3-k1-k1-k1-k1-k1-k1-k1-k1-k1-k7-k3-k1-k2>`__ ss-cl-ms-a-long pass long ago 2017-07-08 `link <https://jenkins.opendaylight.org/releng/view/controller/job/controller-csit-1node-rest-cars-perf-only-carbon/620/robot/controller-rest-cars-perf.txt/Noloss%20Rate%201Node/>`__ ================== ===== ========== ========== ============================================================= ========== For descriptions of test cases, see `description page <tests.html>`_. Note that the link contains current description, the details might have been implemented differently at SR1 release.