Draft, outdated: Carbon SR1 test report¶
Test Case Summary¶
RelEng stability summary.
tba: Recent failures to be analyzed yet: 0.
test: Recent failures caused by wrong assumptions in test: 0.
akka: Recent failures related to pure UnreachableMember: 5.
tell: Recent failures not clearly caused by UnreachableMember: 9.
few: Tests passing unless low frequency failure happens: 22 (21 without duplication). (Low frequency means UnreachableMemeber or “Message was not delivered, dead letters encountered”, both are related to Akka where Controller code has not real control.)
pass: Tests passing consistently: 17 (15 without duplication).
Total: 53 (50 without duplication).
Total minus akka: 48 (45 without duplication).
Total minus akka passing always or mostly: 39 (36 without duplication).
Acceptance rate: 39/48=81.25% (36/45=80.00% without duplication).
Table¶
S017 instead of 2017 means Sandbox run (includes changes not merged to stable/carbon yet).
Last fail is date of last failure not caused by infra (or by a typo in test or by netconf/bgp failing to initialize properly).
“S 17” or “2 17” in Last run means the documented run was superseded by a newer one, but not analyzed yet.
“long ago” means the last real test failue happened before around 2017-05-19, or never.
Scenario name |
Type |
Last fail |
Last run |
Bugs |
Robot link |
|---|---|---|---|---|---|
bgp-1n-1m-a |
pass |
long ago |
2017-07-14 |
||
bgp-1n-300k-t |
pass |
long ago |
2017-07-14 |
||
bgp-3n-300k-ll-t |
akka |
2017-07-14 |
2017-07-14 |
||
bgp-3n-300k-lr-t |
akka |
2017-07-13 |
2017-07-14 |
||
ddb-cls-ms-ll-t |
few |
2017-07-04 |
2017-07-15 |
||
ddb-cls-ms-lr-t |
few |
2017-07-08 |
2017-07-15 |
||
ddb-cls-ps-ll-t |
few |
2017-07-09 |
2017-07-15 |
||
ddb-cls-ps-lr-t |
pass |
long ago |
2017-07-15 |
||
ddb-elm-ms-lr-t |
few |
2017-06-13 |
2017-07-15 |
||
ddb-elm-ms-rr-t |
few |
2017-06-10 |
2017-07-15 |
||
ddb-elm-ms-rl-t |
few |
2017-06-27 |
2017-07-15 |
||
ddb-elm-ps-lr-t |
few |
2017-06-11 |
2017-07-15 |
||
ddb-elm-ps-rr-t |
pass |
long ago |
2017-07-15 |
||
ddb-elm-ps-rl-t |
few |
2017-06-07 |
2017-07-15 |
||
ddb-li-ms-st-t |
tell |
2017-07-15 |
2017-07-15 |
||
ddb-li-ms-dt-t |
tell |
2017-07-15 |
2017-07-15 |
||
ddb-li-ps-st-t |
few |
2017-06-08 |
2017-07-15 |
||
ddb-li-ps-dt-t |
tell |
2017-07-15 |
2017-07-15 |
||
ddb-ci-ms-ll-ct-t |
few |
2017-06-07 |
2017-07-15 |
||
ddb-ci-ms-ll-st-t |
tell |
2017-07-15 |
2017-07-15 |
||
ddb-ci-ms-lr-ct-t |
few |
2017-06-08 |
2017-07-15 |
||
ddb-ci-ms-lr-st-t |
tell |
2017-07-15 |
2017-07-15 |
||
ddb-ci-ps-ll-ct-t |
few |
2017-06-28 |
2017-07-15 |
||
ddb-ci-ps-ll-st-t |
few |
2017-06-28 |
2017-07-15 |
||
ddb-ci-ps-lr-ct-t |
few |
2017-06-28 |
2017-07-15 |
||
ddb-ci-ps-lr-st-t |
few |
2017-06-28 |
2017-07-15 |
||
ddb-ls-ms-lr-t |
tell |
2017-07-15 |
2017-07-15 |
||
ddb-ls-ms-rr-t |
tell |
2017-07-14 |
2017-07-15 |
||
ddb-ls-ms-rl-t |
tell |
2017-07-12 |
2017-07-15 |
||
ddb-ls-ps-lr-t |
pass |
long ago |
2017-07-15 |
||
ddb-ls-ps-rr-t |
few |
2017-06-26 |
2017-07-15 |
||
ddb-ls-ps-rl-t |
pass |
long ago |
2017-07-15 |
||
drb-rpp-ms-a |
pass |
long ago |
2017-07-15 |
||
drb-rph-ms-a |
few |
2017-06-28 |
2017-07-15 |
||
drb-app-ms-a |
pass |
long ago |
2017-07-15 |
||
drb-aph-ms-a |
few |
2017-07-02 |
2017-07-15 |
||
dnb-1n-60k-a |
pass |
long ago |
2017-07-15 |
||
ss-ms-ms-a |
pass |
long ago |
2017-07-15 |
||
ss-ph-ms-a |
few |
2017-06-29 |
2017-07-15 |
||
ss-cl-ms-a |
pass |
long ago |
2017-07-15 |
||
ss-ms-ms-t |
pass |
long ago |
2017-07-15 |
||
ss-ph-ms-t |
few |
2017-07-15 |
2017-07-15 |
||
ss-cl-ms-t |
pass |
long ago |
2017-07-15 |
||
netconf-ba-ms-a |
pass |
long ago |
2017-07-14 |
||
netconf-ok-ms-a |
few |
2017-06-18 |
2017-07-14 |
||
netconf-rr-ms-a |
pass |
long ago |
2017-07-14 |
||
bgp-3n-300k-t-long |
akka |
2017-07-08 |
2017-07-08 |
||
ddb-elm-mc-t-long |
tell |
2017-07-08 |
2017-07-08 |
||
drb-rpp-ms-a-long |
few |
2017-05-07 |
2017-07-08 |
||
drb-rph-ms-a-long |
akka |
2017-07-08 |
2017-07-08 |
||
dnb-1n-60k-a-long |
pass |
long ago |
2017-07-08 |
||
ss-ph-ms-a-long |
akka |
2017-07-08 |
2017-07-08 |
||
ss-cl-ms-a-long |
pass |
long ago |
2017-07-08 |
For descriptions of test cases, see description page. Note that the link contains current description, the details might have been implemented differently at SR1 release.