MD-SAL Data Store benchmarking.

Copyright (c) 2015 Cisco Systems, Inc. and others. All rights reserved.

This program and the accompanying materials are made available under the terms of the Eclipse Public License v1.0 which accompanies this distribution, and is available at http://www.eclipse.org/legal/epl-v10.html

This test suite uses the odl-dsbenchmark-impl feature controlled via dsbenchmark.py tool for testing the MD-SAL Data Store performance. (see the 'https://wiki.opendaylight.org/view/Controller_Core_Functionality_Tutorials:Tutorials:Data_Store_Benchmarking_and_Data_Access_Patterns')

Based on values in test suite variables it triggers required numbers of warm-up and measured test runs: odl-dsbenchmark-impl module generates (towards MD-SAL Data Store) specified structure, type and number of operations. The test suite performs checks for start-up and test execution timeouts (Start Measurement, Wait For Results) and basic checks for test runs results (Check Results). Finally it provides total numbers per operation structure and type (by default in the perf_per_struct.csv, perf_per_ops.csv files) suitable for plotting in system test environment. See also the 'https://wiki.opendaylight.org/view/CrossProject:Integration_Group:System_Test:Step_by_Step_Guide#Optional_-_Plot_a_graph_from_your_job' Included totals can be filtered using the FILTER parameter (RegExp). Because of the way how graphs are drawn, it is recomended to keep all test suite variables unchanged as defined for the 1st build. Parameters WARMUPS, RUNS and accordingly the TIMEOUT value can be changed for each build if needed. Parameter UNITS defines time units returned by odl-dsbenchmark-impl module. The dsbenchmark.py tool always returns values in miliseconds.

When running this robot suite always use --exclude tag for distinguish the run for 3node setup: need a benchmark for leader and follow (--exclude singlenode_setup) the run for 1node setup: no followr present (--exclude clustered_setup)