benchtest

Benchmarking and testing framework. More...

Collaboration diagram for benchtest:

Modules
	assertions
	Macros for expectations and assertions.

Classes
class	testing::Test
	Abstract class to be inherited by test fixtures. More...

Macros
#define	TEST(test_case_name, test_name)
	Declare an inline test. More...
#define	TEST_F(test_fixture, method_name)
	Declare a test using a fixture. More...
#define	TEST_T(test_fixture, type_name, method_name)
	Declare a test using a fixture template. More...

Functions
template<typename T >
void	testing::PrintTo (const T &val,::std::ostream *os)
	Specialized printer for testing results. More...
template<typename T >
::std::string	testing::PrintToString (const T &val)
	Prints val to a string using PrintTo().
static void	testing::Test::SetUpTestCase ()
	Called once before any tests in this case are run.
static void	testing::Test::TearDownTestCase ()
	Called once when all tests in this case are finished.
virtual void	testing::Test::SetUp ()
	Called at the start of every test using this fixture.
virtual void	testing::Test::TearDown ()
	Called at the end of every test using this fixture.
bool	testing::Test::Benchmark (unsigned int max=100)
	Controls execution of a benchmarked section. More...
template<bool = true>
class Reporter *	testing::reporter (class Reporter *newr=nullptr)
	Select which Reporter to use.
int	testing::run ()
	Run the tests. Returns an exit status.

Detailed Description

Benchmarking and testing framework.

The benchtest framework may be used to test your own applications. Usage and syntax is similar, but not 100% compatible, to the Google C++ Testing Framework.

Setting up a new project to test is very easy. Make sure the benchtest include directory is in your compiler include path. Then call the test runner in main(). It is important to return from main() with the result of run(). This is because any testing automation, like continuous integration, depends on the exit status to determine if any failures were encountered.

#include <benchtest.hpp>
int main() {
    testing::reporter(new testing::DefaultReporter);
    return testing::run();
}

A test run consists of test cases that have one or more tests that have expectations, assertions, and benchmarks.

Macro Definition Documentation

#define TEST	(		test_case_name,
			test_name
	)

Declare an inline test.

TEST(MyTestCase, MyTestName){
    EXPECT_EQ(6, 2*3);
}

#define TEST_F	(		test_fixture,
			method_name
	)

Declare a test using a fixture.

Fixtures can be used when you have a number of tests that operate on similar data.

class Vectors : public testing::Test {
protected:
    std::vector<int> * data;
    static void SetUpTestCase() {
        data = new std::vector<int>;
    }
    static void TearDownTestCase() {
        delete data;
    }
    void SetUp() {
        data->assign(32, 0);
    }
    void TearDown() {
        data->resize(0);
    }
    void popper() {
        data->pop_back();
        EXPECT_EQ(31, data->size());
    }
    void pusher() {
        data->push_back(0);
        EXPECT_EQ(33, data->size());
    }
}
TEST_F(Vectors, popper)
TEST_F(Vectors, pusher)

#define TEST_T	(		test_fixture,
			type_name,
			method_name
	)

Declare a test using a fixture template.

A fixture template makes it easier to test code written to work with multiple data types or classes. For example, parts of a dsp library are tested using both float and double types.

template<typename T>
class FFT : public testing::Test {
protected:
    std::array<std::complex<T>, 512> data;
    void SetUp() {
        /* Data loaded with a control set here */
    }
    void dft() {
        FFT::dft(data);
        /* Results validated here */
    }
    void idft() {
        FFT::idft(data);
        /* Results validated here */
    }
}
TEST_T(FFT, float, dft)
TEST_T(FFT, double, dft)
TEST_T(FFT, float, idft)
TEST_T(FFT, double, idft)

Function Documentation

template<typename T >

void testing::PrintTo	(	const T &	val,
		::std::ostream *	os
	)

Specialized printer for testing results.

This function is used to print the value of a type in failure reports. If the type already supplies an operator<< to ::std::ostream it is used by default. It may not be possible to use operator<< or you may consider it bad form to do so only for testing. In that case, specialize this function to print your value.

testing::Test::Benchmark ( unsigned int  max = 100 )

inlineprotected

Controls execution of a benchmarked section.

Parameters

max	Limit the maximum number of executions.

Return values

bool	True indicates the benchmarked section should execute again.

A test may include a single benchmark.

TEST(Lifter, performance){
    while(Benchmark()) {
        do_some_heavy_lifting();
    }
}

The benchmarked section will run a minimum number of times equal to 20% of max. It will continue to run until the max limit is hit or faster executions are not being found. Regardless of max, at least 5 iterations will be run. If you have a very slow algorithm, decreasing max can guard against accidentally long run times when tests are deployed to be run automatically, such as on a continuous integration server.

If you have a very fast algorithm, the timer may not provide enough resolution for good results. In this case it's usually easy to run a set to reduce the timer variance. However, consider that you may be testing too small of a work unit which will report unrealistic results because it would normally be inlined by the compiler or is strongly affected by memory caching.

/* Run a small FFT 100 times... */
TEST(FFT, performance64){
    std::array<std::complex<float>,64> data;
    while(Benchmark()) {
       for(int i=0; i<100; ++i) FFT::dft(data);
    }
}
/* Run a large FFT one time... */
TEST(FFT, performance8192){
    std::array<std::complex<float>,8192> data;
    while(Benchmark()) {
       FFT::dft(data);
    }
}