CMake Cookbook Notes 4: Creating and Running Tests

Chapter 4 Creating and Running Tests

4.1 Creating a simple unit test

Use enable_testing and add_test to create tests with built-in CMake functionality:
cmake
1
2
3
4
5
6
7
add_executable(cpp_test test.cpp)

enable_testing()
add_test(
NAME cpp_test
COMMAND $<TARGET_FILE:cpp_test>
)
After the process of build, the tests can be run with a simple ctest or make test
Interestingly, the test cases can be written in any language supported by the system and the return value is used to indicate whether the test passes: zero for success and non-zero for failure.

4.3 Defining a unit test and linking against Google Test

Use FetchContent to add GTest as dependency in configure time, and then link its library with our application.
It seems that there is a GoogleTest CMake module specifically for Google Test application since v3.9.

4.6 Testing expected failures

We can use set_tests_properties to pass the test cases when it returns non-zero:

cmake

1
2
3

enable_testing()
add_test(example ${PYTHON_EXECUTABLE} ${CMAKE_CURRENT_SOURCE_DIR}/test.py)
set_tests_properties(example PROPERTIES WILL_FAIL true)

4.7 Using timeouts for long tests

We can also use set_tests_properties to set a timeout for test cases:
cmake
1
set_tests_properties(example PROPERTIES TIMEOUT 10)
The test case will terminate and be marked as failure as long as it goes past the timeout.

4.8 Running tests in parallel

Use ctest --parallel N to run test cases in parallel. One thing worth noting is that CMake execute test cases that take long time first in each core and then the shorter cases, in which strategy, the total time costed will be the least. Of course, in the very beginning even before the first execution, CMake is also not aware the time cost of each test case. However, we can specify explicitly the time cost of each test case with like set_tests_properties(example PROPERTIES COST 4.5).

4.9 Running a subset of tests

CMake provides several ways to run a subset of all test cases:
- by test case names (using regular expressions): ctest -R <reExp>
- by test case indexes: ctest -I <begin>,<end>
- by test case labels: ctest -L <label>

Yep, we can attach labels to test cases, still using set_tests_properties:

cmake

set_tests_properties(
  feature-a feature-b feature-c
  PROPERTIES
    LABELS "quick"
)

4.10 Using test fixtures

Usually some test cases require setup actions before running and cleanup actions after completing, we call this test fixtures. Of course create a test fixture is typically the task of testing framework like GTest, while we could also do that in CMake level.

To create test fixture with CMake, use add_test to create test cases for setup and cleanup actions first, and then mark them as setup and cleanup with set_tests_properties:

cmake

set_tests_properties(
  setup
  PROPERTIES
    FIXTURES_SETUP my-fixture
)

set_tests_properties(
  cleanup
  PROPERTIES
    FIXTURES_CLEANUP my-fixture
)

For test cases included in the fixture, mark them as FIXTURES_REQUIRED:

cmake

set_tests_properties(
  feature-a
  feature-b
  PROPERTIES
    FIXTURES_REQUIRED my-fixture
)

Link

https://subscription.packtpub.com/book/application_development/9781788470711/4