-
-
Notifications
You must be signed in to change notification settings - Fork 3.2k
Description
Describe the bug
When using CMake integration to auto-discover tests and run them with CTest, tests which are skipped are reported as failed.
Expected behavior
Skipped tests are reported as skipped.
Platform information:
- OS: Linux (Neon 22.04, based on Ubuntu 22.04)
- Compiler+version: 17.0.6
- Catch version: v3.3.2
Additional context
I think the reason is that ctest runs the tests one-by-one and if a test is skipped, this actually means that no test was run so, since --allow-running-no-tests is not provided, the return code is non-zero.
Now, CTest allows marking skipped tests via a special return code by setting the SKIP_RETURN_CODE property on the test. So, it seems to me, this could be fixed by replacing the
set(properties ${TEST_PROPERTIES})line in CatchAddTests.cmake with
set(properties "${TEST_PROPERTIES};SKIP_RETURN_CODE;4")This works for me as does passing SKIP_RETURN_CODE 4 to catch_discover_tests PROPETIES argument, e.g. like so:
catch_discover_tests(middleware_tests
PROPERTIES
SKIP_RETURN_CODE 4
)