util-linux regression tests
~~~~~~~~~~~~~~~~~~~~~~~~~~~
It's expected that for each invasive change or important bugfix you will
include a test to your patch.
Compile binaries, libs, extra test programs and run the basic tests:
$ make check
Compile extra test programs only:
$ make check-programs
Note that the configure option --disable-static disables many of libmount and
libblkid unit tests.
Run all tests including tests that require root permissions:
# cd tests
# ./run.sh [options, see --help]
Alternatively using sudo and make:
$ make check-programs
$ sudo -E make check TS_OPTS="--parallel=1"
note that as root you have to manually remove output and diff directories
# rm -rf output diff
or run 'make clean' as root.
Run subset of tests:
$ cd tests
$ ./run.sh <test_directory-name>
for example:
$ ./run.sh blkid
$ ./run.sh libmount
or individual test script, for example:
$ ./ts/cal/year
The tests is possible to exclude by ./run.sh --exclude=<list> where the
<list> is blank separated test names in format "testdir/testname", for example:
$ ./run.sh --exclude="mount/move"
The --exclude is evaluated by the ./run.sh script only. See below
TS_OPT_testdir_[testscript_]fake=
environment variable which provides more powerful functionality to skip tests.
*** WARNING for root users ***
The tests touch your /etc/fstab, initialize loop devices or scsi_debug devices
if executed with root permissions.
Please, be careful and use these tests only for development and never on
production system.
fuzz targets
------------
The fuzz targets can be built and run along with the other tests (after installing
clang):
$ ./tools/config-gen fuzz
$ make check
environment variables
---------------------
TS_COMMAND
Evaluated by "make check" to override the default command (run.sh).
Example:
- build all test dependencies, but skip the actual test
$ make check TS_COMMAND="true"
TS_OPTS
Evaluated by "make check" to pass options.to run.sh (see ./run.sh --help).
Examples:
- run utmp tests only
$ make check TS_OPTS="--parallel=1 utmp"
TS_OPT_testdir_[testscript_]fake="<yes|no>"
Evaluated by any test script to skip certain tests.
Examples:
- skip all the tests within "fdisk" test-directory:
$ make check TS_OPT_fdisk_fake="yes"
- skip only "fdisk/bsd" test:
$ make check TS_OPT_fdisk_bsd_fake="yes"
- skip all "fdisk" tests except fdisk/bsd:
$ make check TS_OPT_fdisk_fake="yes" TS_OPT_fdisk_bsd_fake="no"
TS_OPT_testdir_[testscript_]known_fail="<yes|no>"
Similar usage like TS_OPT_*_fake above. "known_fail" means that the given
test will run but (negative) results will be ignored. The build log and test
diffs will still remind you about the issue.
TS_OPT_testdir_[testscript_]verbose="<yes|no>"
Set verbosity for certain tests. Similar usage like TS_OPT_*_fake above.
TS_OPT_testdir_[testscript_]memcheck="<yes|no>"
Run certain tests with valgrind. Similar usage like TS_OPT_*_fake above.
External services
-----------------
Coveralls coverage reports
URL: https://coveralls.io/github/util-linux/util-linux
lgtm CI - automatically executed security code analysis
URL: https://lgtm.com/projects/g/util-linux/util-linux/
Coverity Scan
URL: https://scan.coverity.com/projects/karelzak-util-linux
Fossies codespell report
URL: https://fossies.org/linux/test/util-linux-master.tar.gz/codespell.html
OSS-Fuzz
URL: https://google.github.io/oss-fuzz/
URL: https://oss-fuzz-build-logs.storage.googleapis.com/index.html#util-linux
URL: https://oss-fuzz.com/coverage-report/job/libfuzzer_asan_util-linux/latest
CIFuzz
URL: https://google.github.io/oss-fuzz/getting-started/continuous-integration/
URL: https://github.com/util-linux/util-linux/actions?query=workflow%3ACIFuzz
