Summary

People are using tox -p auto or tox -p all more and more since it exists, (and some are simply using & in shell scripts).

But it's failing with pytest-cov (coveragepy/coveragepy#883, #356, #237, #217).

This is because pytest-cov uses a coverage combine step which tries to combine all .coverage.* files, mixing files from all of the parallels runs. As some are incomplete, this often yield to sqlite errors, but it also sometime just mix the data in strange ways.

A clean fix is to specify a specific coverage file name for each run, so the combine step will search for files with this specific name, avoiding mixing the files.

This can easily be done, for example, in .tox.ini by using:

setenv =
  COVERAGE_FILE=.coverage.{envname}

It make coverage combine search for .coverage.py37.* for example.

I see two strategies:

Either pytest-cov picks a unique coverage file name per run or pytest-cov documents that when used in parallel one should specify a coverage file name to disambiguate the runs.

Do you have a preference?