conftest.py:
@pytest.fixture(scope='session')
def fw_config(request):
logging.info(f'fw_config called with {request.param}')
return request.param
def pytest_generate_tests(metafunc):
if 'fw_config' in metafunc.fixturenames:
configs_to_test = ['ABC', 'DEF']
pattern = metafunc.definition.get_closest_marker('config_pattern')
if pattern is None:
pattern = '.*'
else:
assert len(pattern.args) == 1
pattern = pattern.args[0]
regex = re.compile(pattern)
configs_to_test = filter(regex.match, configs_to_test)
metafunc.parametrize(argnames='fw_config', argvalues=configs_to_test, ids=configs_to_test, scope='session', indirect=True)test file:
class TestGrouping:
@pytest.mark.config_pattern(r'.*ABC.*')
def test_abc(self, fw_config):
time.sleep(4)
return True
@pytest.mark.config_pattern(r'.*DEF.*')
def test_def(self, fw_config):
time.sleep(4)
return True
def test_all(self, fw_config):
time.sleep(4)
return TrueSince fw_config is session-scoped, I am hoping that the tests will run in the following order:
test_abc[ABC]
test_all[ABC]
test_def[DEF]
test_all[DEF]
And that fw_config will be called only twice.
However, instead, they are called in the following order:
test_abc[ABC]
test_def[DEF]
test_all[ABC]
test_all[DEF]
and fw_config is called four times.
I have been trying for almost a month to get this behavior using various methods and have not been able to achieve it.
I am aware that I can just have test_abc skip() if fw_config is 'DEF', but then I have a skipped test in my output. I want to not run the test at all.
fw_config here is a stand-in for an expensive configuration with a name. Some tests need to run on all configs, and some tests need to run on only a subset.
Is there any way to do this? Am I doing something incorrectly?
I am guessing that there are actually three instances of the fixture being created, and pytest is treating them as being totally separated.
conftest.py:
test file:
Since fw_config is session-scoped, I am hoping that the tests will run in the following order:
test_abc[ABC]
test_all[ABC]
test_def[DEF]
test_all[DEF]
And that fw_config will be called only twice.
However, instead, they are called in the following order:
test_abc[ABC]
test_def[DEF]
test_all[ABC]
test_all[DEF]
and fw_config is called four times.
I have been trying for almost a month to get this behavior using various methods and have not been able to achieve it.
I am aware that I can just have test_abc skip() if fw_config is 'DEF', but then I have a skipped test in my output. I want to not run the test at all.
fw_config here is a stand-in for an expensive configuration with a name. Some tests need to run on all configs, and some tests need to run on only a subset.
Is there any way to do this? Am I doing something incorrectly?
I am guessing that there are actually three instances of the fixture being created, and pytest is treating them as being totally separated.