Repository navigation
pytest_generate_tests based on params and marks from previous parametrize #13233
Description
Issue
In userver, we add indirect params to all tests based on marks using pytest_generate_tests.
(See the full story in #13217.)
The issue arises when we want to apply our implicit param only to some of the test items spawned from an already parametrized test:
@pytest.mark.parametrize(
"foo, bar",
[
("a", "x"),
pytest.param("b", "y", marks=pytest.mark.my_mark("hmm")),
pytest.param("c", "z", marks=pytest.mark.my_mark("what")),
])
def my_test(foo, bar, my_fixture, ...):
# ...
def pytest_parametrize_tests(metafunc):
# metafunc represents a test function before the split into items.
# How do we set indirect param "my_fixture" based on marks
# from one of the cases?The issue has been previously explored: #4050
Proposed syntax
What I want to discuss here is an implementation of that idea, starting with a possible syntax.
# pytest
@dataclasses.dataclass(frozen=True)
class ProtoItem(pytest.Node):
# essentially CallSpec2 with a rich API from Node
params: dict[str, object]
own_markers: list[Mark]
# usage
def pytest_parametrize_tests(metafunc):
def param_generator(proto_item: pytest.ProtoItem) -> ParameterSet | Sequence[object] | object:
if (mark := proto_item.get_closest_marker("my_mark")) is not None:
return (compute_param_value_from_mark(mark),)
else:
return (get_some_fallback_value(),)
metafunc.parametrize(("my_fixture",), param_generator)Reacted by Tom Kuson
Activity
RonnyPfannschmidt commentedon Feb 17, 2025
The idea is something under discussion since 2014 as covariant parameters
It's necessary for any parameterisation where you want/need to choose parameter sets dependent on others
It's also a prerequisite for correct lazy fixtures as currently it's not possible to use a parameterized fixture via lazy fixtures
Anton3 commentedon Feb 17, 2025
@RonnyPfannschmidt what about the proposed syntax? I need multi-parametrize that is not just a Cartesian product, and this would be enough for my use cases. Also looks unintrusive enough in terms of API.
RonnyPfannschmidt commentedon Feb 17, 2025
The meaning of the proposed object is very unclear to me, and I'd prefer to be able to work in terms of callspec
So more like a conditional parameterize that chooses based on individual callspecs instead of expanding all calls as product
Anton3 commentedon Feb 19, 2025
@RonnyPfannschmidt I've mostly implemented the feature, but now there's the issue that I want to use params and marks from
pytest.mark.parametrizein mypytest_generate_tests. But I can't get mypytest_generate_teststo be executed afterpytest.mark.parametrize,pytest.mark.parametrizealways seems to be applied last. Any ideas how to change that?RonnyPfannschmidt commentedon Feb 19, 2025
Either a trylast hook or a hookwrapper
Anton3 commentedon Feb 20, 2025
@RonnyPfannschmidt Done, PR is mostly ready for review, but I've left a comment