Skip to content

Commit 1ca5902

Browse files
Add the generic ItemDefinition parametrization protocol (pytest-dev#3926)
Give nodes.ItemDefinition the collect() protocol so a non-Python collector can have its test definitions parametrized through pytest_generate_tests: declare the accepted names in parametrize_argnames, build one item per parameter set in make_item(), and the chosen values arrive on item.callspec.params. Attaching the callspec, its marks and its id to the generated item is done by the protocol rather than left to each make_item(): the callspec drives reordering and high-scoped param caching, so it is not optional. ParametrizeContext moves into _pytest.parametrize as the collector agnostic base of Metafunc, which overrides four seams to bring in the fixture closure, signature validation and direct-parameter desugaring. No fixture resolution on the non-Python path; the four couplings in the way are written up in doc/en/proposals/generic_fixture_closures.rst.
1 parent 2c8c481 commit 1ca5902

11 files changed

Lines changed: 1150 additions & 428 deletions

File tree

‎changelog/14858.feature.rst‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
Added :class:`_pytest.nodes.ItemDefinition`, the collector-agnostic base for "a
2+
definition which generates items". Subclassing it lets a non-Python collector
3+
have its test definitions parametrized through :hook:`pytest_generate_tests`,
4+
exactly like Python test functions: declare the accepted names in
5+
``parametrize_argnames``, build one item per parameter set in ``make_item()``,
6+
and the chosen values arrive on ``item.callspec.params``. Parameter ids,
7+
``pytest.param(...)`` marks and node-id selection all work as usual; there is no
8+
fixture resolution on that path. See :ref:`non-python parametrization`.

‎changelog/14858.misc.rst‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
``ParametrizeContext`` is now the collector-agnostic base of
2+
:class:`pytest.Metafunc`, which adds the fixture closure, signature validation
3+
and direct-parameter desugaring on top of it.

‎doc/en/example/nonpython.rst‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,3 +105,65 @@ interesting to look at the collection tree:
105105
<YamlItem ok>
106106
107107
======================== 2 tests collected in 0.12s ========================
108+
109+
.. _`non-python parametrization`:
110+
111+
Letting non-python tests take part in parametrization
112+
-----------------------------------------------------
113+
114+
.. versionadded:: 9.2
115+
116+
A collector which yields items directly, as ``YamlFile`` above does, produces a
117+
fixed set of tests. To let a test *definition* be parametrized -- by
118+
:hook:`pytest_generate_tests`, exactly like a Python test function -- collect
119+
:class:`~_pytest.nodes.ItemDefinition` nodes instead of items.
120+
121+
Such a node declares the names it accepts in
122+
:attr:`~_pytest.nodes.ItemDefinition.parametrize_argnames` and builds one item
123+
per parameter set in :meth:`~_pytest.nodes.ItemDefinition.make_item`. The values
124+
chosen for a given item arrive on ``item.callspec.params``:
125+
126+
.. code-block:: python
127+
128+
# content of conftest.py
129+
from _pytest import nodes
130+
import pytest
131+
132+
133+
class YamlItem(pytest.Item):
134+
def __init__(self, *, spec, **kwargs):
135+
super().__init__(**kwargs)
136+
self.spec = spec
137+
138+
def runtest(self): ...
139+
140+
def reportinfo(self):
141+
return self.path, 0, self.name
142+
143+
144+
class YamlDefinition(nodes.ItemDefinition):
145+
parametrize_argnames = ("value",)
146+
147+
def make_item(self, parent, *, name, callspec, nodeid, context):
148+
params = dict(callspec.params) if callspec is not None else {}
149+
return YamlItem.from_parent(parent, name=name, nodeid=nodeid, spec=params)
150+
151+
Any ``pytest_generate_tests`` implementation now applies:
152+
153+
.. code-block:: python
154+
155+
def pytest_generate_tests(metafunc):
156+
if "value" in metafunc.fixturenames:
157+
metafunc.parametrize("value", ["good", "bad"])
158+
159+
which collects ``test_simple.yaml::hello[good]`` and
160+
``test_simple.yaml::hello[bad]`` under a single ``<YamlDefinition hello>`` node.
161+
Parameter ids, ``pytest.param(...)`` marks and node-id selection all work as
162+
they do for Python tests.
163+
164+
.. note::
165+
166+
This path covers parametrization only: there is no fixture resolution for
167+
non-Python items, so ``indirect=True`` and requesting fixtures from such an
168+
item are not supported. A ``parametrize()`` call naming anything outside
169+
``parametrize_argnames`` is rejected at collection time.
Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,143 @@
1+
:orphan:
2+
3+
============================================
4+
PROPOSAL: Generic fixture closures for items
5+
============================================
6+
7+
.. warning::
8+
9+
This document is the follow-up half of the ``ItemDefinition`` work: it
10+
outlines what is needed for *non-Python* items to request fixtures. The
11+
parametrization half has shipped, see :ref:`non-python parametrization`.
12+
13+
What shipped
14+
------------
15+
16+
:class:`~_pytest.nodes.ItemDefinition` lets any collector take part in
17+
:hook:`pytest_generate_tests`. A definition declares the names it accepts up
18+
front:
19+
20+
.. code-block:: python
21+
22+
class YamlDefinition(nodes.ItemDefinition):
23+
parametrize_argnames = ("value",)
24+
25+
def make_item(self, parent, *, name, callspec, nodeid, context):
26+
return YamlItem.from_parent(
27+
parent, name=name, nodeid=nodeid, spec=dict(callspec.params)
28+
)
29+
30+
and the values chosen for each parameter set arrive on ``item.callspec.params``.
31+
32+
``ParametrizeContext`` deliberately has no fixture machinery: ``_infer_scope()``
33+
returns ``Scope.Function``, ``_register_direct_params()`` is a no-op, and
34+
``_validate_argnames()`` checks against ``parametrize_argnames`` rather than a
35+
fixture closure. ``FixtureManager.pytest_generate_tests`` returns early for a
36+
context which is not a ``Metafunc``.
37+
38+
That covers parametrization. It does **not** cover:
39+
40+
- ``indirect=True`` -- there is no fixture to hand ``request.param`` to.
41+
- scope inference from fixture scopes -- an unspecified ``scope=`` is always
42+
function scope.
43+
- a non-Python item requesting a fixture at all, including one at the new
44+
``"definition"`` scope.
45+
46+
Why the rest is not a small step
47+
--------------------------------
48+
49+
Fixture resolution is written against ``Function``, not against ``Item``. The
50+
concrete couplings:
51+
52+
**TopRequest assumes a Python function item.**
53+
It is constructed as ``TopRequest(pyfuncitem=item)`` and reads
54+
``item._fixtureinfo``, ``item.funcargs``, ``item.fixturenames`` and
55+
``item.obj``. ``_fillfixtures()`` writes into ``item.funcargs``.
56+
``request.instance`` walks to a :class:`~pytest.Class` node and calls
57+
``newinstance()``.
58+
59+
**getfixtureinfo() is item-scoped and signature-driven.**
60+
Its closure starts from the function's argument names
61+
(``getfuncargnames(func)``) plus autouse names. A non-Python definition has no
62+
signature; its "argnames" are whatever it declares. ``FunctionDefinition``
63+
already has to ``cast(nodes.Item, self)`` to call it -- the same wart, one level
64+
up.
65+
66+
**Direct parametrization desugars into DirectParamFixtureDef.**
67+
That is how a direct param reaches the test at setup time. Without a fixture
68+
system there is nothing to desugar into, which is why the generic path delivers
69+
params on the callspec instead.
70+
71+
**Function.setup() drives everything.**
72+
``nodes.Item.setup()`` is a no-op; the fixture lifecycle only runs because
73+
``Function.setup()`` calls ``self._request._fillfixtures()``.
74+
75+
Proposed shape
76+
--------------
77+
78+
Three steps, each independently useful.
79+
80+
1. Split a node-level closure entry point out of ``getfixtureinfo()``
81+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
82+
83+
Today ``getfixtureinfo(node, func, cls, ignore_args)`` derives ``initialnames``
84+
from ``func``'s signature. Proposed:
85+
86+
.. code-block:: python
87+
88+
def getfixtureinfo_for_argnames(
89+
self, node: nodes.Node, argnames: Sequence[str], *, ignore_args=()
90+
) -> FuncFixtureInfo: ...
91+
92+
with the current signature-based version becoming a thin wrapper which computes
93+
``argnames`` from the function. This removes the ``cast(nodes.Item, self)`` in
94+
``FunctionDefinition`` as a side effect, and gives ``ItemDefinition`` a closure
95+
without inventing a fake function.
96+
97+
2. Make the request object item-generic
98+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
99+
100+
Introduce a small protocol for what ``TopRequest`` needs from its item --
101+
``fixturenames``, somewhere to store the computed values, ``_fixtureinfo``, and
102+
an optional ``instance`` -- and give :class:`~pytest.Item` a default
103+
implementation. :class:`~pytest.Function` keeps its current behaviour
104+
(``funcargs``, bound instance); a generic item gets a plain dict and
105+
``instance is None``.
106+
107+
The open question here is ``request.instance``, whose meaning is genuinely
108+
Python-specific. Returning ``None`` for non-Python items is the honest answer,
109+
but fixtures in the wild do use ``request.instance`` unconditionally.
110+
111+
3. Opt in per definition
112+
~~~~~~~~~~~~~~~~~~~~~~~~
113+
114+
Fixture support should not be forced on every ``ItemDefinition``; a collector
115+
which only wants parametrization should not pay for closure computation.
116+
Suggested switch:
117+
118+
.. code-block:: python
119+
120+
class ItemDefinition(Collector, abc.ABC):
121+
#: Whether items generated from this definition can request fixtures.
122+
supports_fixtures: bool = False
123+
124+
With ``supports_fixtures = True``, ``make_parametrize_context()`` computes a
125+
closure from ``parametrize_argnames`` via step 1 and returns a fixture-aware
126+
context, so ``indirect=True``, scope inference and ``"definition"`` scoped
127+
fixtures all work uniformly. ``FixtureManager.pytest_generate_tests`` would then
128+
key off the context being fixture-aware rather than off
129+
``isinstance(metafunc, Metafunc)``.
130+
131+
Open questions
132+
--------------
133+
134+
- Does ``Metafunc`` stay a distinct class once the generic context is
135+
fixture-aware, or does it collapse into ``ParametrizeContext`` plus the
136+
``module``/``cls``/``function`` attributes?
137+
- ``request.instance`` for non-Python items: ``None``, or an error on access?
138+
- Autouse fixtures currently apply to every item in a subtree. Should they apply
139+
to non-Python items which opted into fixtures, or only to those which declare
140+
the corresponding names?
141+
- Should ``parametrize_argnames`` and the fixture closure be the same list, or
142+
should a definition be able to accept a name for parametrization which is not
143+
requestable as a fixture?

‎doc/en/reference/reference.rst‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -970,6 +970,12 @@ CallInfo
970970
.. autoclass:: pytest.CallInfo()
971971
:members:
972972

973+
CallSpec
974+
~~~~~~~~
975+
976+
.. autoclass:: _pytest.parametrize.CallSpec()
977+
:members:
978+
973979
CollectReport
974980
~~~~~~~~~~~~~
975981

@@ -1043,6 +1049,13 @@ Metafunc
10431049

10441050
.. autoclass:: pytest.Metafunc()
10451051
:members:
1052+
:inherited-members:
1053+
1054+
ParametrizeContext
1055+
~~~~~~~~~~~~~~~~~~
1056+
1057+
.. autoclass:: _pytest.parametrize.ParametrizeContext()
1058+
:members:
10461059

10471060
Parser
10481061
~~~~~~

‎src/_pytest/fixtures.py‎

Lines changed: 11 additions & 43 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,6 @@
2525
from typing import Final
2626
from typing import final
2727
from typing import Generic
28-
from typing import Literal
2928
from typing import NoReturn
3029
from typing import overload
3130
from typing import TYPE_CHECKING
@@ -70,6 +69,7 @@
7069
from _pytest.outcomes import fail
7170
from _pytest.outcomes import skip
7271
from _pytest.outcomes import TEST_OUTCOME
72+
from _pytest.parametrize import _resolve_args_directness
7373
from _pytest.pathlib import absolutepath
7474
from _pytest.pathlib import bestrelpath
7575
from _pytest.scope import HIGH_SCOPES
@@ -85,8 +85,8 @@
8585

8686
if TYPE_CHECKING:
8787
from _pytest.parametrize import CallSpec
88+
from _pytest.parametrize import ParametrizeContext
8889
from _pytest.python import Function
89-
from _pytest.python import Metafunc
9090
from _pytest.reports import CollectReport
9191

9292

@@ -326,7 +326,8 @@ def get_param_argkeys(item: nodes.Item, scope: Scope) -> Iterator[ParamArgKey]:
326326
scoped_item_path = item.path
327327
elif scope is Scope.Class:
328328
scoped_item_path = item.path
329-
item_cls = item.cls # type: ignore[attr-defined]
329+
# Items which are not Python test functions have no class.
330+
item_cls = getattr(item, "cls", None)
330331
elif scope is Scope.Definition:
331332
scoped_item_path = item.path
332333
else:
@@ -1721,45 +1722,6 @@ def pytest_addoption(parser: Parser) -> None:
17211722
)
17221723

17231724

1724-
def _resolve_args_directness(
1725-
argnames: Sequence[str],
1726-
indirect: bool | Sequence[str],
1727-
nodeid: str,
1728-
) -> dict[str, Literal["indirect", "direct"]]:
1729-
"""Resolve if each parametrized argument must be considered an indirect
1730-
parameter to a fixture of the same name, or a direct parameter to the
1731-
parametrized function, based on the ``indirect`` parameter of the
1732-
parametrize() call.
1733-
1734-
:param argnames:
1735-
List of argument names passed to ``parametrize()``.
1736-
:param indirect:
1737-
Same as the ``indirect`` parameter of ``parametrize()``.
1738-
:param nodeid:
1739-
Node ID to which the parametrization is applied.
1740-
:returns:
1741-
A dict mapping each arg name to either "indirect" or "direct".
1742-
"""
1743-
arg_directness: dict[str, Literal["indirect", "direct"]]
1744-
if isinstance(indirect, bool):
1745-
arg_directness = dict.fromkeys(argnames, "indirect" if indirect else "direct")
1746-
elif isinstance(indirect, Sequence):
1747-
arg_directness = dict.fromkeys(argnames, "direct")
1748-
for arg in indirect:
1749-
if arg not in argnames:
1750-
fail(
1751-
f"In {nodeid}: indirect fixture '{arg}' doesn't exist",
1752-
pytrace=False,
1753-
)
1754-
arg_directness[arg] = "indirect"
1755-
else:
1756-
fail(
1757-
f"In {nodeid}: expected Sequence or boolean for indirect, got {type(indirect).__name__}",
1758-
pytrace=False,
1759-
)
1760-
return arg_directness
1761-
1762-
17631725
def pytest_cmdline_main(config: Config) -> int | ExitCode | None:
17641726
if config.option.showfixtures:
17651727
showfixtures(config)
@@ -2054,8 +2016,14 @@ def sort_by_scope(arg_name: str) -> Scope:
20542016

20552017
return fixturenames_closure, arg2fixturedefs
20562018

2057-
def pytest_generate_tests(self, metafunc: Metafunc) -> None:
2019+
def pytest_generate_tests(self, metafunc: ParametrizeContext) -> None:
20582020
"""Generate new tests based on parametrized fixtures used by the given metafunc"""
2021+
from _pytest.python import Metafunc
2022+
2023+
if not isinstance(metafunc, Metafunc):
2024+
# The definition's parametrization is not backed by fixtures, so
2025+
# there are no parametrized fixtures to expand.
2026+
return
20592027

20602028
def get_parametrize_mark_argnames(mark: Mark) -> Sequence[str]:
20612029
args, _ = ParameterSet._parse_parametrize_args(*mark.args, **mark.kwargs)

0 commit comments

Comments
 (0)