@@ -105,46 +105,102 @@ Type Checking
105105=============
106106
107107PyArrow provides type stubs (``*.pyi `` files) for static type checking. These
108- stubs are located in the ``pyarrow-stubs/ `` directory and are automatically
109- included in the distributed wheel packages.
108+ stubs are located in the ``pyarrow-stubs/ `` directory, which is laid out as a
109+ :pep: `561 ` stub-only package for ``pyarrow ``, so type checkers and IDEs pick
110+ it up from the ``arrow/python `` directory. The stubs aren't included in the
111+ distributed wheel packages until they are complete (see
112+ `GH-49831 <https://github.com/apache/arrow/issues/49831 >`_).
113+
114+ Run all the commands in this section from the ``arrow/python `` directory,
115+ which holds the type checker configurations in ``pyproject.toml ``:
116+
117+ .. code-block ::
118+
119+ $ cd arrow/python
110120
111121 Running Type Checkers
112122---------------------
113123
114124We support multiple type checkers. Their configurations are in
115125``pyproject.toml ``.
116126
127+ The easiest way to run all of them is with ``pre-commit ``, which installs
128+ the type checkers and stub packages pinned in ``requirements-stubs.txt `` (with
129+ uv). This also lints and formats the stubs with ruff:
130+
131+ .. code-block ::
132+
133+ $ pre-commit run --hook-stage manual python-stubs --all-files
134+
135+ The type checkers only run on request, while ruff also runs on commit. The
136+ Lint CI job runs all of them as well. CI also runs the type checkers, the type
137+ stub tests and stubtest (see below) against the PyArrow it builds, with
138+ ``ci/scripts/python_test_type_annotations.sh ``.
139+
140+ To run a type checker directly instead, install the pinned type checkers and
141+ stub packages in your PyArrow development environment:
142+
143+ .. code-block ::
144+
145+ $ pip install -r requirements-stubs.txt
146+
147+ Checking the PyArrow tests also needs their dependencies from
148+ ``requirements-test.txt ``.
149+
117150**mypy **
118151
119152To run mypy on the PyArrow codebase:
120153
121154.. code-block ::
122155
123- $ cd arrow/python
124156 $ mypy
125157
126158 The mypy configuration is in the ``[tool.mypy] `` section of ``pyproject.toml ``.
159+ mypy only uses the stubs when it checks them too, so pass ``pyarrow-stubs ``
160+ along with any other files you check:
161+
162+ .. code-block ::
163+
164+ $ mypy pyarrow-stubs pyarrow/tests/test_array.py
127165
128166 **pyright **
129167
130168To run pyright:
131169
132170.. code-block ::
133171
134- $ cd arrow/python
135172 $ pyright
136173
137174 The pyright configuration is in the ``[tool.pyright] `` section of ``pyproject.toml ``.
138175
139176**ty **
140177
141- To run ty (note: currently only partially configured) :
178+ To run ty:
142179
143180.. code-block ::
144181
145- $ cd arrow/python
146182 $ ty check
147183
184+ **stubtest **
185+
186+ mypy's stubtest compares the stubs with the signatures of the installed
187+ PyArrow. Run it with:
188+
189+ .. code-block ::
190+
191+ $ python scripts/run_stubtest.py
192+
193+ Stubs that aren't written yet are skipped. The known mismatches are listed in
194+ ``pyarrow-stubs-tests/stubtest_allowlist.txt ``; remove the entries of the
195+ mismatches you fix, as stubtest fails on unused entries.
196+
197+ Type Checking in an IDE
198+ -----------------------
199+
200+ Editors also run the type checkers from the folder they open. If you open the
201+ whole ``arrow `` repository in VS Code, point the type checker at ``arrow/python ``
202+ instead.
203+
148204Maintaining Type Stubs
149205-----------------------
150206
@@ -169,6 +225,43 @@ in downstream projects and for users' IDEs.
169225Note: ``py.typed `` marker file in the ``pyarrow/ `` directory indicates to type
170226checkers that PyArrow supports type checking according to :pep: `561 `.
171227
228+ Testing Type Stubs
229+ ------------------
230+
231+ The tests in ``pyarrow-stubs-tests/ `` pin down the annotated types of public
232+ APIs. Each check pairs :func: `typing.assert_type `, which every type checker
233+ verifies statically, with a runtime ``isinstance `` check:
234+
235+ .. code-block :: python
236+
237+ from typing import TYPE_CHECKING
238+
239+ import pyarrow as pa
240+
241+ from _check import assert_type, check
242+
243+ def test_list_type () -> None :
244+ lst = pa.list_(pa.int32())
245+ check(assert_type(lst, pa.ListType), pa.ListType)
246+ check(assert_type(lst.value_type, pa.DataType), pa.DataType)
247+
248+ if TYPE_CHECKING :
249+ pa.list_(1 ) # type: ignore [ call -overload ]
250+
251+ The type checkers check these files as part of the regular runs above, and
252+ pytest runs them against the installed PyArrow to make sure the stubs agree
253+ with what PyArrow actually returns:
254+
255+ .. code-block ::
256+
257+ $ pytest pyarrow-stubs-tests
258+
259+ Code under ``if TYPE_CHECKING: `` is never executed. It holds the usage that
260+ should fail type checking, which is asserted with a
261+ ``# type: ignore[<mypy code>] `` comment: unused ignores are reported as errors
262+ in these tests, so the test fails once the usage is no longer rejected. ty
263+ doesn't honor mypy error codes, so it doesn't check invalid usage.
264+
172265Doctest
173266=======
174267
0 commit comments