Skip to content

sphinx.ext.autodoc: Typehints, aliases, and (internal/external) cross-references #10794

Open

Description

@kratsg

Describe the bug

For this issue, the code for the project is found here with conf.py and documentation rendered here.

In the code, we have lines like this for our typehints:

T = TypeVar("T", bound=NBitBase)

Tensor = Union["NDArray[np.number[T]]", "NDArray[np.bool_]"]
FloatIntOrBool = Literal["float", "int", "bool"]
...
...
    def astensor(
        self, tensor_in: ArrayLike, dtype: FloatIntOrBool = 'float'
    ) -> ArrayLike:
        """
        Convert to a NumPy array.
        Example:
            >>> import pyhf
            >>> pyhf.set_backend("numpy")
            >>> tensor = pyhf.tensorlib.astensor([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
            >>> tensor
            array([[1., 2., 3.],
                   [4., 5., 6.]])
            >>> type(tensor)
            <class 'numpy.ndarray'>
        Args:
            tensor_in (Number or Tensor): Tensor object
        Returns:
            `numpy.ndarray`: A multi-dimensional, fixed-size homogeneous array.
        """

which renders as

Screen Shot 2022-08-29 at 7 43 36 AM

and

    def percentile(
        self,
        tensor_in: Tensor[T],
        q: Tensor[T],
        axis: None | Shape = None,
        interpolation: Literal[
            "linear", "lower", "higher", "midpoint", "nearest"
        ] = "linear",
    ) -> ArrayLike:
        r"""
        Compute the :math:`q`-th percentile of the tensor along the specified axis.
        Example:
            >>> import pyhf
            >>> pyhf.set_backend("numpy")
            >>> a = pyhf.tensorlib.astensor([[10, 7, 4], [3, 2, 1]])
            >>> pyhf.tensorlib.percentile(a, 50)
            3.5
            >>> pyhf.tensorlib.percentile(a, 50, axis=1)
            array([7., 2.])
        Args:
            tensor_in (`tensor`): The tensor containing the data
            q (:obj:`float` or `tensor`): The :math:`q`-th percentile to compute
            axis (`number` or `tensor`): The dimensions along which to compute
            interpolation (:obj:`str`): The interpolation method to use when the
             desired percentile lies between two data points ``i < j``:
                - ``'linear'``: ``i + (j - i) * fraction``, where ``fraction`` is the
                  fractional part of the index surrounded by ``i`` and ``j``.
                - ``'lower'``: ``i``.
                - ``'higher'``: ``j``.
                - ``'midpoint'``: ``(i + j) / 2``.
                - ``'nearest'``: ``i`` or ``j``, whichever is nearest.
        Returns:
            NumPy ndarray: The value of the :math:`q`-th percentile of the tensor along the specified axis.
        """

which renders as

Screen Shot 2022-08-29 at 7 44 09 AM

And finally, in conf.py, we have the aliases defined such as 'ArrayLike': 'numpy.typing.ArrayLike'

_type_aliases_inverted = {
    'pyhf.typing': [
        'PathOrStr',
        'ParameterBase',
        'Parameter',
        'Measurement',
        'ModifierBase',
        'NormSys',
        'NormFactor',
        'HistoSys',
        'StatError',
        'ShapeSys',
        'ShapeFactor',
        'LumiSys',
        'Modifier',
        'Sample',
        'Channel',
        'Observation',
        'Workspace',
        'Literal',
    ],
    'numpy.typing': ['ArrayLike', 'DTypeLike', 'NBitBase', 'NDArray'],
}
autodoc_type_aliases = {
    item: f'{k}.{item}' for k, v in _type_aliases_inverted.items() for item in v
}

autodoc_typehints_format = 'fully-qualified'

Here are the issues

  1. In astensor, Literal gets linked to typing.Literal instead of pyhf.typing.Literal (which is fine! but in other python versions, this comes from typing_extensions which is why we import from pyhf.typing which has the code for switching where it comes from) since the docs are rendered using python 3.10. In percentile, Literal does not get cross-referenced, but why?
  2. In astensor, numpy.typing.ArrayLike is not getting cross-referenced at all, but it exists in the numpy intersphinx.
  3. In percentile, Tensor is not getting cross-referenced (from the same module) and Shape is not getting cross-referenced (from pyhf.typing in the same package).

How to Reproduce

Recommend python 3.10+ (Sphinx does not correctly process UnionType in older python versions).

$ git clone https://github.com/scikit-hep/pyhf
$ cd pyhf
$ python -m pip install --upgrade pip setuptools wheel
$ python -m pip --quiet install --upgrade .[docs,test]
$ cd docs
$ make html
$ open _build/html/index.html

Expected behavior

Expected type aliases to be processed with intersphinx (but also internal cross-referencing working too).

Your project

https://pyhf--1969.org.readthedocs.build/en/1969

Screenshots

No response

OS

Mac 10.14.6 (18G9323)

Python version

Python 3.8.6

Sphinx version

5.1.1

Sphinx extensions

'sphinx.ext.autodoc', 'sphinx.ext.autosummary', 'sphinx.ext.coverage', 'sphinx.ext.mathjax', 'sphinx.ext.ifconfig', 'sphinx.ext.viewcode', 'sphinx.ext.githubpages', 'sphinx.ext.intersphinx', 'sphinxcontrib.bibtex', 'sphinx.ext.napoleon', 'sphinx_click.ext', 'nbsphinx', 'sphinx_issues', 'sphinx_copybutton', 'sphinx_togglebutton', 'xref',

Extra tools

No response

Additional context

Related:

Activity

  1. added this to the some future version milestone on Sep 29, 2022
  2. mscuthbert commented on Oct 8, 2022

    @mscuthbert

    Agreed on the importance of this request.

    Also it would be good if argument annotations were not typed with the same class as the argument name itself. It would mkae it much easier to give a different style to type declarations or to use a callback to fi the problems that @kratsg mentions abouve.

  3. cjw296 commented on Dec 2, 2022

    @cjw296

    Seems like there might be overlap between this issue and #10785

  4. jdillard commented on Aug 18, 2026

    @jdillard
    SponsorMember

    After investigating the three reported behaviors, they appear to have separate causes:

    1. The inconsistent Literal linking is still reproducible and appears to be tracked more specifically by autodoc_type_aliases not working with typing.Literal type aliases... #10525.
    2. numpy.typing.ArrayLike is an intersphinx py:class-to-py:data mismatch. The equivalent local mismatch is fixed by Add py:class to py:data fallback for type alias cross-references (#10785) #13808, but the intersphinx fallback remains unresolved. I added a detailed reproduction and proposed fix to Use of TypeVar results in "reference target not found" error #10974, which also has reports of this external behavior.
    3. The Tensor and Shape cases do not appear to be reference-resolution failures. In the linked pyhf revision, neither alias is included in autodoc_type_aliases, and the generated API documentation does not register either as an indexed object. Sphinx therefore has no target to resolve. The local fallback in Add py:class to py:data fallback for type alias cross-references (#10785) #13808 only helps when an alias is already documented as py:data or py:attr.

    I do not think a single fix covers all three reports. I suggest tracking the Literal behavior in #10525 and the intersphinx fallback in #10974.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions