BUG: Sum of ndarray subclassess incorrectly typed as np.ndarray

Describe the issue:

In my code I'm using a slightly customized numpy subclass. When I sum two numpy arrays viewed as this class, the result has a different type. I'd expect the sum to be of the same type as the two addends.

Reproduce the code example:

import numpy as np


class State(np.ndarray):
    pass


a: State = np.random.rand(10, 5).view(State)
b: State = np.random.rand(10, 5).view(State)

reveal_type(a)
reveal_type(b)
reveal_type(a + b)

Error message:

test-mypy-numpy.py:12: note: Revealed type is "test-mypy-numpy.State"
test-mypy-numpy.py:13: note: Revealed type is "test-mypy-numpy.State"
test-mypy-numpy.py:14: note: Revealed type is "numpy.ndarray[Any, numpy.dtype[numpy.bool_]]"

NumPy/Python version information:

Python 3.9.5
Numpy: 1.21.2
Mypy: 0.910

It's unfortunate, but without higher-kinded typevars (python/typing#548) I see no way of fixing this issue.

The problem is that, when working with typevars, we've no way of decoupling the type from its parameters
(i.e. shape and dtype). So we can currently type that the addition of a float and int array returns a float array,
but we cannot do the same for ndarray subclasses (unless you explicitly override the subclasses' methods,
that is).

test-mypy-numpy.py:14: note: Revealed type is "numpy.ndarray[Any, numpy.dtype[numpy.bool_]]"

Interesting, the appearance of the boolean dtype most definitely looks like a mypy bug.
I would expect the type here to be ndarray[Any, Any] due to overload ambiguity, which is exactly what
happens with a: ndarray[Any, Any]; reveal_type(a + a).

Thanks @BvB936! So should I open an issue on the mypy tracker for the bool problem?

Thanks @BvB936! So should I open an issue on the mypy tracker for the bool problem?

Yes, though I'd recommend narrowing it down to a minimal reproducible example first.

Thanks @BvB936! So should I open an issue on the mypy tracker for the bool problem?

I think it is the same as #20099.

@vnmabus

I think it is the same as #20099.

@vnmabus it seems to be the case, yes.

After some trial and error I've managed to narrow the issue down (see below for a minimal example).
In short, there seem to be three requirements for triggering this mypy bug:

The function must be a method.
The method must be overloaded.
self must be annotated with a parametrized generic.

from typing import Any, Generic, overload, TypeVar, TYPE_CHECKING
import numpy as np

_CharType = TypeVar("_CharType", np.str_, np.bytes_)

class CharArray(Generic[_CharType]):
    @overload
    def strip(self: CharArray[np.str_], chars: str = ...) -> CharArray[np.str_]: ...
    @overload
    def strip(self: CharArray[np.bytes_], chars: bytes = ...) -> CharArray[np.bytes_]: ...

ar: CharArray[Any]
ar_U: CharArray[np.str_]
ar_S: CharArray[np.bytes_]

if TYPE_CHECKING:
    reveal_type(ar.strip())  # E: Revealed type is "__main__.CharArray[numpy.str_]"
    reveal_type(ar_U.strip())  # E: Revealed type is "__main__.CharArray[numpy.str_]"
    reveal_type(ar_S.strip())  # E: Revealed type is "__main__.CharArray[numpy.bytes_]"

FYI, I just opened an issue on the mypy tracker (python/mypy#11347).

This is impossible without higher kinded typing, which isn't supported in Python:
python/typing#548

Navigation Menu

Repository navigation

BUG: Sum of ndarray subclassess incorrectly typed as np.ndarray #20072

Describe the issue:

Reproduce the code example:

Error message:

NumPy/Python version information:

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

BUG: Sum of ndarray subclassess incorrectly typed as np.ndarray #20072

Description

Describe the issue:

Reproduce the code example:

Error message:

NumPy/Python version information:

Activity

BvB93 commented on Oct 9, 2021

rdbisme commented on Oct 9, 2021

BvB93 commented on Oct 11, 2021

vnmabus commented on Oct 15, 2021

BvB93 commented on Oct 16, 2021

BvB93 commented on Oct 17, 2021

jorenham commented on Jan 6, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions