You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi, I am having problems understanding what I am seeing here in the below example.
Why can't mypy figure out that after the else statement, we definitely have a DataFrame?
from typing import Union
import pandas as pd
x: Union[pd.Series, pd.DataFrame, pd.Index] = pd.Series([1, 2, 3])
if isinstance(x, (pd.Series, pd.Index)):
reveal_type(x) # Revealed type is "Union[pandas.core.series.TimestampSeries, pandas.core.indexes.numeric.NumericIndex]"
dtype = x.dtype
reveal_type(dtype) # Revealed type is "Union[numpy.dtype[numpy.generic], pandas.core.dtypes.base.ExtensionDtype]"
else: # a DataFrame
reveal_type(x) # Revealed type is "Union[pandas.core.series.Series[Any], pandas.core.frame.DataFrame, pandas.core.indexes.base.Index]"
dtypes = x.dtypes # yields error: Item "Index" of "Union[Series[Any], DataFrame, Index]" has no attribute "dtypes" [union-attr]
reveal_type(dtypes) # "Union[numpy.dtype[numpy.generic], pandas.core.dtypes.base.ExtensionDtype, pandas.core.series.Series[Any], Any]"
dtypes_set = set(x.dtypes) # yield 2 errors:
# first error Item "Index" of "Union[Series[Any], DataFrame, Index]" has no attribute "dtypes" [union-attr]
# second error: Argument 1 to "set" has incompatible type "Union[dtype[generic], ExtensionDtype, Series[Any], Any]"; expected "Iterable[Any]" [arg-type]
I modified the code a bit, and this is a mypy bug. One issue is that you bind x to a Series, so the else part is never entered. So I wrapped it in a function
importpandasaspddeffoo(x: Union[pd.Series, pd.DataFrame, pd.Index]): # = pd.Series([1, 2, 3])ifisinstance(x, (pd.Series, pd.Index)):
reveal_type(x) # Revealed type is "Union[pandas.core.series.TimestampSeries, pandas.core.indexes.numeric.NumericIndex]"dtype=x.dtypereveal_type(dtype) # Revealed type is "Union[numpy.dtype[numpy.generic], pandas.core.dtypes.base.ExtensionDtype]"else: # a DataFramereveal_type(x) # Revealed type is "Union[pandas.core.series.Series[Any], pandas.core.frame.DataFrame, pandas.core.indexes.base.Index]"dtypes=x.dtypes# yields error: Item "Index" of "Union[Series[Any], DataFrame, Index]" has no attribute "dtypes" [union-attr]reveal_type(dtypes) # "Union[numpy.dtype[numpy.generic], pandas.core.dtypes.base.ExtensionDtype, pandas.core.series.Series[Any], Any]"dtypes_set=set(x.dtypes) # yield 2 errors:# first error Item "Index" of "Union[Series[Any], DataFrame, Index]" has no attribute "dtypes" [union-attr]# second error: Argument 1 to "set" has incompatible type "Union[dtype[generic], ExtensionDtype, Series[Any], Any]"; expected "Iterable[Any]" [arg-type]
pyright correctly reports:
issue516a.py:7:21 - information: Type of "x" is "Series[Unknown] | Index"
issue516a.py:9:21 - information: Type of "dtype" is "dtype[generic] | ExtensionDtype"
issue516a.py:11:21 - information: Type of "x" is "DataFrame"
issue516a.py:13:21 - information: Type of "dtypes" is "Series[Unknown]"
mypy incorrectly reports:
issue516a.py:7: note: Revealed type is "Union[pandas.core.series.TimestampSeries, pandas.core.indexes.numeric.NumericIndex]"
issue516a.py:9: note: Revealed type is "Union[numpy.dtype[numpy.generic], pandas.core.dtypes.base.ExtensionDtype]"
issue516a.py:11: note: Revealed type is "Union[pandas.core.series.Series[Any], pandas.core.frame.DataFrame, pandas.core.indexes.base.Index]"
issue516a.py:12: error: Item "Index" of "Union[Series[Any], DataFrame, Index]" has no attribute "dtypes" [union-attr]
issue516a.py:13: note: Revealed type is "Union[numpy.dtype[numpy.generic], pandas.core.dtypes.base.ExtensionDtype, pandas.core.series.Series[Any], Any]"
issue516a.py:14: error: Item "Index" of "Union[Series[Any], DataFrame, Index]" has no attribute "dtypes" [union-attr]
issue516a.py:14: error: Argument 1 to "set" has incompatible type "Union[dtype[generic], ExtensionDtype, Series[Any], Any]"; expected "Iterable[Any]" [arg-type]
With mypy 1.14.1, this now works correctly. Using the original code:
play\issue516.py:7: note: Revealed type is "Union[pandas.core.series.Series[builtins.float], pandas.core.indexes.base.Index[builtins.int]]"
play\issue516.py:9: note: Revealed type is "Union[numpy.dtype[numpy.generic[Any]], pandas.core.dtypes.base.ExtensionDtype]"
play\issue516.py:11: note: Revealed type is "pandas.core.frame.DataFrame"
play\issue516.py:13: note: Revealed type is "pandas.core.series.Series[Any]"
Activity
gandhis1 commentedon Jan 21, 2023
This would appear to be a Mypy bug. It's probably worth trying pyright as well - I suspect they've got it right. Either way, not a
pandas-stubsissue.Dr-Irv commentedon Jan 21, 2023
I modified the code a bit, and this is a mypy bug. One issue is that you bind
xto aSeries, so theelsepart is never entered. So I wrapped it in a functionpyright correctly reports:
mypy incorrectly reports:
Dr-Irv commentedon Jan 22, 2023
Created a mypy issue: python/mypy#14502
jasonboh commentedon Jan 31, 2023
thanks everyone! I have started using pyright in VS Code.
Dr-Irv commentedon Jan 31, 2023
You can also use
pyrightfrom the command line as well.Dr-Irv commentedon Feb 17, 2023
reopening so we can track if mypy ever fixes the bug
Dr-Irv commentedon Jan 3, 2025
With mypy 1.14.1, this now works correctly. Using the original code:
Closing as a result.