Repository navigation
BUG: #1786 resume supporting numpy 2.5 - #1802
Conversation
6eb620e to
10b0629
Compare
numpy 2.5numpy 2.5
loicdiridollou
left a comment
There was a problem hiding this comment.
That sounds like a lot of work and I am a bit worried of loosing track about all the changes and exceptions that were carved out here to make it compliant with numpy 2.5.
Is it something that is bound to stay with numpy? Or later get fixed in one way or another?
e63894f to
23a4987
Compare
…ge, for python 3.12 and 3.13, numpy 2.4 will be installed
Dr-Irv
left a comment
There was a problem hiding this comment.
So I figured out most of the problems and created a mypy issue that explains why we have to do the tests for python 3.12 in various places.
for pd.array, if you comment out:
@overload
def array(
data: Sequence[Any], dtype: BuiltinObjectDtypeArg | None = None, copy: bool = True
) -> NumpyExtensionArray: ...then we get the correct results for all of the pd.array() calls where you needed that test.
And there are other places too (e.g., Timedelta.__eq__(), that have the same issue. So if they fix that in mypy, we won't need any of the funky tests.
I think it is worth changing the tests that say if sys.version_info >= (3, 12): to use a constant NUMPY_GE_251 that is defined in tests/__init__.py to be sys.version_info >= (3, 12), then it is clear that what we are testing for is the newer version of numpy, not for the difference in the python version.
| DatetimeArray, | ||
| ) | ||
| if sys.version_info >= (3, 12): | ||
| # TODO: pandas-dev/pandas-stubs#1786 |
cmp0xff
left a comment
There was a problem hiding this comment.
Thank you @Dr-Irv , I planned to postpone further tunings to the next PRs, but you have already done much of the research.
I think it is worth changing the tests that say
if sys.version_info >= (3, 12):to use a constantNUMPY_GE_251that is defined intests/__init__.pyto besys.version_info >= (3, 12), then it is clear that what we are testing for is the newer version of numpy, not for the difference in the python version.
Unfortunately type checkers do not use such a variable to skip conditional branches, they skip only when seeing if sys.version_info >= (3, 12) explicitly, not when seeing if NUMPY_GE_251 where NUMPY_GE_251 = sys.version_info >= (3, 12).
| dt_nat = cast(list[np.datetime64 | NaTType], [np_dt, pd.NaT]) | ||
| check(assert_type(pd.array(dt_nat), DatetimeArray), DatetimeArray) | ||
| if sys.version_info >= (3, 12): | ||
| # TODO: pandas-dev/pandas-stubs#1786 |
| eq_1d1 = check(assert_type(td == arr_1d, np_1darray_bool), np_1darray_bool) | ||
| ne_1d1 = check(assert_type(td != arr_1d, np_1darray_bool), np_1darray_bool) | ||
| assert (eq_1d1 != ne_1d1).all() | ||
| # TODO: pandas-dev/pandas-stubs#1786 facebook/pyrefly#3977 |
Dr-Irv
left a comment
There was a problem hiding this comment.
Unfortunately type checkers do not use such a variable to skip conditional branches, they skip only when seeing if sys.version_info >= (3, 12) explicitly, not when seeing if NUMPY_GE_251 where NUMPY_GE_251 = sys.version_info >= (3, 12).
OK, but I think we need better comments here as opposed to just cross-referencing the mypy issue or a pyrefly issue.
Every time that you have
if sys.version_info >= (3, 12):there are one of 2 cases:
- We need the test because
numpy >= 2.5has broken behavior withmypyas referred to in the issue, and with python 3.12, we are usingnumpy >= 2.5 - We need the test because
numpy >= 2.5gives us the right behavior, and we are usingnumpy=2.4.6with python 3.11 so we have to add ignores to deal with that.
So I'd prefer a comment in each case (in addition to the cross referencing comments) that says which of those 2 cases is applicable.
Could be as simple as doing
if sys.version_info >= (3, 12): # numpy >= 2.5so that we remember the reason is because of the numpy versioning, NOT because of the python version.
| DatetimeArray, | ||
| ) | ||
| if sys.version_info >= (3, 12): | ||
| # TODO: pandas-dev/pandas-stubs#1786 |
| dt_nat = cast(list[np.datetime64 | NaTType], [np_dt, pd.NaT]) | ||
| check(assert_type(pd.array(dt_nat), DatetimeArray), DatetimeArray) | ||
| if sys.version_info >= (3, 12): | ||
| # TODO: pandas-dev/pandas-stubs#1786 |
| eq_1d1 = check(assert_type(td == arr_1d, np_1darray_bool), np_1darray_bool) | ||
| ne_1d1 = check(assert_type(td != arr_1d, np_1darray_bool), np_1darray_bool) | ||
| assert (eq_1d1 != ne_1d1).all() | ||
| # TODO: pandas-dev/pandas-stubs#1786 facebook/pyrefly#3977 |
…es and synthesized domain chronicles (pandas-dev#1274, pandas-dev#1273, pandas-dev#1867, pandas-dev#1462, pandas-dev#1452, pandas-dev#1385, pandas-dev#1275, pandas-dev#1802, pandas-dev#1397, pandas-dev#848, pandas-dev#282, pandas-dev#238, pandas-dev#1643) Co-authored-by: Antigravity AI <[email protected]>
* fix: partially support numpy 2-5 * fix: typing * fix: typing * fix: rebase * fix: pyrefly * fix: tests * fix: installed stubs * fix: https://github.com/pandas-dev/pandas-stubs/actions/runs/28395061571 * remove _dist discrepency * force numpy 2.5 for python >= 3.12 for local stubs; without this change, for python 3.12 and 3.13, numpy 2.4 will be installed * conditional warnings * pyrefly issue * python/mypy#21733 * conditional warnings revisited * deduplicate + black * regressions in pyrefly * pandas-dev#1802 (review) * revert a non-functional change
…es and synthesized domain chronicles (pandas-dev#1274, pandas-dev#1273, pandas-dev#1867, pandas-dev#1462, pandas-dev#1452, pandas-dev#1385, pandas-dev#1275, pandas-dev#1802, pandas-dev#1397, pandas-dev#848, pandas-dev#282, pandas-dev#238, pandas-dev#1643) Co-authored-by: Antigravity AI <[email protected]>
…es and synthesized domain chronicles (pandas-dev#1274, pandas-dev#1273, pandas-dev#1867, pandas-dev#1462, pandas-dev#1452, pandas-dev#1385, pandas-dev#1275, pandas-dev#1802, pandas-dev#1397, pandas-dev#848, pandas-dev#282, pandas-dev#238, pandas-dev#1643) Co-authored-by: Antigravity AI <[email protected]>
mypyraises errors afternumpy2.5.0 #1786assert_type()to assert the type of any return value)AGENTS.md.