TST: Use legacy numpy 1.25 print by pllim · Pull Request #15096 · astropy/astropy

pllim · 2023-07-26T17:55:18Z

Description

This pull request is to use legacy numpy 1.25 print so the new numpy 2.0 print will not break CI until we are ready to patch it properly over at #15065.

Temporary workaround for #15095

This patch is as suggested by seberg over at astropy/pyvo#464 (comment) and similar to bsipocz patch at astropy/pyvo#465 .

github-actions · 2023-07-26T17:55:58Z

github-actions · 2023-07-26T17:56:02Z

👋 Thank you for your draft pull request! Do you know that you can use [ci skip] or [skip ci] in your commit messages to skip running continuous integration tests until you are ready?

pllim · 2023-07-26T17:57:30Z

pre-commit.ci autofix

pllim · 2023-07-26T18:18:05Z

Hmm didn't fix everything; 10 of the remaining 13 failures are related.

mhvk · 2023-07-26T19:13:45Z

I can push what I had but those remaining test failures should be fixed upstream -- clearly, we missed some legacy if statement here, so pinging @seberg.

seberg · 2023-07-27T11:43:16Z

I retriggered a build with the fix so you can iterate (in 30 minutes). I hope you are not using np.cast which is now removed unless complains come in :).

pllim · 2023-07-27T15:31:18Z

Well, since you brought up np.cast, looks like scipy is using it, @seberg . 😬

Both numpy and scipy are from dev wheels hosted by Scientific Python.

gwcs/wcs.py:18: in <module>
    from scipy import linalg, optimize
scipy/__init__.py:189: in __getattr__
    return _importlib.import_module(f'scipy.{name}')
scipy/linalg/__init__.py:208: in <module>
    from ._basic import *
scipy/linalg/_basic.py:13: in <module>
    from ._decomp import _asarray_validated
scipy/linalg/_decomp.py:22: in <module>
    from numpy import (array, isfinite, inexact, nonzero, iscomplexobj, cast,
E   ImportError: cannot import name 'cast' from 'numpy' (numpy/__init__.py)

Update: Looks like scipy removed it in scipy/scipy#18973 so I guess they have to push a new dev wheel out?

seberg · 2023-07-27T15:34:49Z

Yeah, scipy is working on that, sorry. (not sure if you use it, if you do chime in your opinion if you think it should stay; personally, I actually don't think np.cast is worth the trouble of skipping the deprecation, but would rather get one too many than a few too few.)

pllim · 2023-07-27T15:36:11Z

not sure if you use it

I do not use it personally, no. But I will wait to see what the CI here says until scipy dev wheel gets updated. Thanks for the heads up!

pllim · 2023-07-27T17:44:17Z

Only one related failure left!

____________________ TestStringFunctions.test_array2string _____________________

self = <Quantity 0. Jy>, unit = Unit(dimensionless), equivalencies = []

(a lot of stuff that leads to this...)

        # But not if the formatter is not relevant for us.
>       out4 = np.array2string(self.q, separator=", ", formatter={"int": str})

astropy/units/tests/test_quantity_non_ufuncs.py:1764: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
astropy/units/quantity.py:1844: in __array_function__
    result = super().__array_function__(function, types, args, kwargs)
numpy/core/arrayprint.py:744: in array2string
    return _array2string(a, options, separator, prefix)
numpy/core/arrayprint.py:521: in wrapper
    return f(self, *args, **kwargs)
numpy/core/arrayprint.py:554: in _array2string
    lst = _formatArray(a, format_function, options['linewidth'],
numpy/core/arrayprint.py:897: in _formatArray
    return recurser(index=(),
numpy/core/arrayprint.py:853: in recurser
    word = recurser(index + (-i,), next_hanging_indent, next_width)
numpy/core/arrayprint.py:807: in recurser
    return format_function(a[index])
numpy/core/arrayprint.py:1047: in __call__
    return dragon4_positional(x,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Quantity 0. Jy>

    def __float__(self):
        try:
            return float(self.to_value(dimensionless_unscaled))
        except (UnitsError, TypeError):
>           raise TypeError(
                "only dimensionless scalar quantities can be "
                "converted to Python scalars"
            )
E           TypeError: only dimensionless scalar quantities can be converted to Python scalars

astropy/units/quantity.py:1351: TypeError

mhvk · 2023-08-11T15:51:25Z

@pllim - I pushed a fix to the units error, but am getting lots of errors due to inconsistencies between scipy and numpy (from modelling). I also updated #15065 (but that's best discussed there). It is all a bit of a mess...

pllim · 2023-08-11T15:52:16Z

@WilliamJamieson is aware of the modeling stuff but not sure when he has time to fix those...

mhvk · 2023-08-11T16:16:05Z

Agreed that we should wait out the scipy-dev being brought up to date; they did fix the exceptions being moved in scipy/scipy#19015

pllim · 2023-08-11T21:34:40Z

    else:
        # See whether it covers our dtype.
-        from numpy.core.arrayprint import _get_format_function
+        from numpy.core.arrayprint import _get_format_function, _make_options_dict


@mhvk , should we not use hidden functions from deep within numpy? I am afraid further refactoring upstream will come back to bite us here.

In principle, yes, and clearly here my initial implementation was even wrong! (since without _make_options_dict, the legacy option does not get properly translated into an integer). But the alternative is duplication of the whole arrayprint module...

Anyway, I hope this will become moot: in the initial scalar work, a public version of _get_format_function was included, and I think this is still likely to happen. Once that is there, we can just switch to using it.

so the new numpy 2.0 print will not break CI until we are ready to patch it properly.

from nddata test

because we moved it for pydata-sphinx-theme

tests for modeling when TRFLSQFitter is used

pllim · 2023-08-11T22:55:04Z

@WilliamJamieson , I went ahead and kinda ignore the new modeling warnings from scipy dev. Will open a follow-up issue for you later.

Actually I will just update #15087 when this is merged.

pllim · 2023-08-11T22:56:53Z

pre-commit.ci autofix

for more information, see https://pre-commit.ci

pllim · 2023-08-11T23:15:23Z

argh devdeps finally green...

pllim · 2023-08-12T04:53:25Z

Wut? Why would cosmology doctest fail only on s390x? @nstarman , do you know what is going on here?

_____________________ [doctest] astropy.cosmology._io.yaml _____________________
  004 
  005 This module provides functions to transform a |Cosmology| object to and from a `yaml
  006 <https://yaml.org/>`_ representation. The functions are registered with
  007 ``convert_registry`` under the format name "yaml". This format is primarily intended for
  008 use by other I/O functions, e.g. |Table|'s metadata serialization, which themselves
  009 require YAML serialization.
  010 
  011     >>> from astropy.cosmology import Planck18
  012     >>> yml = Planck18.to_format("yaml")
  013     >>> yml
  Expected:
      "!astropy.cosmology.flrw.FlatLambdaCDM\nH0: !astropy.units.Quantity\n
      unit: !astropy.units.Unit {unit: km / (Mpc s)}\n  value: 67.66\nNeff: 3.046\nOb0:
      0.04897\nOm0: 0.30966\nTcmb0: !astropy.units.Quantity\n  unit: !astropy.units.Unit
      {unit: K}\n  value: 2.7255\nm_nu: !astropy.units.Quantity\n
      unit: !astropy.units.Unit {unit: eV}\n  value: !numpy.ndarray\n
      buffer: !!binary |\n      QUFBQUFBQUFBQUFBQUFBQUFBQUFBTGdlaGV0UnVLNC8=\n
      dtype: float64\n    order: C\n    shape: !!python/tuple [3]\nmeta:
      !!python/tuple\n- !!python/tuple [Oc0, 0.2607]\n- !!python/tuple [n, 0.9665]\n-
      !!python/tuple [sigma8, 0.8102]\n- !!python/tuple [tau, 0.0561]\n- !!python/tuple\n
      - z_reion\n  - !astropy.units.Quantity\n    unit: !astropy.units.Unit {unit:
      redshift}\n    value: 7.82\n- !!python/tuple\n  - t0\n  - !astropy.units.Quantity\n
      unit: !astropy.units.Unit {unit: Gyr}\n    value: 13.787\n- !!python/tuple [flat,
      true]\n- !!python/tuple [reference, 'Planck Collaboration 2018, 2020, A&A, 641, A6
      (Paper\n    VI), Table 2 (TT, TE, EE + lowE + lensing + BAO)']\nname: Planck18\n"
  Got:
      "!astropy.cosmology.flrw.FlatLambdaCDM\nH0: !astropy.units.Quantity\n  unit: !astropy.units.Unit {unit: km / (Mpc s)}\n  value: 67.66\nNeff: 3.046\nOb0: 0.04897\nOm0: 0.30966\nTcmb0: !astropy.units.Quantity\n  unit: !astropy.units.Unit {unit: K}\n  value: 2.7255\nm_nu: !astropy.units.Quantity\n  unit: !astropy.units.Unit {unit: eV}\n  value: !numpy.ndarray\n    buffer: !!binary |\n      QUFBQUFBQUFBQUFBQUFBQUFBQUFBRCt1dUZIcmhSNjQ=\n    dtype: float64\n    order: C\n    shape: !!python/tuple [3]\nmeta: !!python/tuple\n- !!python/tuple [Oc0, 0.2607]\n- !!python/tuple [n, 0.9665]\n- !!python/tuple [sigma8, 0.8102]\n- !!python/tuple [tau, 0.0561]\n- !!python/tuple\n  - z_reion\n  - !astropy.units.Quantity\n    unit: !astropy.units.Unit {unit: redshift}\n    value: 7.82\n- !!python/tuple\n  - t0\n  - !astropy.units.Quantity\n    unit: !astropy.units.Unit {unit: Gyr}\n    value: 13.787\n- !!python/tuple [flat, true]\n- !!python/tuple [reference, 'Planck Collaboration 2018, 2020, A&A, 641, A6  (Paper\n    VI), Table 2 (TT, TE, EE + lowE + lensing + BAO)']\nname: Planck18\n"
  
  /home/runner/work/astropy/astropy/astropy/cosmology/_io/yaml.py:13: DocTestFailure
  _________________ [doctest] astropy.cosmology._io.yaml.to_yaml _________________
  205 
  206     Returns
  207     -------
  208     str
  209         :mod:`yaml` representation of |Cosmology| object
  210 
  211     Examples
  212     --------
  213     >>> from astropy.cosmology import Planck18
  214     >>> Planck18.to_format("yaml")
  Expected:
      "!astropy.cosmology.flrw.FlatLambdaCDM\nH0: !astropy.units.Quantity\n
      unit: !astropy.units.Unit {unit: km / (Mpc s)}\n  value: 67.66\nNeff: 3.046\nOb0:
      0.04897\nOm0: 0.30966\nTcmb0: !astropy.units.Quantity\n  unit: !astropy.units.Unit
      {unit: K}\n  value: 2.7255\nm_nu: !astropy.units.Quantity\n
      unit: !astropy.units.Unit {unit: eV}\n  value: !numpy.ndarray\n
      buffer: !!binary |\n      QUFBQUFBQUFBQUFBQUFBQUFBQUFBTGdlaGV0UnVLNC8=\n
      dtype: float64\n    order: C\n    shape: !!python/tuple [3]\nmeta:
      !!python/tuple\n- !!python/tuple [Oc0, 0.2607]\n- !!python/tuple [n, 0.9665]\n-
      !!python/tuple [sigma8, 0.8102]\n- !!python/tuple [tau, 0.0561]\n- !!python/tuple\n
      - z_reion\n  - !astropy.units.Quantity\n    unit: !astropy.units.Unit {unit:
      redshift}\n    value: 7.82\n- !!python/tuple\n  - t0\n  - !astropy.units.Quantity\n
      unit: !astropy.units.Unit {unit: Gyr}\n    value: 13.787\n- !!python/tuple [flat,
      true]\n- !!python/tuple [reference, 'Planck Collaboration 2018, 2020, A&A, 641, A6
      (Paper\n    VI), Table 2 (TT, TE, EE + lowE + lensing + BAO)']\nname: Planck18\n"
  Got:
      "!astropy.cosmology.flrw.FlatLambdaCDM\nH0: !astropy.units.Quantity\n  unit: !astropy.units.Unit {unit: km / (Mpc s)}\n  value: 67.66\nNeff: 3.046\nOb0: 0.04897\nOm0: 0.30966\nTcmb0: !astropy.units.Quantity\n  unit: !astropy.units.Unit {unit: K}\n  value: 2.7255\nm_nu: !astropy.units.Quantity\n  unit: !astropy.units.Unit {unit: eV}\n  value: !numpy.ndarray\n    buffer: !!binary |\n      QUFBQUFBQUFBQUFBQUFBQUFBQUFBRCt1dUZIcmhSNjQ=\n    dtype: float64\n    order: C\n    shape: !!python/tuple [3]\nmeta: !!python/tuple\n- !!python/tuple [Oc0, 0.2607]\n- !!python/tuple [n, 0.9665]\n- !!python/tuple [sigma8, 0.8102]\n- !!python/tuple [tau, 0.0561]\n- !!python/tuple\n  - z_reion\n  - !astropy.units.Quantity\n    unit: !astropy.units.Unit {unit: redshift}\n    value: 7.82\n- !!python/tuple\n  - t0\n  - !astropy.units.Quantity\n    unit: !astropy.units.Unit {unit: Gyr}\n    value: 13.787\n- !!python/tuple [flat, true]\n- !!python/tuple [reference, 'Planck Collaboration 2018, 2020, A&A, 641, A6  (Paper\n    VI), Table 2 (TT, TE, EE + lowE + lensing + BAO)']\nname: Planck18\n"
  
  /home/runner/work/astropy/astropy/astropy/cosmology/_io/yaml.py:214: DocTestFailure

nstarman · 2023-08-12T05:01:54Z

I noticed something similar to this in a recent PR (#15102) if you want to cherry-pick the fix.

mhvk

Thanks! I think these changes all look fine, and it may be best to just merge this, and work on other issues separately.

p.s. It would seem there is a case for keeping the different commits separate (for possible reversion later on).

pllim · 2023-08-12T18:55:36Z

Thanks, all! I will ~~open follow-up issues~~ update existing issues.

@nstarman , I cherry-picked as you suggested over at #15165

…096-on-v5.3.x Backport PR #15096 on branch v5.3.x (TST: Use legacy numpy 1.25 print)

pllim added Docs no-changelog-entry-needed numpy-dev backport-v5.3.x labels Jul 26, 2023

pllim added this to the v5.3.2 milestone Jul 26, 2023

github-actions Bot added testing utils labels Jul 26, 2023

pllim mentioned this pull request Jul 26, 2023

Adjust to numpy dev scalar format #15065

Merged

5 tasks done

pllim commented Jul 26, 2023

View reviewed changes

Comment thread astropy/conftest.py Resolved

pllim mentioned this pull request Jul 26, 2023

TST: numpy 2.0 float repr incompatibility in tests astropy/pyvo#464

Closed

WilliamJamieson mentioned this pull request Jul 26, 2023

convert numpy scalars to python types before yaml encoding asdf-format/asdf#1605

Merged

eerovaher mentioned this pull request Jul 31, 2023

Cosmo param dataclass #14874

Merged

This comment was marked as resolved.

Sign in to view

pllim commented Aug 11, 2023

View reviewed changes

Comment thread setup.cfg Resolved

pllim force-pushed the numpy-dev-legacy-print-hotfix branch from 12e5d13 to ea2f8e4 Compare August 11, 2023 18:34

mhvk force-pushed the numpy-dev-legacy-print-hotfix branch from ea2f8e4 to d3a463d Compare August 11, 2023 18:58

pllim added the Extra CI Run cron CI as part of PR label Aug 11, 2023

pllim commented Aug 11, 2023

View reviewed changes

pllim and others added 3 commits August 11, 2023 17:57

TST: Use legacy numpy 1.25 print

435da7b

so the new numpy 2.0 print will not break CI until we are ready to patch it properly.

Ensure we properly treat options in array2string override

79b41ae

TST: Remove np.maximum_sctype usage

5433c6d

from nddata test

pllim force-pushed the numpy-dev-legacy-print-hotfix branch from a957751 to 5433c6d Compare August 11, 2023 21:57

pllim added 3 commits August 11, 2023 18:13

TST: Update utils.misc URL for devdocs

142a7a7

because we moved it for pydata-sphinx-theme

TST: Ignore numpy warnings in blackbody tests

a3e884f

TST: Ignore numpy warnings in constraints

691b6a9

tests for modeling when TRFLSQFitter is used

pllim commented Aug 11, 2023

View reviewed changes

[pre-commit.ci] auto fixes from pre-commit.com hooks

23a30ea

for more information, see https://pre-commit.ci

mhvk approved these changes Aug 12, 2023

View reviewed changes

nstarman mentioned this pull request Aug 12, 2023

Cosmo realizations docs #15102

Merged

pllim marked this pull request as ready for review August 12, 2023 18:48

pllim requested a review from a team as a code owner August 12, 2023 18:48

pllim merged commit 4bf05d9 into astropy:main Aug 12, 2023

pllim deleted the numpy-dev-legacy-print-hotfix branch August 12, 2023 18:48

meeseeksmachine mentioned this pull request Aug 12, 2023

Backport PR #15096 on branch v5.3.x (TST: Use legacy numpy 1.25 print) #15164

Merged

meeseeksmachine pushed a commit to meeseeksmachine/astropy that referenced this pull request Aug 12, 2023

Backport PR astropy#15096: TST: Use legacy numpy 1.25 print

b6f4a3b

pllim mentioned this pull request Aug 12, 2023

TST: 3 TRFLSQFitter tests failed with RuntimeWarning with numpy-dev and scipy-dev #15087

Open

1 task

pllim added a commit that referenced this pull request Aug 12, 2023

Merge pull request #15164 from meeseeksmachine/auto-backport-of-pr-15…

2074c96

…096-on-v5.3.x Backport PR #15096 on branch v5.3.x (TST: Use legacy numpy 1.25 print)

pllim mentioned this pull request Aug 15, 2023

BUG: cadc invompatibility with dev versions of numpy astropy/astroquery#2811

Closed

mhvk mentioned this pull request Aug 21, 2023

NEP 51 transition for downstream libraries using doctest numpy/numpy#24470

Open

neutrinoceros mentioned this pull request May 9, 2024

BUG: fix yaml table serialization compatibility with numpy 2 #16416

Merged

1 task

echoix mentioned this pull request Jan 7, 2025

CI(Ubuntu): Install latest version of Python (3.13) for full config OSGeo/grass#4372

Open

Conversation

pllim commented Jul 26, 2023

Description

Uh oh!

github-actions Bot commented Jul 26, 2023

Uh oh!

github-actions Bot commented Jul 26, 2023

Uh oh!

pllim commented Jul 26, 2023

Uh oh!

Uh oh!

pllim commented Jul 26, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mhvk commented Jul 26, 2023

Uh oh!

seberg commented Jul 27, 2023

Uh oh!

pllim commented Jul 27, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

seberg commented Jul 27, 2023

Uh oh!

pllim commented Jul 27, 2023

Uh oh!

pllim commented Jul 27, 2023

Uh oh!

This comment was marked as resolved.

mhvk commented Aug 11, 2023

Uh oh!

pllim commented Aug 11, 2023

Uh oh!

This comment was marked as resolved.

Uh oh!

mhvk commented Aug 11, 2023

Uh oh!

pllim Aug 11, 2023

Choose a reason for hiding this comment

Uh oh!

mhvk Aug 12, 2023

Choose a reason for hiding this comment

Uh oh!

pllim Aug 11, 2023

Choose a reason for hiding this comment

Uh oh!

pllim Aug 11, 2023

Choose a reason for hiding this comment

Uh oh!

pllim commented Aug 11, 2023

Uh oh!

pllim commented Aug 11, 2023

Uh oh!

pllim commented Aug 12, 2023

Uh oh!

nstarman commented Aug 12, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mhvk left a comment

Choose a reason for hiding this comment

Uh oh!

pllim commented Aug 12, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

pllim commented Jul 26, 2023 •

edited

Loading

pllim commented Jul 27, 2023 •

edited

Loading

nstarman commented Aug 12, 2023 •

edited

Loading

pllim commented Aug 12, 2023 •

edited

Loading