Skip to content

numpy-dev using a dtype with a subarray field is deprecated #10815

Closed

Description

@pllim

Description

@astrofrog reported DeprecationWarning: using a dtype with a subarray field is deprecated. warnings using numpy-dev in v4.0.x (example log). Since v4.0.x is installing numpy from master, while master installs from "nightly" (i.e., weekly) wheel, we won't see it until numpy's wheel is updated.

Caused by numpy/numpy#17419

Steps to Reproduce

I think this might trigger the warning:

>>> import numpy as np
>>> from astropy.io.votable import converters, tree
>>> config = {'verify': 'exception'}
>>> field = tree.Field(None, name='c', datatype='floatComplex', arraysize='2x3*', config=config)
>>> c = converters.get_converter(field, config=config)
>>> c._base.format
'(2,)c8'
>>> default = np.array([], dtype=c._base.format)  # Warning?
E       DeprecationWarning: using a dtype with a subarray field is deprecated.
This can lead to inconsistent behaviour due to the resulting dtype being different from the input dtype.
You may try to use `dtype=dtype.base`, which should give the same result for most inputs,
but does not guarantee the output dimensions to match the subarray ones.  (Deprecated NumPy 1.20)

In this case, c._base.format is a string, so I am not sure what dtype.base means in this context. @mhvk , any advise?

Activity

  1. seberg commented on Oct 8, 2020

    @seberg
    Contributor

    Thanks for the ping, I hope this will be easy to solve, these type of deprecations are not easy decisions :( (the issue is that the current behaviour is inconsistent when data is an array, or includes arrays). We could try to walk that line differently, maybe...

    In the examples of the build log, it seems the array always consists of only an empty list, but I assume that is not necessary, since an np.empty(0) call would handle that fine as well? Do you know what data is, or is it arbitrary user input?

  2. pllim commented on Oct 8, 2020

    @pllim
    MemberAuthor

    Probably falls under "arbitrary," as this code is set up to parse in any number of XML data files meeting the VOTable format standards.

  3. seberg commented on Oct 8, 2020

    @seberg
    Contributor

    @pllim but in that case you probably have some control over it? I.e. it might always be an array, or always a sequence of tuples that has the correct shape?

  4. pllim commented on Oct 8, 2020

    @pllim
    MemberAuthor

    you probably have some control over it?

    Theoretically, yes.

  5. mhvk commented on Oct 8, 2020

    @mhvk
    Contributor

    @pllim - the fix that @seberg suggested (replacing np.array([], dtype=) with np.empty(0, dtype=) does seem to work. I'll make a PR...

    p.s. This solves a real annoyance with numpy:

    np.array([], dtype='(2,)c8')
    # array([], shape=(0, 2), dtype=complex64)
    

    So, magically, the output dtype is not the input dtype!

  6. mhvk commented on Oct 8, 2020

    @mhvk
    Contributor

    Oops, it solves the astropy.io.votable problems - I'll now look at the astropy.table ones....

  7. pllim commented on Oct 8, 2020

    @pllim
    MemberAuthor

    Thank you, all! Looking forward to the PR. 😄

  8. mhvk commented on Oct 8, 2020

    @mhvk
    Contributor

    @seberg - while I can fix the failures, I do wonder about the more general case - I thought it would make more sense as a new issue, so numpy/numpy#17511

  9. mhvk commented on Oct 29, 2020

    @mhvk
    Contributor

    I think this will have been fixed by numpy/numpy#17596, so we need to look out whether the deprecation warning error disappears once that hits "nightly".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions