Skip to content

Pickling an astropy.io.fits.CompImageHDU fails #10512

Closed

Description

@corbettht

Description

Attempting to pickle and reload an io.fits.CompImageHDU object produces a TypeError about a missing argument.
This causes failures when using CompImageHDUs with multiprocessing.

Expected behavior

The code below used to successfully recover the hdu object, but now produces the error below.

The configuration I have that is working is:
Linux-5.3.0-51-generic-x86_64-with-debian-buster-sid
Python 3.7.3 (default, Mar 27 2019, 22:11:17)
[GCC 7.3.0]
Numpy 1.16.4
astropy 3.2.1
Scipy 1.3.0
Matplotlib 3.1.1

Actual behavior

TypeError                                 Traceback (most recent call last)
<ipython-input-5-87dc9f19b2d1> in <module>
----> 1 a = pickle.loads(pickle.dumps(hdut))

TypeError: __new__() missing 1 required positional argument: 'table_header'

Steps to Reproduce

import astropy.io.fits as fits
import pickle

hdu = fits.CompImageHDU()
a = pickle.loads(pickle.dumps(hdu))

System Details

Darwin-19.5.0-x86_64-i386-64bit
Python 3.7.6 (default, Jan 8 2020, 13:42:34)
[Clang 4.0.1 (tags/RELEASE_401/final)]
Numpy 1.18.1
astropy 4.0.1.post1
Scipy 1.4.1
Matplotlib 3.1.3

Could also reproduce on:

Linux-5.3.0-51-generic-x86_64-with-debian-buster-sid
Python 3.7.7 (default, May 7 2020, 21:25:33)
[GCC 7.3.0]
Numpy 1.18.1
astropy 4.0.1.post1
Scipy 1.4.1
Matplotlib 3.2.1

and:

Darwin-19.5.0-x86_64-i386-64bit
Python 3.7.6 (default, Jan 8 2020, 13:42:34)
[Clang 4.0.1 (tags/RELEASE_401/final)]
Numpy 1.18.1
astropy 4.2.dev305+ga0f5bbce0
Scipy 1.4.1
Matplotlib 3.1.3

Thanks for reading, glad to provide any other information that would be helpful.

Activity

  1. pllim commented on Jun 23, 2020

    @pllim
    Member

    Thanks for reporting the known version where this used to work. Hopefully that will narrow things down a bit.

  2. pllim commented on Jun 23, 2020

    @pllim
    Member

    Hmm, I suspect it is caused by #9229 . cc @mhvk @saimn @MSeifert04

  3. saimn commented on Jun 26, 2020

    @saimn
    Contributor

    Indeed, it seems to be caused by #9229. But on the other hand it seems that HDU objects are not guaranteed to be pickable. It was working before #9229 if the HDU was instantiated manually, but it doesn't work if it is read from a file, because a file object is not pickable.
    Fixing only CompImageHeader should be easy, but making HDU objects always pickable would require a bit more work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions