Skip to content

Make bz2 import optional for io.fits - #9652

Merged
bsipocz merged 4 commits into
astropy:masterfrom
dhomeier:fix-bz2-dep
Dec 3, 2019
Merged

bsipocz merged 4 commits into
astropy:masterfrom
dhomeier:fix-bz2-dep

Conversation

@dhomeier

Copy link
Copy Markdown
Contributor

Description

Addressing #5967: removed the unconditional import bz2 from io.fits so the module itself can be imported in Python installations without the bz2 module. For these systems only the autodetection of bzip2'ed FITS files based on file signature and of course their transparent decompression will be disabled.

Notes

  • I do not have a Python interpreter built without bzip2 support; thus I only made some checks for this case by forcing the import to fail (misnaming the module).

  • It should be possible to mark the bzip2-tests xfail or check for the appropriate exceptions, if the _make_bzip2_file function for creating the test files was modified to fall back to an os.system("bzip2 -k test0.fits") call or the like. But that would probably fail on Windows, and on other systems we should perhaps not make assumptions on the availability of the command line bzip2 either, so it did not seem worth the effort to me.

  • Should this go under "API Changes" or "Bug Fixes"?

@astropy-bot astropy-bot Bot added the io.fits label Nov 22, 2019

@saimn saimn left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @dhomeier, looks mostly good except a minor comment.

Comment thread astropy/io/fits/file.py Resolved
@saimn saimn added this to the v4.0.1 milestone Nov 22, 2019
@saimn
saimn requested a review from MSeifert04 November 22, 2019 01:42
@saimn

saimn commented Nov 22, 2019

Copy link
Copy Markdown
Contributor

I think this can wait for v4.0.1 (unless there is a chance to get it in v4.0, @bsipocz ?), and would classify this as bug fix.

@bsipocz

bsipocz commented Nov 22, 2019

Copy link
Copy Markdown
Member

@saimn - 4.0.1 is fine and I actually prefer (keep the critical stuff in 4.0 so I see what are the blockers). Otherwise anything in 4.0.1 gets merged before 4.0 is out I'll remilestone them and ship in that release.

@bsipocz

bsipocz commented Nov 22, 2019

Copy link
Copy Markdown
Member

However, for this one, I'm not sure it actually needs a changelog entry?

@saimn

saimn commented Nov 22, 2019

Copy link
Copy Markdown
Contributor

Not having a changelog entry is also an option, it's a rare case and not really a bug in astropy.

@dhomeier

Copy link
Copy Markdown
Contributor Author

Just thought about adding a note to the io.fits docs though.

Comment thread astropy/io/fits/hdu/hdulist.py Resolved
Comment thread astropy/io/fits/file.py Resolved
Comment thread astropy/io/fits/hdu/hdulist.py Outdated
new_file = gzip.GzipFile(name, mode='ab+')
elif self._file.compression == 'bzip2':
if not HAS_BZ2:
raise ValueError(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See above.

@pllim pllim added the Bug label Nov 22, 2019
@MSeifert04

Copy link
Copy Markdown
Contributor

@dhomeier Could you address the open points? Or is there any point which needs further discussion?

@dhomeier

Copy link
Copy Markdown
Contributor Author

I think there is agreement on all the open points.

Comment thread astropy/utils/data.py Resolved
Comment thread astropy/utils/data.py Resolved
Comment thread docs/io/fits/index.rst Outdated
compressed with gzip, bzip2 or pkzip. Note that in this context we are talking
about a FITS file that has been compressed with one of these utilities (e.g., a
.fits.gz file).
.fits.gz file). Opening bzip2-compressed files requires the :mod:`bz2` module

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes it sound like it's an optional dependency. I think it's reasonable to expect most users to have the module so this doesn't need to be explicitly mentioned, especially not on the main index.rst.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, the whole point of the PR was to make it an optional dependency so users with the (rare) setup of a Python built without bz2 support would be able to use Astropy with the remaining functionality at all. It is sitting there quite prominently for such a fringe case, but I did not find any other documentation on the automatic (de)compression functionality.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is sitting there quite prominently for such a fringe case, but I did not find any other documentation on the automatic (de)compression functionality.

Yeah, my suggestion would be to just not add it to the documentation at all.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That would probably be the alternative, as most of the fits documentation sits in index.rst. The exception message is hopefully informative enough whenever this issue might come up in the future.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I think we don't need to mention it here, since a normal Python should come with it.

Comment thread docs/io/fits/index.rst Resolved
@MSeifert04

MSeifert04 commented Nov 29, 2019 •

Copy link
Copy Markdown
Contributor

I think this looks good now.

I'm not sure if we should squash the commits to remove the back-and-forth. What do you think @saimn?

@saimn

saimn commented Dec 3, 2019

Copy link
Copy Markdown
Contributor

Yes, looks good, thanks @dhomeier. Squashing could be useful to clean the history, if you have time, but we can also do without.

@bsipocz

bsipocz commented Dec 3, 2019

Copy link
Copy Markdown
Member

Thanks @dhomeier!

@bsipocz
bsipocz merged commit 274e66d into astropy:master Dec 3, 2019
@bsipocz bsipocz removed this from the v4.0.1 milestone Dec 3, 2019
@bsipocz bsipocz added this to the v4.0 milestone Dec 3, 2019
@dhomeier

dhomeier commented Dec 3, 2019

Copy link
Copy Markdown
Contributor Author

Thanks @MSeifert04 for the review and everyone for their comments!
Now that 7795c8d is in the history, do we need a new issue to get the get_readable_fileobj exceptions sync'ed in a future version?

@pllim

pllim commented Dec 3, 2019

Copy link
Copy Markdown
Member

do we need a new issue

@dhomeier , please go ahead and open a follow-up issue if you haven't already. Thanks!

bsipocz added a commit that referenced this pull request Dec 7, 2019
Make bz2 import optional for `io.fits`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants