You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
io.fits should return table data as Astropy Tables #199
This issue here as a reminder that this work needs to be done at some point (there's also a ticket for this in PyFITS' tracker). Currently PyFITS/astropy.io.fits has its own custom array class for representing FITS table data. Since FITS tables do have a few "special" features it will probably still be necessary to have a special class specifically for FITS tables, though for the sake of consistency it should at least be based on the Astropy Table class (which is a lot cleaner and more usable than what PyFITS currently has anyways).
This won't be doable for the first release--it's a big change and will be very difficult, if not impossible, to implement in a backwards compatible manner. For the first release it's probably better that astropy.io.fits (not to mention the pyfits legacy shim) work as a drop-in replacement for PyFITS.
This work will also have to be backported to PyFITS, so I'll want to tie it to an eventual PyFITS release.
I really need to start working on this soon. I am sick to death of of people having problems with manipulating tables in pyfits. It's not their fault--given a programmable interface to a table they expect it to just work when, for example, they try to add or delete a column, or modify its format. But pyfits just never did this quite right. Or at least there aren't enough tests that the functionality has been well-maintained, if it did ever work. Using a version of the Astropy tables will finally make table manipulation just work (albeit with some inefficiencies for certain operations, but better than nothing).
Once we find someone who can spend significant effort on io.fits, I think this would be a worthy goal to revive. Since the issue was open, we now have mix-in columns, which means that a FITS Table HDU could be represented by a Table instance (or instance of a sub-class) that includes mix-in columns for e.g. the scaled data. We could also optionally use mix-in columns for bit columns used in e.g. event tables (pretty sure @taldcroft had a prototype for this) or for easier bytes/unicode conversion (#4829)
since astropy.table.Table() now supports reading/writing of FITS tables, is this issue still relevant? it seems like io.fits could easily leverage what is already there and provide an option for converting to a Table upon read.
@tepickering - I think the idea here is to use astropy.table.Table directly in io.fits, replacing the FITS_rec class. But this needs quite a big effort...
I also think it is useful to keep this kind of issue open, with more long-term ideas/vision, even if it is unlikely to happy soon given the amount of work that this requires.
I think one important consideration is back-compatibility and not breaking code in a major way. This is mentioned in the original issue description, so it would seem any path forward would involve a new interface which is inpsired by the existing direct io.fits API but using astropy table Column objects or appropriately tweaked mixin objects (as suggested earlier by @astrofrog). At that point the "classic" interface could be deprecated but likely left in for a long time. It'll be a big job but gives an opportunity for a fresh start on API and full integration with astropy table.
One day there will be a major celebration when issue #199 gets closed! Just like our #3000 celebration.
I'd like to point out that there are still some problems with round-tripping and the headers that should probably be fixed up before this happens. i.e. the comments on keywords getting clipped, as well as regular strings in the headers getting re-ordered (should probably file an actual issue for this one, although I'm not sure it's something that needs to be "fixed" or it's just the way things are). Just thought I'd mention those here :D Or would this potentially leave the headers as is, and just change the data part of the HDU to an astropy Table?
Activity
embray commentedon Nov 2, 2012
I really need to start working on this soon. I am sick to death of of people having problems with manipulating tables in pyfits. It's not their fault--given a programmable interface to a table they expect it to just work when, for example, they try to add or delete a column, or modify its format. But pyfits just never did this quite right. Or at least there aren't enough tests that the functionality has been well-maintained, if it did ever work. Using a version of the Astropy tables will finally make table manipulation just work (albeit with some inefficiencies for certain operations, but better than nothing).
astrofrog commentedon Sep 20, 2013
I don't think this will be done by 0.3.0
astrofrog commentedon Jan 14, 2015
This won't happen for 1.0 (and is not critical) so removing milestone.
Merge pull request astropy#199 from astrofrog/spherical-patches
astrofrog commentedon Dec 16, 2016
Once we find someone who can spend significant effort on io.fits, I think this would be a worthy goal to revive. Since the issue was open, we now have mix-in columns, which means that a FITS Table HDU could be represented by a Table instance (or instance of a sub-class) that includes mix-in columns for e.g. the scaled data. We could also optionally use mix-in columns for bit columns used in e.g. event tables (pretty sure @taldcroft had a prototype for this) or for easier bytes/unicode conversion (#4829)
tepickering commentedon May 1, 2018
since
astropy.table.Table()now supports reading/writing of FITS tables, is this issue still relevant? it seems likeio.fitscould easily leverage what is already there and provide an option for converting to aTableupon read.saimn commentedon May 1, 2018
@tepickering - I think the idea here is to use
astropy.table.Tabledirectly inio.fits, replacing theFITS_recclass. But this needs quite a big effort...pllim commentedon May 2, 2018
This wish is 6 years old now. I propose we close this as won't fix.
bsipocz commentedon May 2, 2018
well, I feel this is still relevant and would definitely be useful. And basically as a feature request can stay open indefinite.
pllim commentedon May 2, 2018
OK.
saimn commentedon May 2, 2018
I also think it is useful to keep this kind of issue open, with more long-term ideas/vision, even if it is unlikely to happy soon given the amount of work that this requires.
taldcroft commentedon May 2, 2018
I think one important consideration is back-compatibility and not breaking code in a major way. This is mentioned in the original issue description, so it would seem any path forward would involve a new interface which is inpsired by the existing direct
io.fitsAPI but using astropy tableColumnobjects or appropriately tweaked mixin objects (as suggested earlier by @astrofrog). At that point the "classic" interface could be deprecated but likely left in for a long time. It'll be a big job but gives an opportunity for a fresh start on API and full integration with astropy table.One day there will be a major celebration when issue #199 gets closed! Just like our #3000 celebration.
SaOgaz commentedon May 22, 2018
I'd like to point out that there are still some problems with round-tripping and the headers that should probably be fixed up before this happens. i.e. the comments on keywords getting clipped, as well as regular strings in the headers getting re-ordered (should probably file an actual issue for this one, although I'm not sure it's something that needs to be "fixed" or it's just the way things are). Just thought I'd mention those here :D Or would this potentially leave the headers as is, and just change the data part of the HDU to an astropy Table?
Adds changelog entry for astropy#199 [skip ci]
Merge pull request astropy#199 from dpshelio/doctestOut