Repository navigation
Allow writing of Table to cds format #11257
Description
It would be nice to be able to serialize astropy tables in CDS format (.readme and table), as it would make uploading to Vizier/CDS easier. Currently, there is already a small python module from CDS that uses astropy to create the byte-by-byte description and data: https://github.com/cds-astro/cds.pyreadme/ and it may be relatively straightforward to base a writer on that (or, at the very least, link to it in our documentation).
Xref #11239 (comment)
Activity
gilleslandais commentedon Jan 22, 2021
CDS format in astropy output is a great idea, in particular to generate the byte-by-byte. Authors are really enthousiast each time we provide a tool to create the byte-by-byte (for example, the VizieR sumission web interface – coded in java).
I would like to suggest the MRT (Machine Readable Table) rather than file+ReadMe. MRT includes the byte-by-byte in the file header ; it is the format used for AAS publication and a format very similar than CDS (MRT are more strict with NULL values)
ex : article 10.3847/1538-3881/ab58d3 , https://iopscience.iop.org/1538-3881/159/1/17/suppdata/ajab58d3t4_mrt.txt which exists also in CDS ftp://cdsarc.u-strasbg.fr/pub/cats/J/AJ/159/17)
taldcroft commentedon Feb 3, 2021
Definitely 👍 on output to a single file with embedded header information.
gilleslandais commentedon Jun 7, 2021
Hello -
some feedbacks coming from cds.pyreadme - I get a request from @pmaxted who provides ESA package for CHEOPS mission and who is interested to include cds.pyreadme as a pip dependency.
I could do it - but I could also suggest your implemntation - what is the MRT status implementation in astropy ?
taldcroft commentedon Jun 7, 2021
@gilleslandais - right now there is no progress on the MRT implementation in astropy. To be honest I was not sure if you wanted to try doing the astropy port or if we were thinking of one of the astropy devs doing that. We have a bit of a template for some of intricacies of
astropy.io.asciiwith the recent #11256 that added support for QDP reading/writing.This is a good project and worth prioritizing, so we can probably get it done by the 5.0 release (which will be in about 6 months). Unfortunately there is nothing useful available now in astropy for @pmaxted.
taldcroft commentedon Jun 8, 2021
@gilleslandais - is there a documentation page describing "MRT"? From what I infer this is just the standard ReadMe and data set for exactly one table. Is it possible to store two tables in an MRT file?
taldcroft commentedon Jun 8, 2021
Relatedly, does the current
cds.pyreadmesupport the full specification for the MRT ReadMe header as shown in http://vizier.u-strasbg.fr/doc/catstd-3.1.htx ? Is it possible to generate exactly that ReadMe via the Python interface?gilleslandais commentedon Jun 8, 2021
The cdspyreadme idea is to generate a ReadMe "template" from a list of tables (which are also standardized in aligned ascii format) -
It includes the byte-by-byte, and different sections - the current code enables to fill some of the sections: title, abstract, references
-but not all-
some (optional) section like the Object section can't be filled with the API yet - (they need to be edited by hand)
So, you can't generate exactly the same file yet..
In the case of MRT - a simple file gathering the tables values and the byte-by-byte header is may be more adapted in astropy context? (example https://iopscience.iop.org/1538-3881/159/1/17/suppdata/ajab58d3t4_mrt.txt )
gilleslandais commentedon Jun 8, 2021
MRT table is similar than CDS for the data description - it is describe @ https://journals.aas.org/mrt-standards/
I haven't see any MRT containing several tables yet - I don't think that it is accepted by AAS.