astropy.io.ascii and astropy.io.votable fail with this table #463

Activity

1. mdboom commented on Nov 6, 2012

   

   mdboom

   on Nov 6, 2012

   Contributor

   More actions

   I'll speak to the votable example.

   I get this exception:

   ERROR: ValueError: nph-query.xml:48:288: ValueError: invalid literal for
   int() with base 10: 'c' (i
   n row 0, col 'extd_flg') [astropy.io.votable.exceptions]

   The column "extd_flg" is declared as datatype="int", yet the value in row 0
   is "C". Perhaps this intended to be hexadecimal? In that case, it would
   need to be "0xC". (Obviously it's ambiguous to use hexadecimal without the
   "0x" prefix).

   As for null values, the header doesn't declare any null values, so the only
   null handling possible would be for completely empty cells, of which there
   are none in the first row.

   Mike

   On Tue, Nov 6, 2012 at 2:45 PM, Adam Ginsburg [email protected]:

   From an IRSA request on the 2MASS catalog:
   io.ascii.Ipac fails to parse this:

   http://irsa.ipac.caltech.edu/cgi-bin/Gator/nph-query?size=300&catalog=pt_src_cat&outfmt=1&objstr=83.808+-5.391&spatial=Box

   votable fails to parse this:

   http://irsa.ipac.caltech.edu/cgi-bin/Gator/nph-query?size=300&catalog=pt_src_cat&outfmt=3&objstr=83.808+-5.391&spatial=Box

   In both cases, I think it is a failed treatment of 'null' values.

   —
   Reply to this email directly or view it on GitHubhttps://github.com//issues/463.

   Michael Droettboom
   http://www.droettboom.com/
2. keflavich commented on Nov 6, 2012

   

   keflavich

   on Nov 6, 2012

   ContributorAuthor

   More actions

   @mdboom So is this an issue we ought to take up with IPAC?
3. mdboom commented on Nov 6, 2012

   

   mdboom

   on Nov 6, 2012

   Contributor

   More actions

   There's definitely a number of things funky with the file. It's well-formed either:

   >xmllint nph-query.xml 
   nph-query.xml:249: parser error : Opening and ending tag mismatch: TR line 247 and TABLEDATA
   </TABLEDATA>
               ^
   nph-query.xml:250: parser error : Opening and ending tag mismatch: TABLEDATA line 47 and DATA
   </DATA>
          ^
   nph-query.xml:251: parser error : Opening and ending tag mismatch: DATA line 46 and TABLE
   </TABLE>
           ^
   nph-query.xml:252: parser error : Opening and ending tag mismatch: TABLE line 19 and RESOURCE
   </RESOURCE>
              ^
   nph-query.xml:253: parser error : Opening and ending tag mismatch: RESOURCE line 7 and VOTABLE
   </VOTABLE>
             ^
   nph-query.xml:254: parser error : Premature end of data in tag VOTABLE line 3
   
   ^
   
4. mdboom commented on Nov 6, 2012

   

   mdboom

   on Nov 6, 2012

   Contributor

   More actions

   This should probably be reported to the authors of the software that generated the file.
5. taldcroft commented on Nov 7, 2012

   

   taldcroft

   on Nov 7, 2012

   Member

   More actions

   Regarding the io.ascii failure, this table can be read using:

   In [8]: dat = ascii.read('ipac.dat', fill_values=('null', '-99'))

   In this case the root cause is that the data file doesn't define a specific null value for columns that are filled with "null". However, with the new masked Tables it will be possible to cleanly read this and just define those table elements as masked without defining a fill value. I'll do this.

   There is a related issue #361 which I plan to address at the same time.
6. keflavich commented on Nov 7, 2012

   

   keflavich

   on Nov 7, 2012

   ContributorAuthor

   More actions

   Thanks Tom. That's helpful. FYI, I reported the VOtable bug to IPAC and they said they're on it - no solution yet, but I expect one is forthcoming.
7. keflavich commented on Nov 10, 2012

   

   keflavich

   on Nov 10, 2012

   ContributorAuthor

   More actions

   IPAC reports that the votable bug is fixed; I haven't tested yet.
8. embray commented on Jan 11, 2013

   

   embray

   on Jan 11, 2013

   Member

   More actions

   This was fixed with #483.
9. embray

   closed this as completedon Jan 11, 2013
10. keflavich

    added 2 commits that reference this issue on Oct 9, 2013

    Correctly support IPAC table null values

    ...

    c27ff78

    Merge pull request astropy#483 from taldcroft/ascii-table

    ...

    a9cc271
11. jeffjennings

    added a commit that references this issue on Jul 2, 2025

    Merge pull request astropy#463 from dhomeier/omnifit-2021

    ...

    0404cda