Repository navigation
votable binary parsing appears to fail on null columns #8737
Description
Example:
>>> import requests
>>> url = "http://vizier.u-strasbg.fr/viz-bin/votable/-b64?-source=V/127A/mash1&-out.max=20&-out.meta=hu2D"
>>> result = requests.get(url)
>>> with open('test.vot', 'w') as fh:
... fh.write(result.text)
>>> from astropy.table import Table
>>> tbl = Table.read('test.vot')
>>> print(tbl)
results in:
<Table masked=True length=18>
n_PNG PNG Name ... ObsDate img AssocData
... d
bytes1 bytes12 bytes13 ... int32 bytes5 bytes4
------ ------------ ------------- ... ---------- ------ ---------
T G209.1-08.2 PHR0615-0025 ... 808464672 fitsP G227
. 3-12.0 PHR06 33-1808 @X��� ... 1718187123 TG212 .2-0
4 .7 PHR0633-0 135 @X�z�G�� ... 1346843185 4.2-0 2.4
P HR0645-0217 @YP������a�P ... 858666541 06.8 PHR0
6 46-1235 @Yf� �,_��)2���n\B ... 858667296 PHR06 48-0
7 19 @Y��z�G�
R��,_�B
... 1346916912 650+0 013
@ Y������?�'�} '�|B� ... 892415280 257 @ Y��
�Q���
��r ... 892411968 Y�,_� ���#
� Q�
�Bp ... 1506109904 6��) Z+<M
^ oC; ... 744472613 �g��� �A�
A� E ... 801727228 �BD B<
�1012 fitsLG2 ... 155354624
3 fitsTG217.2 ... 4338176
E a tsTG214.6+02. ... 0 R
8 1015 fitsTG2 27.2-03.4 PHR ... 1 E 1016
fitsTG222.9- 01.1 PHR0705- ... 2452667 1017 fits
P G237.9-07.2 FP0711-2531 ... 825241912 fits TG22
6 .4-01.3 PHR0 711-1238 @Z�� ... 543582580 sLG22 5.2+
If you drop the 'CS' column (which is a null column), the table parses correctly, so it looks like the astropy votable is incorrectly handling the null column.
Activity
astrofrog commentedon May 20, 2019
What do you mean by a null column? It looks like CS should be a string column and it doesn't look any different to say Morph? (in terms of type). Are all the cells empty?
keflavich commentedon May 20, 2019
Yes, the cells are empty.
pllim commentedon May 20, 2019
@tomdonaldson or @tboch , since you expressed interest in maintaining
io.votable, perhaps you'll be interested to take a look?pllim commentedon May 24, 2019
Is it possible to have a version of this table in a non-binary format and see if you have the same problem parsing it? That would let us see if the bug is in binary-specific parser or not... 🤷♀
keflavich commentedon May 24, 2019
pllim commentedon May 24, 2019
There is BINARY and BINARY2 -- Did you try both?
FWIW, the binary parse is here
astropy/astropy/io/votable/tree.py
Line 2603 in 51db6a6
keflavich commentedon May 24, 2019