Consistent handling of VOTable duplicate column names across different sub-formats #20741

Activity

1. pllim

   added

   io.votable

   on Oct 28, 2026
2. jdavies-st commented on Oct 29, 2026

   

   jdavies-st

   on Oct 29, 2026

   Contributor

   More actions

   I'm just putting my comment from #20513 (comment) here with slight edits.

   ---

   So after some poking around, trying to understand what's going on, I see that since astropy 7.0.0, when the format is tabledata_format="tabledata" and you list the same column twice, you get it back once, and it is silent on the duplication. So #20513 is just aligning binary/binary2 formats with existing tabledata format behavior.

   Before astropy 7, all formats (tabledata/binary/binary2) returned a masked array table with just the single column having data, or (nans), and this goes way back to when columns=[...] was introduced. So it has been this way for years. It was only the astropy 7+ fix of returning a table without the columns instead of as masked that introduced the regression fixed in #20513.

   On main and tag 7.2.2:

   from astropy.table import Table
   from astropy.io.votable import parse_single_table
   
   Table({"a": [1.0, 2.0], "b": [3.0, 4.0]}).write(
       "test.vot", format="votable", tabledata_format="tabledata"
   )
   
   parsed = parse_single_table("test.vot", columns=["a", "a"])
   print(parsed)
   #  a 
   # ---
   # 1.0
   # 2.0

   With astropy 6, tabledata format returned a masked array table when you asked for "a" twice:

    a   b 
   --- ---
   1.0  --
   2.0  --
   

   and binary/binary2 format returned nans:

    a   b 
   --- ---
   1.0 nan
   2.0 nan
   

   So it has never raised except for binary/binary2 format in astropy 7+, which we fixed in #20513.

   I think if we wanted it to raise an exception intentionally, then we should change that for all 3 formats, and it would be an API change.

   👍React with 👍1Reacted by P. L. Lim
3. jdavies-st commented on Oct 29, 2026

   

   jdavies-st

   on Oct 29, 2026

   Contributor

   More actions

   Further

   ---

   If I pass Table.read("file.vot", format="votable", columns=["a", "b"]) and the input is a file on disk, then the file is parsed with columns=... and my resulting table has 2 columns. If I pass Table.read() an already-parsed VOTableFile, then all those kwargs are ignored, including columns=.... So if the table has more than 2 columns, you get back all of them. A small quirk, but it is because columns only gets passed to parse().

   👍React with 👍1Reacted by P. L. Lim
4. darshpatel1052 commented on Nov 3, 2026

   

   darshpatel1052

   on Nov 3, 2026

   More actions

   I’d like to help with this. I’d first add a test matrix for repeated column requests across TABLEDATA, BINARY and BINARY2, keeping #20513’s fix intact. That should make the remaining policy decision explicit. Should duplicates continue to be silently deduplicated in every format, or should a warning/error be considered? I’d keep the already-parsed VOTableFile kwargs behaviour as a separate follow-up unless you prefer otherwise.
5. jdavies-st commented on Nov 6, 2026

   

   jdavies-st

   on Nov 6, 2026

   Contributor

   More actions

   I actually don't think any behavior needs to change. It might be useful for the user to get back a warning, indicating that they requested a column twice, which will allow them to fix their code. Other than that, column order is what it is in the file, or returned stream from a service, and as long as the columns= parameter is documented as such (it is now), then it is clear what it can and can't do.