Skip to content

Allow additional kwargs when reading VOTables using the Unified I/O - #11643

Merged
pllim merged 5 commits into
astropy:mainfrom
eerovaher:votable-read-kwargs
Apr 28, 2021
Merged

pllim merged 5 commits into
astropy:mainfrom
eerovaher:votable-read-kwargs

Conversation

@eerovaher

Copy link
Copy Markdown
Member

Description

Reading in VOTables with the functions

def parse_single_table(source, **kwargs):
or
def parse(source, columns=None, invalid='exception', verify=None,

allows the user to specify many keyword arguments, which are not available when using the Unified File Read/Write Interface. This pull request would allow all keyword arguments that are valid for the parse() function to be specified when calling the Table.read() or QTable.read() functions.

An example of where this would be useful is reading in Gaia archive VOTables as QTables. Calling something like QTable.read(filename, unit_format='vounit') would be enough to ensure that composite units get parsed properly.

Fixes #10791

The file 'astropy/io/votable/tests/table_test.py' contained multiple
functions that imported Table and Column from astropy.table. All those
import statements are now replaced with a single line placed among the
other import statements.
When reading in VOTables with the `astropy.io.votable.table.parse()`
function it is possible to specify many keyword arguments, and there is
no obvious reason why those arguments should not be usable through the
Unified File Read/Write Interface. This commit adds a test to check that
keyword arguments meant for `parse()` can be specified using
'Table.read()'.
If a VOTable is read using the Unified File Read/Write Interface, the
function `astropy.io.votable.table.parse()` get called through the
function `read_table_votable()` in 'astropy/io/votable/connect.py'. Both
the Unified I/O Interface and `parse()` can be called with many useful
keyword arguments. This commit enables `read_table_votable()` to pass
those keyword arguments from one to the other.
@eerovaher

Copy link
Copy Markdown
Member Author

The changes introduced in this pull request are simple enough and the functionality of the code intuitive enough that I am not sure if this would be a new feature or a bug fix. I will create the change log entry once a milestone has been assigned.

@pllim pllim added this to the v4.3 milestone Apr 27, 2021
@pllim

pllim commented Apr 27, 2021

Copy link
Copy Markdown
Member

I think it is more of a feature than bug fix, since it changes the API? I wouldn't backport this, personally.

We have changed the way we do change log, so you don't need to know the milestone first now. See https://github.com/astropy/astropy/blob/main/docs/changes/README.rst .

Thanks!

@pllim
pllim requested review from pllim and tomdonaldson April 27, 2021 04:33

@tomdonaldson tomdonaldson left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eerovaher Thanks for adding this behavior. It's definitely useful!

The changes look good to me.

@eerovaher

Copy link
Copy Markdown
Member Author
  1. Am I correct in thinking that this would be best described as an API change?
  2. What is the best location for the change log entry? All the code changes are made in astropy/io/votable/, but what changes for the users is how they can call Table.read() or QTable.read().
  3. Should I skip CI tests on the final commit that only adds the change log entry if the previous commits have already passed the tests?

@pllim

pllim commented Apr 28, 2021

Copy link
Copy Markdown
Member

Am I correct in thinking that this would be best described as an API change?

Yes, let's do that.

What is the best location for the change log entry? All the code changes are made in astropy/io/votable/, but what changes for the users is how they can call Table.read() or QTable.read().

Write your change log entry in a file named 11643.feature.rst inside https://github.com/astropy/astropy/tree/main/docs/changes/io.votable directory. Even though it affects unified I/O, the code is ultimately within io.votable. But if you feel strongly about it, I think you can also create a second entry (same file name) under https://github.com/astropy/astropy/tree/main/docs/changes/table . @saimn , did I get this right?

Should I skip CI tests on the final commit that only adds the change log entry if the previous commits have already passed the tests?

Yes. Please use the [ci skip] directive in your commit message.

Comment thread astropy/io/votable/tests/table_test.py Outdated
Comment on lines +17 to +18
from astropy.table import Column, Table
from astropy.units import UnrecognizedUnit

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder why these weren't top level before. Perhaps we used to have circular import problem? Anyways, CI passed, so this should probably be okay... 🤞

Comment thread astropy/io/votable/tests/table_test.py Outdated
# Table.read() should pass on keyword arguments meant for parse()
filename = get_pkg_data_filename('data/nonstandard_units.xml')
t = Table.read(filename, format='votable', unit_format='generic')
assert not isinstance(t['Flux1'].unit, UnrecognizedUnit)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it more explicit to test that the unit is actually what you want it to be, rather than simply not UnrecognizedUnit?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have updated the test.

@pllim pllim left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left some minor comments. I will approve after they are addressed and change log is added.

If you change the code as suggested, let's let the CI run again rather than skipping.

Thank you!

@saimn

saimn commented Apr 28, 2021

Copy link
Copy Markdown
Contributor

Yes, you can create a second fragment file, but in this case I think io.votable is sufficient.

@pllim pllim left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks!

@pllim
pllim merged commit ed5b80c into astropy:main Apr 28, 2021
@eerovaher
eerovaher deleted the votable-read-kwargs branch May 3, 2021 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

VOTable unit parsing

4 participants