How to support Quantity as a configuration item

Description

The best case scenario is that astropy.config natively supports Quantity. But the next best thing is some documentation to show the recommended work around when a configuration item has a unit attached to it.

Additional context

Motivated by spacetelescope/stsynphot_refactor#101

@pllim

@pllim I am not sure that I completely understood feature request can you please discuss in detail what is the feature request asking to implement.

👍 to allowing Quantity in configuration. As a concrete example within astropy:

astropy/astropy/utils/iers/iers.py

Lines 113 to 116 in de9db8b

    
           auto_max_age = _config.ConfigItem( 
        
               30.0, 
        
               'Maximum age (days) of predictive data before auto-downloading. ' 
        
               'Default is 30.')

Here, it would be nice if one could just give the unit, i.e., 30.0 * u.day.

This means that the astropy.config.configuration has to learn about quantities. Looking at that file, it seems validation is done by astropy.extern.configobj.validate - so one would have to see whether that module allows one to add custom validation for quantities (presumably quantity strings). It does seem like the Validator class is meant to be extendable, but it would need much more than a cursory look to make sure...

@mhvk

@mhvk , I did think about it a little but the prospect of having to modify configobj itself seems daunting. Should we be modifying a bundled code? Where does the modification go?

@astrofrog

It really depends on how the Validator object actually works, which I haven't looked into, but the docstrings do talk about being able to extend it (presumably by registering some type of validation function). Or one could subclass it. I think we'd need some of the people who wrote astropy.config to pipe in... But looking at our team, I do not know who it is! @astrofrog, @eteq?

It was mainly Mike D., I think. At least, he spearheaded the big 0.4 refactoring (seems so long ago now!).

@DShivansh

@DShivansh , while your interest is greatly appreciated, this issue requires some sort of familiarity with the project and a lot of unsupervised code-diving. If it is not the sort of work you are looking for, I recommend you try a different issue instead. Thank you!

@pllim

@pllim I am interested in doing some package-expert issues but the problem is that I am not quite familiar with the package and I am also trying to learn the use cases of astropy.config in order to better understand the problem.
P.S. I am not sure if I can handle package-expert issues yet but I am trying my best to fully understand the request and then try to solve it.

@DShivansh

@DShivansh - one advantage of this issue is that it is quite limited in scope - but you mostly will need to understand the configobj module. It might be an idea to give it an hour or two, and move on if it just seems too confused.

@mhvk

@mhvk do you have any recommendation for a package-novice to get good understanding of a package if he/she does not have a physics background.

Beyond the documentation, it probably means taking astronomy classes...

@pllim

@pllim @mhvk I think we can create different cfgtypes for different quantities and then corresponding isquantity functions inside validator to return the required value for the corresponding quantity.

@DShivansh

@DShivansh , ideally all of them. I think the solution should be generic.

@pllim

@pllim but inside validator we have to create a function corresponding to each cfgtype.
like this

def is_integer(value, min=None, max=None):
    """
    A check that tests that a given value is an integer (int, or long)
    and optionally, between bounds. A negative value is accepted, while
    a float will fail.

    If the value is a string, then the conversion is done - if possible.
    Otherwise a VdtError is raised.

    >>> vtor.check('integer', '-1')
    -1
    >>> vtor.check('integer', '0')
    0
    >>> vtor.check('integer', 9)
    9
    >>> vtor.check('integer', 'a')
    Traceback (most recent call last):
    VdtTypeError: the value "a" is of the wrong type.
    >>> vtor.check('integer', '2.2')
    Traceback (most recent call last):
    VdtTypeError: the value "2.2" is of the wrong type.
    >>> vtor.check('integer(10)', '20')
    20
    >>> vtor.check('integer(max=20)', '15')
    15
    >>> vtor.check('integer(10)', '9')
    Traceback (most recent call last):
    VdtValueTooSmallError: the value "9" is too small.
    >>> vtor.check('integer(10)', 9)
    Traceback (most recent call last):
    VdtValueTooSmallError: the value "9" is too small.
    >>> vtor.check('integer(max=20)', '35')
    Traceback (most recent call last):
    VdtValueTooBigError: the value "35" is too big.
    >>> vtor.check('integer(max=20)', 35)
    Traceback (most recent call last):
    VdtValueTooBigError: the value "35" is too big.
    >>> vtor.check('integer(0, 9)', False)
    0
    """
    (min_val, max_val) = _is_num_param(('min', 'max'), (min, max))
    if not isinstance(value, (int, long, string_type)):
        raise VdtTypeError(value)
    if isinstance(value, string_type):
        # if it's a string - does it represent an integer ?
        try:
            value = int(value)
        except ValueError:
            raise VdtTypeError(value)
    if (min_val is not None) and (value < min_val):
        raise VdtValueTooSmallError(value)
    if (max_val is not None) and (value > max_val):
        raise VdtValueTooBigError(value)
    return value

this is the function corresponding to integer cfgtype

At a high level, I am thinking something like:

Check if given unit is valid (you can parse it into Unit).
Check if given unit is compatible with the required unit.

Min/max can be a later enhancement, if necessary. I don't think we use it much? I am also not sure how much we should support equivalency in a config item.

oh, do you mean that we create a generic cfgtype named units and the function corresponding to that will check whether the provided unit is present inside the astropy.units or not and then just return the unit as it is?

Am I interpreting it correctly?

Something generic, yes. But unfortunately, I don't have time to code dive configobj to tell you the exact implementation details. That is open to research & development.

Example usage (highly theoretical and subject to change) can be like:

conf.my_quantity = 1 * u.s

But then it needs to be store in a .cfg file, so at some point, we need to parse to/from string. How do we differentiate that this string is a Quantity and not something else?

Does this make sense?

Yes it makes sense, for now, I haven't given a thought to string parsing, I am thinking first I try and implement

auto_max_age = _config.ConfigItem( 
     30.0*u.day, 
     'Maximum age (days) of predictive data before auto-downloading. ' 
     'Default is 30.')

the above example type of functionality and then think a way to parse strings, maybe hard-coding different regex at that time may help but it would be a pretty lengthy and repetitive task :(

@DShivansh

@DShivansh - I would indeed work from the test case. I think you can simply store the configuration as a string and parse it with something like Quantity(<string>, default.unit). But like @pllim I've not dived into configobj (nor will have time), so my suggestion is to try to get your example working, and if it doesn't succeed in a couple of hours, move on.

@mhvk

@mhvk I will be trying to get my example working this weekend and would let you know if I succeed.

@mhvk

@mhvk @pllim when I am trying to import quantity inside the config.py file the circular dependency occurs between the quantity and config because now they are mutually importing each other hence the error occurs. First, the interpreter goes inside config and then tries to import the units file but units file itself needs config for its proper functioning hence units give an error that some function that is inside config.py does not exist.
Do you have some ideas as to how to handle such error?

Import inline in config might do the trick.

@mhvk

@mhvk the inline import is also not working, I think I should leave the issue to someone who has more knowledge about the package than me :(

Navigation Menu

Repository navigation

How to support Quantity as a configuration item #9893

Description

Additional context

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	auto_max_age = _config.ConfigItem(
	30.0,
	'Maximum age (days) of predictive data before auto-downloading. '
	'Default is 30.')

How to support Quantity as a configuration item #9893

Description

Description

Additional context

Activity

DShivansh commented on Feb 4, 2020

mhvk commented on Feb 4, 2020

pllim commented on Feb 4, 2020

mhvk commented on Feb 4, 2020

pllim commented on Feb 4, 2020

pllim commented on Feb 5, 2020

DShivansh commented on Feb 6, 2020

mhvk commented on Feb 6, 2020

DShivansh commented on Feb 6, 2020

mhvk commented on Feb 6, 2020

DShivansh commented on Feb 6, 2020

pllim commented on Feb 6, 2020

DShivansh commented on Feb 6, 2020

pllim commented on Feb 6, 2020

DShivansh commented on Feb 6, 2020

pllim commented on Feb 6, 2020

DShivansh commented on Feb 6, 2020

mhvk commented on Feb 6, 2020

DShivansh commented on Feb 7, 2020

DShivansh commented on Feb 8, 2020

mhvk commented on Feb 8, 2020

DShivansh commented on Feb 9, 2020

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions