I could be mistaken, but I thought the "default" config file, being that it contains only sample values and does not provide the actual defaults to the library, was going to be named something like astropy.cfg.sample or the like. Then nothing needs to be commented out, and it only works once renamed or copied to astropy.cfg.

I think that plan was rejected, because it means if the user copies it to the "real" config file, and then does nothing, we can't ever change the defaults for config values because the old defaults will still be in the config file from a previous version. It seems that commented-out config entries is the only sane way to detect what the user actively changed -- it's what we do in matplotlib, and I think it works pretty well, but matplotlib has no facilities at all for saving values to the config file. If we remove that from astropy as well, we're golden -- maybe that's something to consider.

I think that if the user choose to copy the config file they are actively taking responsibility for it (whereas if they just leave the default then we can replace it without them noticing). I think we should remove the ability to save changes to the config file from code. I think that with the other restructuring we're doing, that becomes less useful, if not even dangerous.

I agree about removing the ability to save, it solves this problem quite handily.

Whether to comment out values in the template then becomes a separate question -- I think it's handy to have everything in there, even if you don't care about all of them at the time. If you have them uncommented, you have to remove the entries you don't care about if you want upgrading to be smooth. If you have everything in the template commented, only the entries you actively changed will require mental effort when upgrading.

It seems like there were no objections to removing the ability to save, so maybe we can go ahead with that assumption? (and update the APE itself too?)

@mdboom - actually I think you can ignore my comments above - this doesn't yet implement splitting configuration into global vs run-time configuration items, right?

@astrofrog: Yes -- I'm going to do that as a second phase. This is just getting the infrastructure together.

This is now ready for review. I won't have time to respond over the next week or so, but thought I'd give others a chance to comment now.

This patch ended up much larger than I had hoped -- the vast majority of the lines of code have to do with backward compatibility. I have no idea how many users even use the config file and how important that was. I, for one, will be glad to rip most of this out in the astropy 0.5 cycle.

Which config items become science state items is still up for debate, but the infrastructure is there to do just about any kind of move we want.

The unit tests are all passing here, and it's ready for a review. I suggest starting with docs/config/config_0_4_transition.rst to get a roadmap for everything that follows.

@mdboom - it looks like it needs a rebase - will you rebase after we've reviewed it? (since rebasing can sometimes remove in-line comments)

Sure. I can hold off on rebasing.

@mdboom - actually I meant that you could do it now since we haven't started reviewing it yet - but I'll leave it up to you :)

Rebased.

@mdboom - thanks! I will try and review this today or tomorrow.

@mdboom - this looks fantastic! Thank you for all your work in refactoring this!

Judging from the code, I was expecting a deprecation warning here:

In [6]: from astropy.utils.console import USE_COLOR

In [7]: USE_COLOR.set(True)

In [9]: USE_COLOR()
Out[9]: True

but didn't see one. Also, from the code, it looks like calling should emit a warning, but shouldn't we make set emit one too?

Minor bug:

In [8]: USE_COLOR
Out[8]: <repr(<astropy.config.configuration.ConfigAlias at 0x10ba39f10>) failed: KeyError: u'_items'>

@mdboom - I ran into the folllowing error:

In [1]: from astropy.coordinates import Angle
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-1-d2823155051d> in <module>()
----> 1 from astropy.coordinates import Angle

/Users/tom/Library/Python/2.7/lib/python/site-packages/astropy-0.4.dev7684-py2.7-macosx-10.8-x86_64.egg/astropy/coordinates/__init__.py in <module>()
     15 from .transformations import *
     16 from .builtin_systems import *
---> 17 from .name_resolve import *
     18 from .matching import *
     19 from .old_builtin_systems_names import *  # TODO: remove this in next version, along with module file

/Users/tom/Library/Python/2.7/lib/python/site-packages/astropy-0.4.dev7684-py2.7-macosx-10.8-x86_64.egg/astropy/coordinates/name_resolve.py in <module>()
     65 SESAME_DATABASE = state.ScienceStateAlias(
     66     "0.4", "SESAME_DATABASE", "sesame_database", sesame_database,
---> 67     cfgtype="list")
     68 
     69 

/Users/tom/Library/Python/2.7/lib/python/site-packages/astropy-0.4.dev7684-py2.7-macosx-10.8-x86_64.egg/astropy/utils/state.pyc in __init__(self, since, python_name, config_name, science_state, cfgtype, module)
    122         # file, if defined.  This is what pulls any old values in the
    123         # config file and applies them to the science state.
--> 124         value = super(ScienceStateAlias, self).__call__()
    125 
    126         # We got a value in the config file

/Users/tom/Library/Python/2.7/lib/python/site-packages/astropy-0.4.dev7684-py2.7-macosx-10.8-x86_64.egg/astropy/config/configuration.pyc in __call__(self)
    415             return self._validate_val(val)
    416         except validate.ValidateError as e:
--> 417             raise TypeError('Configuration value not valid:' + e.args[0])
    418 
    419     def _validate_val(self, val):

TypeError: Configuration value not valid:the value "all" is of the wrong type.

@astrofrog wrote:

Judging from the code, I was expecting a deprecation warning here:

I get the deprecation warning when I do that.

In [1]: from astropy.utils.console import USE_COLOR

In [2]: USE_COLOR.set(True)                                                                                                                                                                                                         
WARNING: AstropyDeprecationWarning: Since 0.4, config parameter 'astropy.utils.console.USE_COLOR' is deprecated. Use 'astropy.conf.use_color' instead. [astropy.config.configuration]

I wonder what's different about your environment...

@astrofrog: Your import error -- perhaps caused by mixing versions of astropy? Can you try a clean build and install?

EDIT: Scratch that. I see the problem, and will submit a fix.

This has been updated so that affiliated packages still using the astropy 0.3 conventions will continue to work.

Note to self: #2219 will add a new configuration item. This PR should be updated to include that config item updated for the new style.

Insane rebase completed.

Thanks! I wonder why the conflict happened in this case. Let's merge this once it passes, and before any other PR!

There were significant changes in VO, which uses config items pretty heavily, so a lot of the renames had to be redone etc.

My bad - I should have merged this first. Thanks for your work on this! I will merge this first thing tomorrow morning (once the Travis queue clears), and we should make sure we don't merge anything else in the mean time.

Pretty sure there woulda been merge conflicts either way 🤷

@mdboom - the Travis failures appear to be genuine. I'm going to restart the 3.2 build, but I think it's done that every time.

Ok, the 3.2 failure was not real, but the sphinx failure is.

The Sphinx errors are trivial to fix, so to save time, I'm going to go ahead and merge this now, then fix the warnings directly in master.

I fixed the warnings in d814e7e

Two more merge leftovers in #2436.

Thanks for cleaning these up --- sorry for rushing to get this out the door on Friday.