Let user have more control over deprecation warning to exception - #6223

This looks good to me. Would be nice, but I'm not sure how we can test this.

Also, this will definitely need a changelog under astropy.tests as this is a feature for affiliates, and they will need to know about it. Maybe adding to the narrative docs is an option, too. I'm adding the milestone, but as you said it's not critical just something nice to have.

I tested this manually like as follows. First, I have a dummy module:

# filename: ztmp1.py
import warnings

warnings.warn('Oh no!', DeprecationWarning)


def myfunc():
    warnings.warn('Bozo', DeprecationWarning)


def badfunc():
    warnings.warn('Foo', DeprecationWarning)

This session demonstrates how things work by default for module import error:

>>> from astropy.tests.helper import enable_deprecations_as_exceptions, treat_deprecations_as_exceptions
>>> enable_deprecations_as_exceptions()
>>> treat_deprecations_as_exceptions()
>>> import ztmp1
# Some traceback
DeprecationWarning: Oh no!

This session demonstrate how things work with user configuration to bypass module import error but retain default function-level error:

>>> from astropy.tests.helper import enable_deprecations_as_exceptions, treat_deprecations_as_exceptions
>>> enable_deprecations_as_exceptions(modules_to_ignore_on_import=['ztmp1']))
>>> treat_deprecations_as_exceptions()
>>> import ztmp1  # No error on import now
>>> ztmp1.myfunc()  # Default function-level error
# Some traceback
DeprecationWarning: Bozo

This session demonstrate how things work with user configuration to bypass both module import and function-level errors (the latter, only for Python 3.5 or later):

>>> from astropy.tests.helper import enable_deprecations_as_exceptions, treat_deprecations_as_exceptions
>>> enable_deprecations_as_exceptions(
...     modules_to_ignore_on_import=['ztmp1'], warnings_to_ignore_by_pyver={(3, 5): ['Bozo']})
>>> treat_deprecations_as_exceptions()
>>> import ztmp1  # No error on import
>>> ztmp1.myfunc() # Still no error
>>> ztmp1.badfunc()  # Just to show it still works on others
# Some traceback
DeprecationWarning: Foo

I don't know how to turn this into a regression test suite, as I have to do each of those in a separate (fresh) Python session. I am not sure if I want to mess with the warning registry within a regression test; it could ripple through other modules.

Change log added. Given the queue, I didn't bother to redo the CI even though I rebased when I added the change log. But if you want, I can squash and force push to make CI run one more time; Just let me know.

As for narrative doc, it would be more obvious over at the package template? See astropy/package-template#259 .

@pllim - I think this is good enough for now, if someone has a cunning plan for testing it then it can be added in a follow-up PR.

Thanks! I briefly thought about using mock but seems overkill for this... 😅