Skip to content

v1.0rc2 testing errors in modeling during second test run #3499

New issue
New issue
Closed
Closed
v1.0rc2 testing errors in modeling during second test run#3499
Labels
Bugtesting

Description

@bsipocz
bsipocz
opened on Feb 12, 2015

I got errors on all the systems I tested when testing with the option remote_data=True unrelated to remote_data

========================================================================================= ERRORS =========================================================================================
_________________________________________________________________ ERROR collecting astropy/modeling/tests/test_models.py _________________________________________________________________
_pytest.runner:137: in __init__
>   ???
_pytest.main:418: in _memocollect
>   ???
_pytest.main:295: in _memoizedcall
>   ???
_pytest.main:418: in <lambda>
>   ???
_pytest.python:527: in collect
>   ???
_pytest.python:321: in collect
>   ???
_pytest.python:333: in makeitem
>   ???
_pytest.main:161: in call_matching_hooks
>   ???
_pytest.core:380: in pcall
>   ???
_pytest.core:387: in _docall
>   ???
_pytest.core:288: in execute
>   ???
_pytest.python:227: in pytest_pycollect_makeitem
>   ???
_pytest.python:349: in _genfunctions
>   ???
_pytest.core:380: in pcall
>   ???
_pytest.core:387: in _docall
>   ???
_pytest.core:288: in execute
>   ???
_pytest.python:145: in pytest_generate_tests
>   ???
_pytest.python:817: in parametrize
>   ???
_pytest.python:698: in setmulti
>   ???
_pytest.python:681: in _checkargnotcontained
>   ???
E   ValueError: duplicate u'model_class'
_________________________________________________________________ ERROR collecting astropy/modeling/tests/test_models.py _________________________________________________________________
_pytest.runner:137: in __init__
>   ???
_pytest.main:418: in _memocollect
>   ???
_pytest.main:295: in _memoizedcall
>   ???
_pytest.main:418: in <lambda>
>   ???
_pytest.python:527: in collect
>   ???
_pytest.python:321: in collect
>   ???
_pytest.python:333: in makeitem
>   ???
_pytest.main:161: in call_matching_hooks
>   ???
_pytest.core:380: in pcall
>   ???
_pytest.core:387: in _docall
>   ???
_pytest.core:288: in execute
>   ???
_pytest.python:227: in pytest_pycollect_makeitem
>   ???
_pytest.python:349: in _genfunctions
>   ???
_pytest.core:380: in pcall
>   ???
_pytest.core:387: in _docall
>   ???
_pytest.core:288: in execute
>   ???
_pytest.python:145: in pytest_generate_tests
>   ???
_pytest.python:817: in parametrize
>   ???
_pytest.python:698: in setmulti
>   ???
_pytest.python:681: in _checkargnotcontained
>   ???
E   ValueError: duplicate u'model_class'

One of the full logs: https://gist.github.com/bsipocz/7880e511664a5c30abd1

Activity

  1. bsipocz commented on Feb 12, 2015

    @bsipocz
    bsipocz
    on Feb 12, 2015
    MemberAuthor

    This must be something trivial, and not related to remote_data, as it only happens for a later astropy.test() call, but never for the first one from the given interpreter.

  2. bsipocz
    changed the title [-]v1.0rc2 remote_data testing errors in modeling[/-] [+]v1.0rc2 testing errors in modeling during second test run[/+] on Feb 12, 2015
  3. embray
    added
    testing
    on Feb 13, 2015

  4. embray commented on Feb 13, 2015

    @embray
    embray
    on Feb 13, 2015
    Member

    Yeah there's nothing in there that would have to do with remote_data. The other failures maybe, but that's part of why remote_data is disabled by default. That's bizarre though--it looks like it has more to do with py.test than anything else. I'll see if I can reproduce.

    If I understand correctly, this is just from running astropy.test() twice in the same session?

  5. bsipocz commented on Feb 13, 2015

    @bsipocz
    bsipocz
    on Feb 13, 2015
    MemberAuthor

    @embray - yes, it's running astropy.test() twice. The other failures
    are due to the error, and I got no extra failures even for remote_data
    when run it first in a session.

    On 13 February 2015 at 17:24, Erik Bray [email protected] wrote:

    Yeah there's nothing in there that would have to do with remote_data. The
    other failures maybe, but that's part of why remote_data is disabled by
    default. That's bizarre though--it looks like it has more to do with
    py.test than anything else. I'll see if I can reproduce.

    If I understand correctly, this is just from running astropy.test() twice
    in the same session?

    —
    Reply to this email directly or view it on GitHub
    #3499 (comment).

  6. astrofrog commented on Feb 14, 2015

    @astrofrog
    astrofrog
    on Feb 14, 2015
    Member

    I can confirm the issue with:

    $ python -c "import astropy; astropy.test(package='modeling'); astropy.test(package='modeling')"

    weird!

  7. astrofrog commented on Feb 14, 2015

    @astrofrog
    astrofrog
    on Feb 14, 2015
    Member

    Hmm for some reason pytest is writing a __pycache__ directory to astropy/modeling when running the tests (but should be writing to a temporary directory) - unrelated, removing this between the two test runs changes nothing

  8. astrofrog commented on Feb 14, 2015

    @astrofrog
    astrofrog
    on Feb 14, 2015
    Member

    Minimal failing example:

    Put following in test_double.py

    import pytest

class TestA(object):
    pass

@pytest.mark.parametrize(('model_class'), [1,2,3])
class TestB(TestA):
    def test_1(self, model_class):
        pass

    Run with py.test test_double.py test_double.py

    Gives:

    E   ValueError: duplicate 'model_class'

  9. astrofrog commented on Feb 14, 2015

    @astrofrog
    astrofrog
    on Feb 14, 2015
    Member

    So the issue is in test_models.py with the following code:

    @pytest.mark.parametrize(('model_class', 'test_parameters'), models_1D.items())
class TestFittable1DModels(Fittable1DModelTester):
    pass


@pytest.mark.parametrize(('model_class', 'test_parameters'), models_2D.items())
class TestFittable2DModels(Fittable2DModelTester):
    pass

    and is related to decorating the classes.

    Reported with pytest here:

    https://bitbucket.org/hpk42/pytest/issue/683/bug-with-parameterized-classes-when

    In the mean time, a workaround is to decorate the methods instead of the class, but the issue is that the two classes need different parameters.

    In any case, not critical for 1.0 so we don't have to fix it before the final release.

  10. embray commented on Feb 16, 2015

    @embray
    embray
    on Feb 16, 2015
    Member

    If the issue is indeed with py.test it must be a regression, since I hadn't seen this happen before even under this context.

  11. embray
    added
    Bug
    on Feb 16, 2015

  12. embray commented on Feb 16, 2015

    @embray
    embray
    on Feb 16, 2015
    Member

    Huh...well, I tested this back to py.test v.2.4.2 and still had this problem so maybe it isn't new. Wonder why it never came up before. Maybe that test just didn't exist last time I was testing multiple subsequent test runs.

  13. embray commented on Feb 16, 2015

    @embray
    embray
    on Feb 16, 2015
    Member

    I tried coming up with a workaround to this, but it's even trickier than I thought. Might be possible to just monkey-patch the transfer_markers function that I identified as the problem. But might not even be worth the effort :)

  14. astrofrog commented on Feb 16, 2015

    @astrofrog
    astrofrog
    on Feb 16, 2015
    Member

    I don't think this is 1.0-critical. Honestly I think running tests twice is a rare use case and we could just list it in the known issues if we really care.

  15. bsipocz commented on Nov 25, 2015

    @bsipocz
    bsipocz
    on Nov 25, 2015
    MemberAuthor

    Oops, run into this again with 1.1rc1 testing :)
    The difference that now there are some additional fails on top of the errors.

    Will rerun everything tomorrow, and refresh the table.

  16. astrofrog commented on Nov 25, 2015

    @astrofrog
    astrofrog
    on Nov 25, 2015
    Member

    Huh, the py.test bug report link is gone. Checking.

  17. astrofrog commented on Nov 25, 2015

    @astrofrog
    astrofrog
    on Nov 25, 2015
    Member

    Looks like it moved here:

    pytest-dev/pytest#683

    and got fixed in py.test here:

    pytest-dev/pytest@e8f4819

    The bundled py.test is 2.5.1. We need to update the bundled version to 2.8.0 or later. I'm on it.

  18. astrofrog commented on Nov 25, 2015

    @astrofrog
    astrofrog
    on Nov 25, 2015
    Member

    See #4349

  19. eteq commented on Feb 12, 2016

    @eteq
    eteq
    on Feb 12, 2016
    Member

    I think this is fixed by #4349, so closing. But if it turns out not to fix it, feel free to re-open!

  20. eteq
    closed this as completedon Feb 12, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Bugtesting

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions