Skip to content

4.1rc2: test_compound_fitting_with_units fails on 32-bit platforms #10675

New issue
New issue
Closed as not planned
Closed as not planned
4.1rc2: test_compound_fitting_with_units fails on 32-bit platforms#10675
Labels
modelingtesting

Description

@bnavigator
bnavigator
opened on Aug 24, 2020

Description

The model in test_compound_fitting_with_units does not only produce different results between Linux and macOS (#10414, #10415) but also between Linux x86_64 and i586. The former passes all tests, the latter produces this failure:

[   96s] + /usr/bin/python3 -B -c 'import astropy, sys; sys.exit(astropy.test(args="-v "))'
[   99s] ============================= test session starts ==============================
[   99s] platform linux -- Python 3.8.5, pytest-4.6.9, py-1.9.0, pluggy-0.13.1 -- /usr/bin/python3
[  100s] cachedir: .pytest_cache
[  100s] 
[  100s] Running tests with Astropy version 4.1rc2.
[  100s] Running tests in /usr/lib/python3.8/site-packages/astropy.
[  100s] 
[  100s] Date: 2020-08-24T12:59:50
[  100s] 
[  100s] Platform: Linux-5.8.0-1-pae-i686-with-glibc2.1.3
[  100s] 
[  100s] Executable: /usr/bin/python3
[  100s] 
[  100s] Full Python Version: 
[  100s] 3.8.5 (default, Jul 20 2020, 17:41:41) [GCC]
[  100s] 
[  100s] encodings: sys: utf-8, locale: UTF-8, filesystem: utf-8
[  100s] byteorder: little
[  100s] float info: dig: 15, mant_dig: 15
[  100s] 
[  100s] Package versions: 
[  100s] Numpy: 1.19.1
[  100s] Scipy: 1.5.2
[  100s] Matplotlib: 3.3.0
[  100s] h5py: 2.10.0
[  100s] Pandas: 1.0.5
[  100s] PyERFA: 1.7.0
[  100s] Cython: 0.29.21
[  100s] Scikit-image: not available
[  100s] asdf: 2.6.0
[  100s] 
[  100s] Using Astropy options: remote_data: none.
[  100s] 
[  100s] hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/abuild/rpmbuild/BUILD/.hypothesis/examples')
[  100s] Matplotlib: 3.3.0
[  100s] Freetype: 2.6.1
[  100s] rootdir: /usr/lib/python3.8/site-packages/astropy
[  100s] plugins: filter-subpackage-0.0.0, astropy-header-0.1.2, hypothesis-5.19.0, openfiles-0.5.0, mpl-0.11, remotedata-0.3.2, asdf-2.6.0, cov-2.10.0, doctestplus-0.7.0, arraydiff-0.3
[  155s] collecting ... collected 15020 items / 1 skipped / 15019 selected
...
[ 1012s] =================================== FAILURES ===================================
[ 1012s] _______________________ test_compound_fitting_with_units _______________________
[ 1012s] 
[ 1012s]     @pytest.mark.skipif('not HAS_SCIPY')
[ 1012s]     def test_compound_fitting_with_units():
[ 1012s]         x = np.linspace(-5, 5, 15) * u.Angstrom
[ 1012s]         y = np.linspace(-5, 5, 15) * u.Angstrom
[ 1012s]     
[ 1012s]         fitter = fitting.LevMarLSQFitter()
[ 1012s]         m = models.Gaussian2D(10*u.Hz,
[ 1012s]                               3*u.Angstrom, 4*u.Angstrom,
[ 1012s]                               1*u.Angstrom, 2*u.Angstrom)
[ 1012s]         p = models.Planar2D(3*u.Hz/u.Angstrom, 4*u.Hz/u.Angstrom, 1*u.Hz)
[ 1012s]         model = m + p
[ 1012s]     
[ 1012s]         z = model(x, y)
[ 1012s]         res = fitter(model, x, y, z)
[ 1012s]         assert isinstance(res(x, y), np.ndarray)
[ 1012s]         assert all([res[i]._has_units for i in range(2)])
[ 1012s]     
[ 1012s]         model = models.Gaussian2D() + models.Planar2D()
[ 1012s]         res = fitter(model, x, y, z)
[ 1012s]         assert isinstance(res(x, y), np.ndarray)
[ 1012s]         assert all([res[i]._has_units for i in range(2)])
[ 1012s]     
[ 1012s]         # A case of a mixture of models with and without units
[ 1012s]         model = models.BlackBody(temperature=3000 * u.K) * models.Const1D(amplitude=1.0)
[ 1012s]         x = np.linspace(1, 3, 10000) * u.micron
[ 1012s]     
[ 1012s]         with NumpyRNGContext(12345):
[ 1012s]             n = np.random.normal(3)
[ 1012s]     
[ 1012s]         y = model(x)
[ 1012s]         res = fitter(model, x, y * (1 + n))
[ 1012s]         # The large rtol here is due to different results on linux and macosx, likely
[ 1012s]         # the model is ill-conditioned.
[ 1012s] >       np.testing.assert_allclose(res.parameters, [3000, 2.1433621e+00, 2.647347e+00], rtol=0.4)
[ 1012s] E       AssertionError: 
[ 1012s] E       Not equal to tolerance rtol=0.4, atol=0
[ 1012s] E       
[ 1012s] E       Mismatched elements: 2 / 3 (66.7%)
[ 1012s] E       Max absolute difference: 1.2675296
[ 1012s] E       Max relative difference: 0.54769477
[ 1012s] E        x: array([3.000000e+03, 9.694539e-01, 3.914877e+00])
[ 1012s] E        y: array([3.000000e+03, 2.143362e+00, 2.647347e+00])
[ 1012s] 
[ 1012s] /usr/lib/python3.8/site-packages/astropy/modeling/tests/test_quantities_fitting.py:256: AssertionError
...
[ 1012s] = 1 failed, 14592 passed, 360 skipped, 68 xfailed, 65 warnings in 912.96 seconds =

(I am still waiting on the armv7l results while aarch64 passes)

Steps to Reproduce

  • Install astropy 4.1rc2 on a Linux with 32-bit libraries and run the unit tests

OR

System Details

See above pytest header

Activity

  1. pllim
    added
    testing
    on Aug 24, 2020
  2. pllim
    added this to the v4.1 milestone on Aug 24, 2020
  3. pllim
    added
    modeling
    on Aug 24, 2020

  4. pllim commented on Aug 24, 2020

    @pllim
    pllim
    on Aug 24, 2020
    Member

    @nden or @perrygreenfield , do you know if this has been fixed in master or needs looking into? Thanks.

  5. bnavigator commented on Aug 24, 2020

    @bnavigator
    bnavigator
    on Aug 24, 2020
    ContributorAuthor

    master still has the same code:

    astropy/astropy/modeling/tests/test_quantities_fitting.py

    Lines 254 to 256 in c1cea19

    # The large rtol here is due to different results on linux and macosx, likely
    # the model is ill-conditioned.
    np.testing.assert_allclose(res.parameters, [3000, 2.1433621e+00, 2.647347e+00], rtol=0.4)

  6. perrygreenfield commented on Aug 24, 2020

    @perrygreenfield
    perrygreenfield
    on Aug 24, 2020
    Member

    The first thing that should be looked at is whether in fact the matrix is ill-conditioned as the comment suggests. If it is, it simply a bad test and should be changed.

  7. pllim commented on Aug 24, 2020

    @pllim
    pllim
    on Aug 24, 2020
    Member

    Looks like @nden added the test in #10415 (milestone: 4.0.2)

  8. nden commented on Aug 24, 2020

    @nden
    nden
    on Aug 24, 2020
    Contributor

    I had problems with that particular test - it gave different results on Linux and MacOSX.

  9. bnavigator commented on Aug 28, 2020

    @bnavigator
    bnavigator
    on Aug 28, 2020
    ContributorAuthor

    New failure for same test on openSUSE Linux x84_64 (!) with the bundled libs:

    [  865s] >       np.testing.assert_allclose(res.parameters, [3000, 2.1433621e+00, 2.647347e+00], rtol=0.4)
[  865s] E       AssertionError: 
[  865s] E       Not equal to tolerance rtol=0.4, atol=0
[  865s] E       
[  865s] E       Mismatched elements: 1 / 3 (33.3%)
[  865s] E       Max absolute difference: 0.86022509
[  865s] E       Max relative difference: 0.4013438
[  865s] E        x: array([3.000000e+03, 1.283137e+00, 2.957823e+00])
[  865s] E        y: array([3.000000e+03, 2.143362e+00, 2.647347e+00])

    The change in build environment between pass and fail are updates to the following packages:

    fdupes
libimagequant0
libjson-c5
libsystemd0
libudev1
python3-asdf
systemd

  10. pllim commented on Aug 28, 2020

    @pllim
    pllim
    on Aug 28, 2020
    Member

    Hmm... python3-asdf

  11. bnavigator commented on Aug 28, 2020

    @bnavigator
    bnavigator
    on Aug 28, 2020
    ContributorAuthor

    FWIW, that one also only got a rebuild trigger becuase of some packages already listed:
    https://build.opensuse.org/packages/python-asdf/build_reason/devel:languages:python:numeric/openSUSE_Tumbleweed/x86_64

  12. olebole commented on Sep 17, 2020

    @olebole
    olebole
    on Sep 17, 2020
    Member

    I also see this on armhf, while it succeeded on i386 during the build of the Debian package.

    In two subsequent builds on amd64 for 4.1rc2, I see one success and one failure. As far as I can see (they were automatically build), both environments were identical.

  13. pllim commented on Sep 17, 2020

    @pllim
    pllim
    on Sep 17, 2020
    Member

    @nden should we xfail or redesign the test or something if it is flaky in its current state?

  14. nden commented on Sep 17, 2020

    @nden
    nden
    on Sep 17, 2020
    Contributor

    I think we should xfail it for now.

  15. eteq commented on Oct 6, 2020

    @eteq
    eteq
    on Oct 6, 2020
    Member

    #10746 xfails this, so I think this is not a relase blocker, but @nden can you clarify if you think we should keep this issue open to fix later or think of as a perma-xfail?

  16. eteq
    modified the milestones: v4.1, v4.1.1 on Oct 6, 2020

  17. bnavigator commented on Oct 10, 2020

    @bnavigator
    bnavigator
    on Oct 10, 2020
    ContributorAuthor

    Also need to skip this test on 4.0.2 now.

  18. bsipocz
    modified the milestones: v4.1.1, v4.2.1 on Nov 25, 2020
  19. pllim
    modified the milestones: v4.2.1, v4.2.2 on Apr 8, 2021
  20. eteq
    modified the milestones: v4.2.2, v4.3.2 on Jul 27, 2021
  21. bsipocz
    removed this from the v4.3.2 milestone on Mar 10, 2023

  22. pllim commented on Oct 27, 2023

    @pllim
    pllim
    on Oct 27, 2023
    Member

    I don't think we will prioritize any 32-bit fixes going forward. If anything, we been trying to get rid of it...

  23. pllim
    closed this as not plannedon Oct 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    modelingtesting

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions