Skip to content

BUG: avoid test pollution in astropy.units - #17750

Merged
mhvk merged 1 commit into
astropy:mainfrom
neutrinoceros:units/test/avoid_test_pollution
Feb 10, 2025
Merged

mhvk merged 1 commit into
astropy:mainfrom
neutrinoceros:units/test/avoid_test_pollution

Conversation

@neutrinoceros

Copy link
Copy Markdown
Contributor

Description

Partially address #17745

This fixes two issues:

  • UfuncHelpers.import_module was needlessly destructive (removing keys from self.modules), which led made test_scipy_registered fail if run after scipy.special was imported.
  • test_scipy_registration relies on being run before any scipy.special method is called (which implicitly triggers registration), so it cannot possibly succeed when run at a later point. I don't see a good solution here and I would like to avoid mocking global state too much, so I hope that simply skipping the test on a second run is acceptable.

reprod (requires scipy)

pytest astropy/units/tests/test_quantity_ufuncs.py::TestScipySpecialUfuncs::test_erf_scalar  astropy/units/tests/test_quantity_ufuncs.py::test_scipy_registration astropy/units/tests/test_quantity_ufuncs.py::TestUfuncHelpers
  • By checking this box, the PR author has requested that maintainers do NOT use the "Squash and Merge" button. Maintainers should respect this when possible; however, the final decision is at the discretion of the maintainer that merges the PR.

@github-actions

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.

  • Do the proposed changes actually accomplish desired goals?
  • Do the proposed changes follow the Astropy coding guidelines?
  • Are tests added/updated as required? If so, do they follow the Astropy testing guidelines?
  • Are docs added/updated as required? If so, do they follow the Astropy documentation guidelines?
  • Is rebase and/or squash necessary? If so, please provide the author with appropriate instructions. Also see instructions for rebase and squash.
  • Did the CI pass? If no, are the failures related? If you need to run daily and weekly cron jobs as part of the PR, please apply the "Extra CI" label. Codestyle issues can be fixed by the bot.
  • Is a change log needed? If yes, did the change log check pass? If no, add the "no-changelog-entry-needed" label. If this is a manual backport, use the "skip-changelog-checks" label unless special changelog handling is necessary.
  • Is this a big PR that makes a "What's new?" entry worthwhile and if so, is (1) a "what's new" entry included in this PR and (2) the "whatsnew-needed" label applied?
  • At the time of adding the milestone, if the milestone set requires a backport to release branch(es), apply the appropriate "backport-X.Y.x" label(s) before merge.

@github-actions

Copy link
Copy Markdown
Contributor

👋 Thank you for your draft pull request! Do you know that you can use [ci skip] or [skip ci] in your commit messages to skip running continuous integration tests until you are ready?

@neutrinoceros neutrinoceros changed the title BUG: avoid test pollution in astropy.units BUG: avoid test pollution in astropy.units Feb 10, 2025
@neutrinoceros
neutrinoceros marked this pull request as ready for review February 10, 2025 13:40

@mhvk mhvk left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great that you found the solution, though this leaves me a little confused -- and it would be good to understand why before merging...

def test_scipy_registration():
"""Check that scipy gets loaded upon first use."""
assert sps.erf not in qh.UFUNC_HELPERS
if sps.erf in qh.UFUNC_HELPERS:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add a comment?

# Generally, scipy will not be loaded here, but in a double run it might.

I also think it may make sense to test instead a fresh instance of UfuncHelpers, but perhaps for another PR.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note that TestScipySpecialUfuncs (the class immediately following this test) also relies on this test being run first, so I can't simply replace the test.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(I added your comment)

"""
with self._lock:
module_info = self.modules.pop(module)
module_info = self.modules[module]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm super confused why this would be necessary. What test fails if you keep it? It should not be possible to get here more than once...

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test_scipy_registered is the test that fails without this. See the reprod invoke in the PR description.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And yes, we don't get here more than once, but that test explicitly checks for a state that is changed by this line.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'd prefer to change the tests then, skipping that with @pytest.mark.skipif(HAS_SCIPY)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense too, but I don't get what you don't like about my less destructive import_module ?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The advantage of what I have here is that, if ever something when wrong with checking membership, we will know, rather than everything becoming very slow as for every call the whole module gets loaded back in.

But, more generally, I prefer not to change code to fix the double tests if it is arguably the test that was written sloppily. We cannot know whether somewhere in the wild code relies on the current behaviour.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fair enough. Adopted !

@neutrinoceros
neutrinoceros force-pushed the units/test/avoid_test_pollution branch from 3d95f04 to 322a27d Compare February 10, 2025 14:34
@neutrinoceros
neutrinoceros force-pushed the units/test/avoid_test_pollution branch from 322a27d to 9f9d92a Compare February 10, 2025 15:35

@mhvk mhvk left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, let's go with this!

@pllim

pllim commented Feb 10, 2025

Copy link
Copy Markdown
Member

I am afraid to ask but I should: Does this mean we need to add yet another job to CI to smoke these out automatically in the future?

pllim added a commit that referenced this pull request Feb 10, 2025
…750-on-v7.0.x

Backport PR #17750 on branch v7.0.x (BUG: avoid test pollution in `astropy.units`)
@mhvk

mhvk commented Feb 10, 2025

Copy link
Copy Markdown
Contributor

I think the idea is to make the double test depend at least also on scipy (maybe just "recommended dependencies"? could even do "all" and remove one of the ones that already does that - if it works two times, that means it worked one time too!).

@pllim

pllim commented Feb 10, 2025

Copy link
Copy Markdown
Member

Like this?

@pllim pllim mentioned this pull request Feb 10, 2025
1 task
@neutrinoceros
neutrinoceros deleted the units/test/avoid_test_pollution branch February 11, 2025 09:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants