Skip to content

Use nox as a test runner instead of tox - #1735

Closed
namurphy wants to merge 55 commits into
PlasmaPy:mainfrom
namurphy:nox
Closed

namurphy wants to merge 55 commits into
PlasmaPy:mainfrom
namurphy:nox

Conversation

@namurphy

@namurphy namurphy commented Sep 9, 2022

Copy link
Copy Markdown
Member

This PR adds a prototype testing configuration with nox instead of tox. As described in #1734, the motivations for using nox are:

  • The test configuration files are written in Python.
  • We will be able to more easily (less difficultly?) dynamically generate tests.

The configuration file is noxfile.py, which I'm adapting from the test setup for XRTpy.

Closes #1734.

This is very close to the noxfile.py that I added for XRTpy.
@namurphy namurphy added testing CI Related to continuous integration prototype 🏗️ Trying out an implementation on a trial basis labels Sep 9, 2022
@namurphy

namurphy commented Sep 9, 2022

Copy link
Copy Markdown
Member Author

Yay it's doing stuff!

An issue it's running into now is that it's not finding Python 3.9 for macOS, which is also happening on XRTpy (though it did work in the not-too-distant past, I think).

Comment thread .github/workflows/testing.yml Outdated
uses: codecov/codecov-action@v3
with:
file: ./coverage.xml
python-version: '3.10'

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line needs reverting - as it is, it's installing 3.10 in all three currently attempted test envs. 3.8 Seems to already be there for 3.8/Windows, which is why that one isn't failing. I suspect it should fix the 3.9/MacOS failure

so that the python version matrix actually has an effect!
@github-actions

Copy link
Copy Markdown

Thank you for contributing to PlasmaPy! The project's future depends deeply on contributors like you, so we deeply appreciate it! 🌱 The following checklist will be used by the code reviewer to help guide the code review process.

  • Overall
    • Does the PR do what it intends to do?
    • Except for very minor changes, is a changelog entry included and consistent with the changelog guide?
    • Are the continuous integration checks passing? (Most linter problems can be automagically fixed by commenting on this PR with pre-commit.ci autofix.)
  • Code
    • Is new/updated code understandable and consistent with the coding guide?
    • Are there ways to greatly simplify the implementation?
    • Are there any large functions that should be split up into shorter functions?
  • Tests
    • Are tests added/updated as required, and consistent with the testing guide?
    • Are the tests understandable?
    • Do the tests cover all important cases?
  • Docs
    • Are docs added/updated as required, and consistent with the doc guide?
    • Are the docs understandable?
    • Do the docs show up correctly in the preview, including Jupyter notebooks?

Based on https://github.com/PlasmaPy/PlasmaPy/actions/runs/3039593035/jobs/4894695172#step:6:22, it looks like the docs environment is failing because it can't import `plasmapy.utils.pytest_helpers`. If so, I guess we should include `pytest` as a dependency of the [docs] extra. Hell, we could even do, in `requirements/docs.txt`,

```
sphinx
pyramid
hieroglyph
...
-r tests.txt
```

or some such, so that `tests` is strictly a subset of `docs`. But getting just pytest in is probably faster on the CI executor.
@StanczakDominik

Copy link
Copy Markdown
Member

I hope you don't mind me tinkering with your branch a little 😅 see comment on the latest commit for my reasoning!

@namurphy

Copy link
Copy Markdown
Member Author

I hope you don't mind me tinkering with your branch a little sweat_smile see comment on the latest commit for my reasoning!

Not a problem! I might not have a chance to look into this for a few days, so please tinker away!

@StanczakDominik StanczakDominik self-assigned this Sep 13, 2022
@StanczakDominik
StanczakDominik requested review from a team and StanczakDominik and removed request for a team September 13, 2022 18:37
Comment thread noxfile.py Resolved
Comment thread noxfile.py
sphinx_no_notebooks = ["-D", "nbsphinx_execute=never"]
sphinx_nitpicky = ["-n"]

pytest_options = ["--showlocals"]

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Make sure to keep all of the options that are currently in tox.ini

@namurphy

namurphy commented Oct 6, 2022

Copy link
Copy Markdown
Member Author

pre-commit.ci autofix

@namurphy

namurphy commented Oct 6, 2022

Copy link
Copy Markdown
Member Author

Hm...there's still an issue with one test for coverage that's seemingly not being run:

Python 3.8, all tests, code coverage Expected — Waiting for status to be reported

@namurphy

namurphy commented Dec 5, 2022 •

Copy link
Copy Markdown
Member Author

The main testing functionality I still need to cross over:

  • Enable code coverage via nox
  • Enable nox env for codespell in testing.yml
  • Building wheels, etc.

Other stuff...

  • Update testing guide to describe nox instead of tox

@namurphy namurphy mentioned this pull request Dec 9, 2022
4 of 5 tasks
@namurphy

Copy link
Copy Markdown
Member Author

Before marking this as ready for review, I'll need to:

  • Verify that changes made in .github/workflows/testing.yml in the last few months have been ported over, including:
  • Printing out link to troubleshooting section of contributor guide in test output
  • Printing out link to testing guide in test output

@StanczakDominik StanczakDominik left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're doing this in halves, then it might make sense to merge this to a feature branch first. Then I'll work off that before we put this on main.

Comment thread requirements.txt
# jupyter-client
nodeenv==1.7.0
# via pre-commit
nox

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remember that you should pin a version here! The command to do that is up top, but it won't work until you add nox to pyproject.toml first.

Comment thread noxfile.py


@nox.session
def build_docs_no_examples(session):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def build_docs_no_examples(session):
def build_docs_no_examples(session): # TODO: "parametrize" this? the same

I would absolutely not want to hold this up, but I'm curious, and I just want to make sure I notice this when picking it up later 😉

Comment thread noxfile.py
sphinx_opts = sphinx_paths + sphinx_fail_on_warnings + sphinx_builder
sphinx_skip_notebooks = ["-D", "nbsphinx_execute=never"]

post_doc_build_comments = """

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I half expect to see this become tenured_doc_build_comments later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI Related to continuous integration nox Related to the nox automation software prototype 🏗️ Trying out an implementation on a trial basis size: large 100 ≤ changed lines < 500 status: dormant PRs that are stalled testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Run tests with Nox instead of tox

2 participants