Skip to content

Travis CI: Test Python 3.8#13347

Merged
adamjstewart merged 6 commits intospack:developfrom
adamjstewart:tests/python3.8
Oct 31, 2019
Merged

Travis CI: Test Python 3.8#13347
adamjstewart merged 6 commits intospack:developfrom
adamjstewart:tests/python3.8

Conversation

@adamjstewart
Copy link
Copy Markdown
Member

@adamjstewart adamjstewart commented Oct 21, 2019

This PR adds Python 3.8 to our list of tested versions on Travis CI. It also fixes several bugs with our Python 3.8 support

  • Fixes erroneous F811 error in spack flake8
  • Fixes error in spack list --format=html

@adamjstewart adamjstewart added python tests General test capability(ies) travis labels Oct 21, 2019
@adamjstewart adamjstewart requested a review from alalazo October 21, 2019 02:13
@alalazo

This comment has been minimized.

@adamjstewart

This comment has been minimized.

if sys.version_info > (3, 1):
from html import escape
else:
from cgi import escape
Copy link
Copy Markdown
Member Author

@adamjstewart adamjstewart Oct 29, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to https://docs.python.org/3.7/library/cgi.html, cgi.escape has been deprecated since Python 3.2, and html.escape should be used instead. These methods are identical, but cgi.escape defaults to quotes=False, while html.escape defaults to quotes=True. See https://docs.python.org/3/whatsnew/3.8.html#api-and-feature-removals for the announcement of the cgi.escape removal.

@adamjstewart
Copy link
Copy Markdown
Member Author

adamjstewart commented Oct 29, 2019

The test_flake8 unit test works locally for me, so I'm having trouble debugging it. spack flake8 correctly adds noqa to the redefinition line:

    # '@when' decorated functions are exempt from redefinition errors           
    @when('@2.0')  # noqa: F811                                                 
    def install(self, spec, prefix):                                            
        pass 

EDIT: Okay, I'm now able to reproduce this, I needed to build flake8 with Python 3.8 for the bug to surface. This may be a bug in flake8 or one of its dependencies.

EDIT: Looks like a change was made in flake8 or one of its dependencies. Previously,

    @when('@2.0')  # noqa: F811                                                 
    def install(self, spec, prefix):                                            

was required. Now,

    @when('@2.0')                                                 
    def install(self, spec, prefix):  # noqa: F811                                            

is required.

EDIT: Opened an issue: PyCQA/pyflakes#487

@adamjstewart
Copy link
Copy Markdown
Member Author

Okay, looks like the behavior of things changed in Python 3.8: https://gitlab.com/pycqa/flake8/issues/583

We'll have to figure out how to add the noqa to the right line.

@adamjstewart
Copy link
Copy Markdown
Member Author

Okay, with the latest commit, we get:

    # '@when' decorated functions are exempt from redefinition errors
    @when('@2.0')  # noqa: F811
    def install(self, spec, prefix):  # noqa: F811
        pass

I think this is as robust as we can get for now.

@adamjstewart
Copy link
Copy Markdown
Member Author

adamjstewart commented Oct 30, 2019

Thoughts on switching all tests from Python 3.7 to 3.8 (including flake8, docs, and build tests)?

Pros:

  • Greater confidence in Python 3.8 support

Cons:

  • Python 3.8 is still a little buggy

EDIT: Let's see what happens 😄

ignore_f811_on_previous_line = True
elif ignore_f811_on_previous_line:
line_errors.append('F811')
ignore_f811_on_previous_line = False
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be equivalent to:

ignore_f811_on_previous_line = 'F811' in line_errors and line_errors.append('F811')

but sometimes too much magic is a bad thing.

language: python
env: [ TEST_SUITE=build, 'SPEC=mpich' ]
- python: '3.6'
- python: '3.8'
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this docker test actually run?

Copy link
Copy Markdown
Member

@ax3l ax3l Oct 30, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it might be that we still use this to push packages.spack.io - have to migrate this over to Dockerhub builds some time soonish

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I understand the docker test is a CD step that is always run on push to develop and master:

spack/.travis.yml

Lines 113 to 114 in 4626c28

- name: 'docker build'
if: type = push AND branch IN (develop, master)

In fact at every commit we currently build many docker images on Dockerhub (each with a different base OS).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you check the Build History on Travis you should see the docker step run last.

Copy link
Copy Markdown
Member

@ax3l ax3l Oct 30, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but we are in the process of migrating those builds to automatic builds on Dockerhub itself. For now, we still need it in Travis CI.

@adamjstewart adamjstewart merged commit 536486f into spack:develop Oct 31, 2019
@adamjstewart adamjstewart deleted the tests/python3.8 branch October 31, 2019 19:20
tgamblin pushed a commit that referenced this pull request Nov 1, 2019
* Travis CI: Test Python 3.8

* Fix use of deprecated cgi.escape method

* Fix version comparison

* Fix flake8 F811 change in Python 3.8

* Make flake8 happy

* Use Python 3.8 for all test categories
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python tests General test capability(ies)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants