Skip to content

[wptrunner] Two small fixes for python3 compatibility#24993

Merged
stephenmcgruer merged 1 commit intomasterfrom
smcgruer/gittree_py3
Aug 18, 2020
Merged

[wptrunner] Two small fixes for python3 compatibility#24993
stephenmcgruer merged 1 commit intomasterfrom
smcgruer/gittree_py3

Conversation

@stephenmcgruer
Copy link
Contributor

These were discovered in #24952,
but apply equally to other platforms. There are two fixes here:

  1. For is_git_root in vcs.py, compare bytes against bytes, rather than against string
  2. Make sure the revision recorded in WPTTest is a string, not bytes

The first of these would cause us to always return None for the revision in Python 3, because is_git_root would always be false. After that was fixed, the latter was discovered where we would write bytes into the WPTTest information, which we would later try to dump as JSON - which fails for byte data.

@wpt-pr-bot wpt-pr-bot added infra wptrunner The automated test runner, commonly called through ./wpt run labels Aug 13, 2020
@stephenmcgruer stephenmcgruer assigned Hexcles and unassigned jgraham Aug 13, 2020
@stephenmcgruer
Copy link
Contributor Author

stephenmcgruer commented Aug 13, 2020

So there are a bunch more bugs in the tree.py code, but as I was writing a set of tests for it I discovered that it's even buggy in py2 in places -_-. It looks like most of the code isn't really used (I suspect the mercurial version of the code is what is mostly used, and GitTree was just coded to theory). So ultimately I just ended up doing these specific fixes.

@Hexcles
Copy link
Member

Hexcles commented Aug 13, 2020

Looking at https://github.com/web-platform-tests/wpt/blob/master/tools/wptrunner/wptrunner/update/tree.py , I see mostly (if not all) native strings when interacting with git output. Shall we perhaps always decode in vcs?

@jgraham
Copy link
Contributor

jgraham commented Aug 14, 2020

I think the hg code for tree.py was almost never used, but the git code maybe was for gecko (historically) and Servo (possibly still). It's definitely not very well maintained though.

@stephenmcgruer
Copy link
Contributor Author

Looking at https://github.com/web-platform-tests/wpt/blob/master/tools/wptrunner/wptrunner/update/tree.py , I see mostly (if not all) native strings when interacting with git output. Shall we perhaps always decode in vcs?

I thought about this, but wasn't sure. It is reasonable to expect both git and hgto always return utf-8 encoded data? In any case there are a lot of callers in wptrunner and I'm wary of changing this without mypy support (which is a big undertaking).

@stephenmcgruer
Copy link
Contributor Author

I'm going to go ahead and merge this, since it tripped me up when recently trying to run MacOS + Py3 again.

@stephenmcgruer stephenmcgruer merged commit 60adaca into master Aug 18, 2020
@stephenmcgruer stephenmcgruer deleted the smcgruer/gittree_py3 branch August 18, 2020 00:58
@Hexcles
Copy link
Member

Hexcles commented Aug 18, 2020

I thought about this, but wasn't sure. It is reasonable to expect both git and hg to always return utf-8 encoded data?

Not sure about hg, but git is almost encoding-agnostic, so assuming UTF-8 isn't technically correct, sigh...

In any case there are a lot of callers in wptrunner and I'm wary of changing this without mypy support (which is a big undertaking).

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

infra python3 wptrunner The automated test runner, commonly called through ./wpt run

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants