Update testing for 1.2, moving to 3.5 by default. #4456
Conversation
mhvk
force-pushed
the
new-travis-setup
branch
3 times, most recently
from
a9c9b3a to
9743945
Compare
.travis.yml
Outdated
There was a problem hiding this comment.
you need to list this above in the matrix too, and list the duplicate of the exact same env call under allow_failures.
There was a problem hiding this comment.
Ah, I wondered about that duplication -- should have realised it had a reason!
There was a problem hiding this comment.
np, I probably spent way more time than what's healthy playing with travis :)
|
@mhvk - Sorry for the nitpicking, but I think a numpy 1.9 build would be needed, too. |
|
Yes, should have added numpy 1.9. Darn, still so many tests! |
mhvk
changed the title
.travis.yml
Outdated
There was a problem hiding this comment.
@mhvk - Python 3.5 does support beautiful soup, but you need to use the new package name, beautifulsoup4
mhvk
force-pushed
the
new-travis-setup
branch
from
af73e42 to
66a7a7b
Compare
mhvk
changed the title
astrofrog
changed the title
|
@mhvk - can you fix the commit that says 1.12 to say 1.2 instead? |
.travis.yml
Outdated
There was a problem hiding this comment.
Can you change the order back to the original one?
mhvk
force-pushed
the
new-travis-setup
branch
from
66a7a7b to
477ae0b
Compare
|
OK, both done. |
.travis.yml
Outdated
There was a problem hiding this comment.
Should we have NUMPY_VERSION=stable here to always update to the latest stable version?
mhvk
force-pushed
the
new-travis-setup
branch
2 times, most recently
from
1f23f55 to
02cb31d
Compare
|
The sphinx documentation build on python3 seems to have quite a few different issues, including some documentation or code docstrings assuming python2, so I moved that back to python2 for now. See new issue #4481. |
|
It seems the main tests finally do not fail any more. Does anyone understand why coverage went down? |
|
If you rebase it should work again. We saw this before (I just can't find the issue), and my work hypothesis was that it happens if coverage is using an older than master base for the PR but compares it to the latest master's coverage. Thus if the coverage increased between the base of this and master it takes it as it "decreases" with this PR. |
mhvk
force-pushed
the
new-travis-setup
branch
from
56ca758 to
35327cb
Compare
|
@bsipocz - OK, thanks, that sounds plausible. Rebased now. |
|
@bsipocz - maybe plausible, but sadly not true. Looking closer at the coverage report, essentially all python files seem to be missing, and the travis |
|
@mhvk - maybe |
|
Hmm, looking at the logs it seems that it was a problem before, too. Maybe @embray or @eteq have a good idea what is going on? Why aren't the c files found? Also why are they tried to be tested at all in |
mhvk
force-pushed
the
new-travis-setup
branch
5 times, most recently
from
27744fb to
ac00d8b
Compare
mhvk
force-pushed
the
new-travis-setup
branch
from
ac00d8b to
4182cd4
Compare
|
With #4645 in, so that numpy 1.10 is tested, I now returned to moving our tests to python3 by default. It seems to work, except that the coverage decreased, in a way I find very hard to parse. Any suggestions? Is this a problem? Note that this keeps the sphinx build on python2; moving that to python3 is the goal of @bsipocz's #4556. |
|
It seems that something goes wrong with the c code coveralls with the py3 run. I don't think that should be a blocker for this PR though, but maybe open a separate issue to tackle this. |
|
However something is fishy with these summary numbers (in the coverage run): This PR: Master: |
|
I've started some debug runs on my travis: https://travis-ci.org/bsipocz/astropy/builds/113411787 |
|
@bsipocz - OK, thanks! If it is too unclear, I can also move coverage back to 2.7 for now, and raise a separate issue about that (as is already happening for sphinx...). I guess the upside to all of this is that we're finding and fixing python3 issues! |
|
Well, I would even argue that we could leave the coverage with the py3 run, and fix the problem separately. Now it seems coverage unrelated, some of those tests involving the c files are not picked up (or something similar) in the py3 run. |
|
@bsipocz - Since it is only the C coverage that is affected, either just moving coverage to 3.5 and investigating further after, or keeping it at 2.7 for now and moving later are fine by me. Maybe get further opinions... @astrofrog? @eteq? |
|
@mhvk - OK, one mystery is solved. The difference in the number of tests has nothing to do with the c coverage, and due to |
|
Oh, interesting. So I guess this is our "real" test coverage then? :( Perhaps we need an improve-coverage sprint in the near future... At any rate, I think this is all set, right? @astrofrog, do you agree we want to do this? (i.e., switch to 3.5 as the default) If so, I say go for it! |
|
Something seems wrong with this coverage report : |
|
By the way, I fixed issues with coverage in Python 3.x in #4822 |
|
However, as mentioned in #4826, coverage testing is way slower in Python 3, so I don't think we want to use that yet until this is figured out. I'm removing the 1.2 milestone since this is not critical for 1.2 |
|
Enough had changed in |
6 participants
This is an alternative to #4225, in which I tried to make a mininal set and move to python 3. For this purpose, I made an explicit python 3.5 default, and removed the non-dependency tests on 2.7 and 3.5 (since those tests are already done with dependencies anyway).
Now see if it actually passes...