You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Some tests run much slower with coverage testing on Python 3 compared to Python 2 #4826
============= 1658 passed, 3 xfailed in 335.95 seconds =============
Note that Python 3 runs fewer tests (because it doesn't do the tests with unicode_literals) and even then it's 3 times slower. I'm not sure if the bottleneck is in our functionality, or the coverage testing itself.
I wonder if this is a pytest problem rather than an Astropy problem. If only there is a convenient way to also run coverage in both Python 2 and 3 for something that is not related to Astropy and see if it has the same timing difference. Hmm...
@mhvk , I don't either, since @MSeifert04 claimed that his PR is not supposed to fix this particular problem... Maybe Travis CI timing will confirm for sure.
Let me move the discussion back here: @astrofrog - do you get much longer times on your own computer? I ask since both @pllim and I seemed to get short ones, at least with the simple python3 setup.py test --coverage -P units line you had at the very top of this file (while travis obviously still is very slow).
I am running a series of coverage tests, once per module, and then altogether, and recording the run times on both Python versions. Hopefully it will be finished before Thanksgiving.
@pllim and @mhvk What versions of coverage are you using locally? As I noted in #5308 (comment) it may just be something that was fixed in a newer version of coverage.
I'm using what Debian testing provides, which is 4.2 (i.e., the newest according to the comment you linked to). Since we seem to install apt, it may be possible to get it via that? It would seem a good possibility!
Attached is my report from running coverage tests on each of the sub-package, and then the whole thing in the end. My conclusion is that the slowness in Python 3 is no longer an issue (at least with coverage 4.2). I agree with @MSeifert04 and @mhvk that the culprit might be the older coverage version, not Astropy itself. The one failure you see in the report is reported separately in #5316.
Activity
astrofrog commentedon May 2, 2016
Once this is solved, we can enable coverage testing in
.travis.ymlfor Python 3.xpllim commentedon May 2, 2016
Just to re-iterate something I said in #4822 -- Is it slow in all Python 3.x or just 3.4?
pllim commentedon May 2, 2016
@astrofrog said slow in both 3.4 and 3.5.
astrofrog commentedon May 2, 2016
Yep - the above timings were done on Python 3.5
astrofrog commentedon May 2, 2016
I think the next thing to look at would be to try and output the times for all the tests and see if there's a specific kind of test that is slow.
astrofrog commentedon May 2, 2016
On my laptop, the tests take about 15 minutes to run in Python 3.5 with coverage enabled:
pllim commentedon May 2, 2016
Just want to be clear -- So, when coverage is not enabled, the testing times are comparable between Python 2 and 3?
astrofrog commentedon May 2, 2016
With no coverage:
Python 2
Python 3
With coverage:
Python 2
Python 3
pllim commentedon May 2, 2016
I wonder if this is a
pytestproblem rather than an Astropy problem. If only there is a convenient way to also run coverage in both Python 2 and 3 for something that is not related to Astropy and see if it has the same timing difference. Hmm...astrofrog commentedon May 2, 2016
@pllim - yes, I think it would good to do that test. I might try with glue.
astrofrog commentedon May 2, 2016
I don't think it's a pytest issue. Results for glue:
Without coverage:
Python 2
Python 3
With coverage:
Python 2
Python 3
pllim commentedon May 2, 2016
😱 @astrofrog , thanks for confirming!
40 remaining items
pllim commentedon Sep 6, 2016
@mhvk , I don't either, since @MSeifert04 claimed that his PR is not supposed to fix this particular problem... Maybe Travis CI timing will confirm for sure.
mhvk commentedon Sep 6, 2016
Let me move the discussion back here: @astrofrog - do you get much longer times on your own computer? I ask since both @pllim and I seemed to get short ones, at least with the simple
python3 setup.py test --coverage -P unitsline you had at the very top of this file (while travis obviously still is very slow).pllim commentedon Sep 7, 2016
I am running a series of coverage tests, once per module, and then altogether, and recording the run times on both Python versions. Hopefully it will be finished before Thanksgiving.
mhvk commentedon Sep 7, 2016
If you're thinking of Canadian thanksgiving, we only have to wait about a month...
MSeifert04 commentedon Sep 7, 2016
@pllim and @mhvk What versions of
coverageare you using locally? As I noted in #5308 (comment) it may just be something that was fixed in a newer version ofcoverage.pllim commentedon Sep 7, 2016
The latest I could get via conda update on both Python versions. I'll check when I'm back at work tomorrow.
mhvk commentedon Sep 8, 2016
I'm using what Debian testing provides, which is 4.2 (i.e., the newest according to the comment you linked to). Since we seem to install
apt, it may be possible to get it via that? It would seem a good possibility!pllim commentedon Sep 8, 2016
See potential fix in astropy/ci-helpers#132
pllim commentedon Sep 8, 2016
Attached is my report from running coverage tests on each of the sub-package, and then the whole thing in the end. My conclusion is that the slowness in Python 3 is no longer an issue (at least with
coverage4.2). I agree with @MSeifert04 and @mhvk that the culprit might be the oldercoverageversion, not Astropy itself. The one failure you see in the report is reported separately in #5316.zcov_result.txt
MSeifert04 commentedon Sep 8, 2016
Would be nice if it's indeed the coverage version.
bsipocz commentedon Oct 10, 2016
Travis runs 2 mins faster for me for astropy when testing for @pllim's PR in ci-helpers, so hopefully some of that gain is real. https://travis-ci.org/bsipocz/astropy/builds/166497095
mhvk commentedon Oct 12, 2016
I think this is mostly solved, so closing!
holvianssi commentedon Jul 28, 2022
As this happens to be first hit on Google, and I just spent a lot of time chasing why coverage is slow, I'm going to leave a hint here for the future:
Check you have C extensions installed in your coverage package!