Make travis builds use wheels again #2576

mwcraig · 2014-05-28T23:16:59Z

@embray @Cadair -- this should not be merged until the -vvv option is removed, have that in so any remaining problems can be debugged.

Feel free, if you want this fix immediately and I'm not around to update this PR, to close it and do your own with the fix.

Fix was to remove the deprecated --use-mirrors

mwcraig · 2014-05-29T02:34:49Z

Never mind -- this obviously did not do the trick!

embray · 2014-05-29T15:00:31Z

Reopening this anyways because we should still remove the --use-mirrors.

embray · 2014-05-29T15:09:19Z

If I just run

pip install --find-links http://wheels.astropy.org/ --use-wheel scipy==0.13.1

at the command line that works.

embray · 2014-05-29T15:11:58Z

This does seem to be working. The only change I think is to remove the -vvv which isn't needed now.

mwcraig · 2014-05-29T15:41:44Z

Even the runs that are passing, though, are still not picking up the wheels...from the build at https://travis-ci.org/astropy/astropy/jobs/26306631

Downloading/unpacking scipy==0.13.1
  Getting page https://pypi.python.org/simple/scipy/
  URLs to search for versions for scipy==0.13.1:
  * https://pypi.python.org/simple/scipy/0.13.1
  * https://pypi.python.org/simple/scipy/
  * http://wheels.astropy.org/
  http://wheels.astropy.org/ uses an insecure transport scheme (http). Consider using https if wheels.astropy.org has it available
  * http://wheels2.astropy.org/
  http://wheels2.astropy.org/ uses an insecure transport scheme (http). Consider using https if wheels2.astropy.org has it available
  Skipping link http://wheels.astropy.org/ (from -f); not a file
  Skipping link http://wheels2.astropy.org/ (from -f); not a file
  Getting page https://pypi.python.org/simple/scipy/0.13.1

embray · 2014-05-29T15:59:14Z

I think you need to scroll further down:

 Using version 0.13.1 (newest of versions: 0.13.1, 0.13.1, 0.13.1)

Downloading scipy-0.13.1-cp27-none-linux_x86_64.whl (19.7MB):

Downloading from URL https://www.mpia-hd.mpg.de/%7erobitaille/wheelhouse/scipy-0.13.1-cp27-none-linux_x86_64.whl (from https://www.mpia-hd.mpg.de/~robitaille/wheelhouse/)

...Downloading scipy-0.13.1-cp27-none-linux_x86_64.whl (19.7MB): 19.7MB downloaded

Installing collected packages: scipy

Successfully installed scipy

I think the debug messages you're referring to are from code that first tries to process the URL as a file path (and then skipping it because it's not a local file path).

embray · 2014-05-29T16:03:04Z

Oh, how strange though. It's only for the build_sphinx run where it's failing.

embray · 2014-05-29T16:04:27Z

Seems like that may have been a temporary issue though:

 Could not fetch URL http://wheels.astropy.org/: connection error: HTTPConnectionPool(host='wheels.astropy.org', port=80): Max retries exceeded with url: / (Caused by <class 'socket.error'>: [Errno 111] Connection refused)

Will skip URL http://wheels.astropy.org/ when looking for download links for scipy

let me try re-running that one.

embray · 2014-05-29T16:05:58Z

@mwcraig Perhaps one other thing we should add is --no-index. This will prevent it from going to PyPI if the wheels can't be downloaded. That will of course cause the build to fail, but better have it fail immediately than have it sit there building scipy.

mwcraig · 2014-05-29T16:14:22Z

@embray -- I've seen some intermittent failures from wheels.astropy.org in the package_template builds too. I agree that --no-index would be useful.

I'll update this PR, removing -vvv and adding --no-index and a changelog, once we see what the restarted build does.

mwcraig · 2014-05-29T16:22:42Z

@embray -- Should the sphinx version be changed too? It is currrently pegged at 1.2b3.

embray · 2014-05-29T16:23:13Z

No, that's unrelated.

mwcraig · 2014-05-29T16:25:04Z

True, but sphinx is getting built from source right now because the wheelhouses don't have 1.2b3...

embray · 2014-05-29T16:34:54Z

I think that's expected and okay for now. I know there was something discussed about this a while back, but Sphinx has been at 1.2b3 for some time now for Reasons (that I can't recall).

mwcraig · 2014-05-29T18:51:50Z

We are down to fails because --no-index is on and there are no wheels for coveralls or sphinx 1.2b3.

Could code these in to not use wheels or build wheels for them or put --no-index back in.

I'd lean towards getting the wheels made, but am fine with any of these.

embray · 2014-05-29T20:38:34Z

Ah, what about this. Two commands:

$PIP_PYPY=pip install --use-wheels

$PIP_ASTROPY=pip install --find-links http://wheels.astropy.org --find-links http://wheels2.astropy.org --use-wheels --no-index

Use $PIP_ASTROPY for dependencies we know we have our own wheels built for. Use $PIP_PYPI for any others.

eteq · 2014-05-29T20:42:33Z

PyPI now has a sphinx 1.2.2, so it may well be safe to do sphinx>=1.2b3 instead of == ... if you change it to that and it works, feel free to change it to that, @mwcraig. Alternatively, it looks like wheels2.astropy.org has sphinx 1.2.2, so it should probably be able to get the wheel from there.

mwcraig · 2014-05-30T14:34:15Z

I like the $PIP_PYPI approach -- I'll do that (and change the sphinx requirement). Originally 1.2b3 was required due to some bug (don't recall details) in 1.1.x.

embray · 2014-05-30T15:46:51Z

@mwcraig Thanks for taking this on. I hope this approach will do the trick. We really need this since this is causing all the builds to go haywire 😾

mwcraig · 2014-05-30T15:49:10Z

Not a problem...I think I still come out way ahead in the cost/benefit sense as far as astropy goes :)

mwcraig · 2014-05-30T16:47:44Z

@embray -- do you want me to squash before this is merged? Some of the runs are still taking a long time (15-25 min), about 12min of that actually running the tests in some cases. Am guessing that is out of our control, though...

embray · 2014-05-30T17:08:49Z

If I were to hazard a guess it would be the sudo apt-get install steps that are sometimes taking a long time. Those can be quite slow when their servers are under heavy load. Not our problem though. Looks good! You can squash if you want.

eteq · 2014-05-30T21:54:38Z

I see this is squashed and tests pass, so I'm going to go ahead and merge this. Thanks @mwcraig !

Make travis builds use wheels again

mwcraig closed this May 29, 2014

embray reopened this May 29, 2014

embray mentioned this pull request May 29, 2014

Parallel testing fixes #2565

Merged

Remove deprecated pip and force use of wheels

ea43055

eteq added a commit that referenced this pull request May 30, 2014

Merge pull request #2576 from mwcraig/fix-travis

bf6a6ac

Make travis builds use wheels again

eteq merged commit bf6a6ac into astropy:master May 30, 2014

mwcraig deleted the fix-travis branch June 4, 2014 20:18

Uh oh!

Make travis builds use wheels again #2576

Make travis builds use wheels again #2576

Uh oh!

Conversation

mwcraig commented May 28, 2014

Uh oh!

mwcraig commented May 29, 2014

Uh oh!

embray commented May 29, 2014

Uh oh!

embray commented May 29, 2014

Uh oh!

embray commented May 29, 2014

Uh oh!

mwcraig commented May 29, 2014

Uh oh!

embray commented May 29, 2014

Uh oh!

embray commented May 29, 2014

Uh oh!

embray commented May 29, 2014

Uh oh!

embray commented May 29, 2014

Uh oh!

mwcraig commented May 29, 2014

Uh oh!

mwcraig commented May 29, 2014

Uh oh!

embray commented May 29, 2014

Uh oh!

mwcraig commented May 29, 2014

Uh oh!

embray commented May 29, 2014

Uh oh!

mwcraig commented May 29, 2014

Uh oh!

embray commented May 29, 2014

Uh oh!

eteq commented May 29, 2014

Uh oh!

mwcraig commented May 30, 2014

Uh oh!

embray commented May 30, 2014

Uh oh!

mwcraig commented May 30, 2014

Uh oh!

mwcraig commented May 30, 2014

Uh oh!

embray commented May 30, 2014

Uh oh!

eteq commented May 30, 2014

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants