This repository was archived by the owner on Aug 11, 2023. It is now read-only.
have travis try to use wheels #24
Description
Right now travis builds for affiliated packages have to build all dependencies. With astropy/astropy#1029, we are using the "wheel" format to store binaries of scipy, numpy, and Cython, which speeds up the tests by more than 2x. We should change the affiliated package template's .travis.yml to mirror this approach.
Two things needed first, though:
- First, though, we should figure out where we are going to store the wheels on a long-term basis. One possibility might be to use something like "wheels.astropy.org" and have that redirect to some place temporary for now (where the core package wheels are) and move it around as we get better servers.
- We may also want to add a wheel for astropy itself for use by the affiliated packages. This could be tricky for packages that want both stable and "latest" dev versions of astropy, but the approach from Change travis to use wheel binaries astropy#1029 was designed with this in mind - it will try to download a wheel, and if it doesn't find it it will fall back on the source code build.
Metadata
Metadata
Assignees
Labels
No labels
Activity
astrofrog commentedon May 4, 2013
I had the same thought today about using a sub-domain like
wheels.astropy.orgto alleviate the long-term storage issue - i think that would work.cdeil commentedon Aug 13, 2013
Status?
(
astropyandastroqueryare already using wheels fromhttp://www.mpia.de/~robitaille/python/wheels)astrofrog commentedon Aug 13, 2013
First I think we may want to upgrade the way Astropy deals with it based on code @Cadair has developed. I'll open an issue for this.
Cadair commentedon Aug 17, 2013
Hello,
I have been working on this for SunPy. I came to the conclusion that the easiest way to build wheels was on Travis so I created this repo: https://github.com/Cadair/travis_wheels that builds all the dependencies for SunPy and AstroPy and stores them here: http://sunpy.cadair.com/wheelhouse/.
I am happy to share this wheelhouse and accept PR's to the build repo for extra packages if people need them. There will need to be some restructuring of the build script if more packages need adding as the total build time is close to the Travis CI limit of 50 minutes.
Also I found a better solution to the installing packages by pip using wheels. pip can trawl a html file and use it as a package index so you can point pip to a wheelhouse by doing: --find-links http://yourdomian.com/wheelhouse/index.html --use-wheel which you can see an example of in the SunPy .travis.yml file: https://github.com/sunpy/sunpy/blob/staging/.travis.yml
HTH
Stuart
cdeil commentedon Aug 17, 2013
@Cadair If you find that approach of creating the wheel on travis-ci problematic for some reason, another simple way it to create locally (or in the cloud, I guess) a VM with the same Ubuntu version as travis-ci and build the wheels there. This is what I did for the ROOT library which takes too long to build for travis-ci and it works.
Cadair commentedon Aug 17, 2013
@cdeil Yes, that is an option. However, I thought that building the wheels in a way where people can see and change it, while making the code open was a good idea ;)
I can see if I can get ROOT working in the repo if you wish, I have a few ideas to get around the total build limit ;)
Cadair commentedon Aug 17, 2013
Also you should be able to use the wheels to expand your number of Travis configs, as you no doubt already know.
astrofrog commentedon Aug 17, 2013
@Cadair @cdeil - I've opened a pull request on the core package first so we can work on this:
astropy/astropy#1355
Once this works fine, we can port over these changes to the package template.
Merge pull request #936 from mdboom/docs/minimum-sphinx-version
astrofrog commentedon Sep 21, 2013
I'll start working on this. Note to self: also update minimum sphinx version to 0.2b2
cdeil commentedon Feb 9, 2014
@astrofrog Can you update the package-template
.travis.ymlto use wheels?I've got an example in the sphinx build that uses
scipyand thus fails with the current.travis.ymlfile:https://travis-ci.org/gammapy/gammapy/jobs/18529539#L974
astrofrog commentedon Feb 9, 2014
@cdeil - I'm on vacation for a week so won't be able to do it. But I agree this is a good idea. Could you open a pull request for this? Alternatively, you could just do it in your affiliated package but not add it to the template.
cdeil commentedon Feb 20, 2014
@astrofrog If you can show us how to use wheels for affiliated packages here in the package-template, that would be great!
weaverba137 commentedon May 6, 2014
Yet another wheels-related ticket that lots of people are waiting on.
embray commentedon May 6, 2014
IIRC I believe there might be movement away from that soon as Travis is getting better support for dependency caching. I'm not exactly sure what the last word was on that though. In the meantime there's nothing stopping anyone from updating their own
.travis.ymlto try pip installing from the wheel links.weaverba137 commentedon May 6, 2014
You're saying track the Travis configuration of astropy/astropy? I can do that, but it kind of diminishes the importance of package-template.
cdeil commentedon May 6, 2014
@embray Should we wait for #46 to be merged before updating the
.travis.ymlfor affiliated packages? Or is the.travis.ymlfile independent of these changes?embray commentedon May 6, 2014
No, #46 doesn't really have any impact on this that I'm aware of.
cdeil commentedon May 7, 2014
@mwcraig is implementing this in #44.
astrofrog commentedon May 23, 2014
Closed by #44