fix docker builds/remove extra builds/add ci builds by opadron · Pull Request #11621 · spack/spack

opadron · 2019-06-04T17:40:41Z

Fixes several issues that prevented the docker image builds from completing. Also, removes and simplifies much of the current docker offerings.

Removed Images

spack/ubuntu
spack/scilinux
spack/opensuse
spack/fedora
spack/centos
spack/archlinux

Now, all images meant for general use are under the single spack/spack image name, with tags to differentiate between versions of Spack and OS. In addition, this PR integrates work from @scottwittenburg that brings the docker images we use for our Gitlab CI process into the Spack repo. These images are under the spack/ci name.

Added Images and Tags

The tags follow the general format {SPACK_VERSION}-${OS_NAME_AND_VERSION}. Aliasing tags are included so that SPACK_VERSION and/or OS_NAME_AND_VERSION can be omitted. The default version of Spack is whatever is the "latest" version at the time, and the default OS + version is "centos-7". Note that per Docker tagging standards, the tag latest is used for the case where both are omitted, as it is the default tag used when none is provided. This is a special case, and there is no other corresponding tag as in e.g.: spack/spack:latest-ubuntu-18.04; such a tag is simply spelled spack/spack:ubuntu-18.04.

Image & Tag	Contents
spack/spack:{VERSION}-centos-6	The given Spack version on centos 6.
spack/spack:{VERSION}-centos-7	The given Spack version on centos 7.
spack/spack:{VERSION}	The given Spack version on centos 7.
spack/spack:{VERSION}-ubuntu-16.04	The given Spack version on ubuntu 16.04.
spack/spack:{VERSION}-ubuntu-xenial	The given Spack version on ubuntu 16.04.
spack/spack:{VERSION}-ubuntu-18.04	The given Spack version on ubuntu 18.04.
spack/spack:{VERSION}-ubuntu-bionic	The given Spack version on ubuntu 18.04.
spack/spack:centos-6	The latest Spack version on centos 6.
spack/spack:centos-7	The latest Spack version on centos 7.
spack/spack:latest	The latest Spack version on centos 7.
~~spack/ci:centos-7~~	~~CI image with the latest version of Spack on centos 7.~~
~~spack/ci:ubuntu-18.04~~	~~CI image with the latest version of Spack on ubuntu-18.04.~~

Related to-do items.

Remove old dockerhub repos/add new dockerhub repos.
Decide on a policy for mapping Spack version to docker tag.
- Right now, the current PR just checks for the output of spack --version and assumes that we are always building using the "latest" version of Spack. Is this behavior enough?
~~Finish S3 work (S3 upload and download #11117) and start populating the public mirror with binary packages.~~
- I expect this step to be necessary because it takes a very long time to build e.g.: llvm in a docker container (for the centos CI image); far longer than I expect travis to run before timing out. Once binary packages become available, this issue should go away. EDIT: This should no longer be necessary, since we have eliminated the CI images in favor of "bootstrapping" compilers as part of the CI process.
Also, once these builds are running successfully and consistently, I think we should turn them "back on" in the Travis build matrix, that is, allow their failures to fail the whole build. Looking at docker hub, it's clear that we haven't been building up-to-date images for the last several months. If not from Travis, we should at least have some kind of notification that our docker images have stopped building.

EDIT: Added Travis todo item.

scottwittenburg

I didn't try building with these, but just compared your new stuff with what we had before. It looks good to me.

Your comment in the description about how long it takes to build llvm made me wonder if we should find a way to get the llvm binary pushed into the mirror (by hook or by crook) sooner rather than later. My branch (from #11612) could do it fairly easily, let me know if you think that makes sense.

share/spack/docker/ci/Dockerfile

scottwittenburg · 2019-06-06T15:51:45Z

@opadron I noticed that my PR (#11612 ) has the same Travis CI issue as this one, it's forever stuck in "Waiting for status to be reported". However, other PRs seem to be getting tested ok. Do you have any idea what's going on there? I haven't looked into yet, but I will soon, assuming you don't already know.

opadron · 2019-06-28T13:30:46Z

@tgamblin PTAL.

A few questions:

Should we delete the Spack repos on dockerhub that won't be used, anymore? (e.g.: spack/archlinux)
1. What about keeping spack/ubuntu and spack/centos? Might be nice to add a few extra aliasing tags under those repos (e.g.: spack/ubuntu:0.12.1-bionic as a shorter way to spell spack/spack:0.12.1-ubuntu-bionic). On the other hand, seems a bit awkward sandwiching the spack version between the OS and OS-version. Maybe have just e.g.: spack/ubuntu:0.12.1 based on whatever is the latest ubuntu image?
One of my TODO items listed above relates to detecting Spack version.

Decide on a policy for mapping Spack version to docker tag.
Right now, the current PR just checks for the output of spack --version and assumes that we are always building using the "latest" version of Spack. Is this behavior enough?

Would like to know what you think.

ax3l

LGTM.
I am uncertain why @tgamblin prefers a naming convention for tags over individual repos with simple version tags, but if you decide for this I am fine with it :)

ax3l · 2019-07-18T22:24:08Z

share/spack/docker/handle-prompt.sh


 __revparse_head() {
-    head="`git -C /spack rev-parse $@ HEAD 2>/dev/null`"
+    head="`git -C "$SPACK_ROOT" rev-parse $@ HEAD 2>/dev/null`"


Instead of fancy quote handlings I am a big fan of proper variable notations, if that helps: ${SPACK_ROOT}

Suggested change

head="`git -C "$SPACK_ROOT" rev-parse $@ HEAD 2>/dev/null`"

head="$(git -C ${SPACK_ROOT} rev-parse $@ HEAD 2>/dev/null)"

The original syntax was chosen for POSIX compliance.

oh, didn't know that syntax breaks it. good to know

I'm confused about why @ax3l's suggestion isn't POSIX compliant. I agree we need quotes for this. But the use of $() is POSIX-compliant and, honestly, preferred over un-nestable backticks.

ax3l · 2019-07-18T22:26:35Z

@opadron please rebase to fix the merge conflict :)

ax3l · 2019-07-18T23:53:27Z

@opadron CI says: E FileNotFoundError: [Errno 2] No such file or directory: '/home/travis/build/spack/spack/share/spack/docker/os-container-mapping.yaml'

tgamblin

This is good for now and it's a huge improvement over the previous set of scripts. I'm going to go ahead and merge it.

That said, I do think we should soon move to something like what @ax3l suggests for the naming scheme.

The table above is complicated and the special case is annoying. It would be more intuitive to just do FROM spack/centos7, FROM spack/ubuntu-xenial, FROM spack/ubuntu-xenial:0.12, etc. I think that scheme ultimately makes more sense to users, too. Added bonux: it also make it easier to set up automatic DockerHub integration and just let DockerHub build the images, and in that configuration the Dockerfile from each distro-specific repo would be automatically associated with the image on DockerHub, so we would get some points for reproducibility there. The fact that the Dockerfiles are still stored in the Spack repo is, I think, confusing to people who are used to the usual single-repo-with-Dockerfile convention.

ax3l · 2019-07-20T22:49:18Z

@opadron there seam to be some hickups left in CI: https://travis-ci.org/spack/spack/jobs/561513871

$ share/spack/qa/run-$TEST_SUITE-tests
share/spack/qa/run-docker-tests: line 75: ../../../bin/spack: No such file or directory
The command "share/spack/qa/run-$TEST_SUITE-tests" exited with 127.

tgamblin · 2019-07-20T23:18:16Z

@opadron: I created an example repo for centos6 on DockerHub. Seems like Dockerhub itself can do the container builds in response to GitHub, and all we really have to do is point to the centos6 Dockerfile. We can automatically add versions to the Centos6 image according to tags on the Spack repo. This seems like a good option if we don't want to fight with Travis, esp. since it automates the tagging.

ax3l · 2019-07-21T00:23:39Z

👍 Just as a side note: singularity hub operates the same way, even exclusively.

tgamblin · 2019-07-21T00:39:07Z

Build on that is done and the Dockerhub page already looks better:

https://hub.docker.com/r/spack/centos6

And it took a few min to set up. @opadron: let's use this. Can you set up some like this for centos6, ccentos7, ubuntu16, and ubuntu18? I think it's fine if the main spack container repo tracks latest ubuntu, if we want to keep that.

Note: I've done this for develop/latest. We likely need to backport the new Docker support onto the releases/v0.12 branch to get this supported properly in a 0.12 release. I think if we made a 0.12.2 cut to support docker images that would be fine. Later releases will work fine now that this is in.

* fix docker builds/remove extra builds/add ci builds * preinstall vim in CI builder images * simplify & streamline docker resources * restore os-container-mapping.yaml file

opadron added mirrors repos containers docker binary-packages ecp utilities labels Jun 4, 2019

opadron requested a review from scottwittenburg June 4, 2019 17:40

opadron mentioned this pull request Jun 4, 2019

Rethink release workflow in terms of environments #11612

Merged

opadron requested a review from tgamblin June 4, 2019 18:22

scottwittenburg approved these changes Jun 4, 2019

View reviewed changes

share/spack/docker/ci/Dockerfile Outdated Show resolved Hide resolved

share/spack/docker/ci/Dockerfile Outdated Show resolved Hide resolved

share/spack/docker/ci/Dockerfile Outdated Show resolved Hide resolved

opadron requested a review from ax3l July 18, 2019 22:17

Omar Padron added 2 commits July 18, 2019 18:22

fix docker builds/remove extra builds/add ci builds

3ba11ca

preinstall vim in CI builder images

3fe4792

ax3l approved these changes Jul 18, 2019

View reviewed changes

simplify & streamline docker resources

b3526a4

opadron force-pushed the docker-fix-and-streamline branch from d39e965 to b3526a4 Compare July 18, 2019 22:26

ax3l approved these changes Jul 18, 2019

View reviewed changes

restore os-container-mapping.yaml file

0eae41d

tgamblin approved these changes Jul 20, 2019

View reviewed changes

tgamblin merged commit 1d5ab13 into spack:develop Jul 20, 2019

tgamblin mentioned this pull request Aug 22, 2019

Create and push to DockerHub some images we can use for building release packages. #9200

Closed

opadron deleted the docker-fix-and-streamline branch September 2, 2019 15:27

ax3l mentioned this pull request Nov 20, 2019

Package Index: Build in Dockerhub #13810

Merged

2 tasks

	head="`git -C "$SPACK_ROOT" rev-parse $@ HEAD 2>/dev/null`"
	head="$(git -C ${SPACK_ROOT} rev-parse $@ HEAD 2>/dev/null)"

Conversation

opadron commented Jun 4, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Removed Images

Added Images and Tags

Related to-do items.

Uh oh!

scottwittenburg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

scottwittenburg commented Jun 6, 2019

Uh oh!

opadron commented Jun 28, 2019

Uh oh!

ax3l left a comment

Choose a reason for hiding this comment

Uh oh!

ax3l Jul 18, 2019

Choose a reason for hiding this comment

Uh oh!

opadron Jul 18, 2019

Choose a reason for hiding this comment

Uh oh!

ax3l Jul 18, 2019

Choose a reason for hiding this comment

Uh oh!

tgamblin Jul 20, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ax3l commented Jul 18, 2019

Uh oh!

ax3l commented Jul 18, 2019

Uh oh!

tgamblin left a comment

Choose a reason for hiding this comment

Uh oh!

ax3l commented Jul 20, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tgamblin commented Jul 20, 2019

Uh oh!

ax3l commented Jul 21, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tgamblin commented Jul 21, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

opadron commented Jun 4, 2019 •

edited

Loading

tgamblin Jul 20, 2019 •

edited

Loading

ax3l commented Jul 20, 2019 •

edited

Loading

ax3l commented Jul 21, 2019 •

edited

Loading