travis: add Bionic (Ubuntu 18.04) to test matrix by thaJeztah · Pull Request #3606 · containerd/containerd

thaJeztah · 2019-08-30T07:49:32Z

Using bionic (current LTS) as default, and adding xenial (Ubuntu 16.04 LTS) to the matrix, to test the previous LTS release as well

(see discussion on rootless-containers/usernetes#111)

thaJeztah · 2019-08-30T07:57:08Z

Okay, not yet there; it's now running the following Linux combinations:

distro	TEST_RUNTIME
bionic	io.containerd.runc.v1
bionic	io.containerd.runc.v2
bionic	io.containerd.runtime.v1.linux
xenial	io.containerd.runc.v1

So I either have to write-out all combinations, or at least 3 for xenial

theopenlab-ci · 2019-08-30T08:09:27Z

Build succeeded.

containerd-build-arm64 : SUCCESS in 8m 48s (non-voting)

codecov-io · 2019-08-30T08:13:18Z

Codecov Report

Merging #3606 into master will increase coverage by 0.04%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3606      +/-   ##
==========================================
+ Coverage   42.38%   42.43%   +0.04%     
==========================================
  Files         126      126              
  Lines       13900    13919      +19     
==========================================
+ Hits         5891     5906      +15     
- Misses       7123     7125       +2     
- Partials      886      888       +2

Flag	Coverage Δ
#linux	`45.96% <ø> (+0.05%)`	⬆️
#windows	`37.2% <ø> (ø)`	⬆️

Impacted Files	Coverage Δ
cio/io_unix.go	`75% <0%> (+0.88%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 95301fe...fa546dc. Read the comment docs.

thaJeztah · 2019-08-30T08:23:29Z

All working now:

distro	TEST_RUNTIME
bionic	io.containerd.runc.v1
bionic	io.containerd.runc.v2
bionic	io.containerd.runtime.v1.linux
xenial	io.containerd.runc.v1
xenial	io.containerd.runc.v2
xenial	io.containerd.runtime.v1.linux

thaJeztah · 2019-08-30T08:27:44Z

The only difficulty is that travis doesn't clearly show what distro each build is in the overview page; I added a dummy env (TRAVIS_DISTRO=xenial) for easier identification

thaJeztah · 2019-08-30T08:35:09Z

@AkihiroSuda this is working now; ptal 👍

theopenlab-ci · 2019-08-30T08:38:10Z

Build succeeded.

containerd-build-arm64 : SUCCESS in 8m 45s (non-voting)

thaJeztah · 2019-08-30T09:01:09Z

force-pushed to trigger CI again (linter took to long)

theopenlab-ci · 2019-08-30T09:10:04Z

Build succeeded.

containerd-build-arm64 : SUCCESS in 8m 18s (non-voting)

thaJeztah · 2019-08-30T10:28:56Z

👍 all green now

crosbymichael · 2019-08-30T19:02:30Z

This increases test time for a PR from 20min to 40min. I'm not sure its worth the change, we want our PRs to be fast and provide quick feedback to users.

Since containerd does not code to a specific distribution, I don't think it makes sense to have a matrix for this.

estesp · 2019-08-30T19:10:51Z

This runs 7 jobs per PR and merge to master and we get 5 parallel from Travis; similar to ppc64le (which also had a perf. issue) means longer PR runs especially when we get multiple PRs in process. Any chance we could minimize runs on the older distro (less of the runtime shim variants maybe?)

thaJeztah · 2019-08-30T19:21:48Z

Ah, didn't know there was a limit of 5 parallel.

I initially wanted to update to Bionic, but @AkihiroSuda mentioned we may want to also test on the older LTS, that's why I added the matrix;

I can remove two of those, or create different combinations

thaJeztah · 2019-08-30T19:23:12Z

Any preference which runtime(s) to test on Xenial / Bionic?

crosbymichael · 2019-08-30T19:53:01Z

Can we make it so that master tests on all of them but PRs only run on the new ones?

thaJeztah · 2019-08-30T19:56:19Z

good one; let me google a bit if travis can do different things based on the branch

thaJeztah · 2019-08-30T20:00:15Z

looks like there's such an option; travis-ci/travis-ci#7149 (comment)

Using bionic (current LTS) as default, and add xenial (Ubuntu 16.04 LTS) to the matrix, to test the previous LTS release as well on master Signed-off-by: Sebastiaan van Stijn <[email protected]>

Signed-off-by: Sebastiaan van Stijn <[email protected]>

theopenlab-ci · 2019-09-01T12:43:10Z

Build succeeded.

containerd-build-arm64 : SUCCESS in 9m 30s (non-voting)

The branch (for pull requests), is the base/target branch, so for pull requests against master, these would still be run. From the travis documentation: > branch (the current branch name; for pull requests: the base branch name) This patch excludes these jobs by not running them for pull request (event type=pull_request or event type=push (when rebasing a pull request)). Signed-off-by: Sebastiaan van Stijn <[email protected]>

thaJeztah · 2019-09-01T12:49:49Z

I think I have it setup correctly now; just filtering on branch apparently doesn't work, because for pull requests, Travis uses the base branch as branch (so they would still run); I added an extra condition to only run on pull_request and push events (assuming the first gets triggered on pull requests, and the second when rebasing/pushing).

Not sure how to test if it now correctly runs on master (without merging this PR)

theopenlab-ci · 2019-09-01T12:57:49Z

Build succeeded.

containerd-build-arm64 : SUCCESS in 9m 33s (non-voting)

thaJeztah · 2019-09-03T20:21:24Z

@crosbymichael @estesp ptal

crosbymichael · 2019-09-03T20:25:24Z

LGTM

estesp

LGTM

estesp · 2019-09-03T20:41:27Z

hmm.. https://travis-ci.org/containerd/containerd/builds/580419692
(edit: for clarity--this link is the master "merge PR" Travis run after PR merge)

Maybe needs some more tweaking to get master merge runs on xenial added?

thaJeztah · 2019-09-03T23:23:05Z

Maybe needs some more tweaking to get master merge runs on xenial added?

Ah booh! The docs is quite confusing on this 😞, and I have no idea how to test before actually merging the changes 😢

I wonder what type of event is needed to run in on "master" (would that be a push event?)

Also, looking at this again; filtering on branch may not even be needed (as in; once merged, I think we want to run this on other branches (e.g. release/xxx) as well

if: branch = master AND type NOT IN (push, pull_request)

Perhaps just this would do;

if: type NOT IN (pull_request)

crosbymichael · 2019-09-04T15:18:25Z

I think it's a push event on master

thaJeztah · 2019-09-04T16:59:22Z

yup; changed it in #3614 and looks like that's working

thaJeztah force-pushed the bump_travis_bionic branch from 265816b to 874a4a4 Compare August 30, 2019 07:59

thaJeztah marked this pull request as ready for review August 30, 2019 08:34

thaJeztah force-pushed the bump_travis_bionic branch from 24018dc to 5a446ac Compare August 30, 2019 09:00

AkihiroSuda approved these changes Aug 30, 2019

View reviewed changes

thaJeztah added 2 commits September 1, 2019 14:30

travis: add Bionic (Ubuntu 18.04 LTS) to test matrix

214b832

Using bionic (current LTS) as default, and add xenial (Ubuntu 16.04 LTS) to the matrix, to test the previous LTS release as well on master Signed-off-by: Sebastiaan van Stijn <[email protected]>

travis.yml: add TRAVIS_DISTRO env-var for easier identification

94e4b68

Signed-off-by: Sebastiaan van Stijn <[email protected]>

thaJeztah force-pushed the bump_travis_bionic branch from 5a446ac to 94e4b68 Compare September 1, 2019 12:32

estesp approved these changes Sep 3, 2019

View reviewed changes

estesp merged commit 48fb479 into containerd:master Sep 3, 2019

thaJeztah deleted the bump_travis_bionic branch September 3, 2019 23:16

thaJeztah mentioned this pull request Sep 3, 2019

travis: fix Xenial tests not being run on master #3614

Merged

Conversation

thaJeztah commented Aug 30, 2019

Uh oh!

thaJeztah commented Aug 30, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

theopenlab-ci Bot commented Aug 30, 2019

Uh oh!

codecov-io commented Aug 30, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

thaJeztah commented Aug 30, 2019

Uh oh!

thaJeztah commented Aug 30, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thaJeztah commented Aug 30, 2019

Uh oh!

theopenlab-ci Bot commented Aug 30, 2019

Uh oh!

thaJeztah commented Aug 30, 2019

Uh oh!

theopenlab-ci Bot commented Aug 30, 2019

Uh oh!

thaJeztah commented Aug 30, 2019

Uh oh!

crosbymichael commented Aug 30, 2019

Uh oh!

estesp commented Aug 30, 2019

Uh oh!

thaJeztah commented Aug 30, 2019

Uh oh!

thaJeztah commented Aug 30, 2019

Uh oh!

crosbymichael commented Aug 30, 2019

Uh oh!

thaJeztah commented Aug 30, 2019

Uh oh!

thaJeztah commented Aug 30, 2019

Uh oh!

theopenlab-ci Bot commented Sep 1, 2019

Uh oh!

thaJeztah commented Sep 1, 2019

Uh oh!

theopenlab-ci Bot commented Sep 1, 2019

Uh oh!

thaJeztah commented Sep 3, 2019

Uh oh!

crosbymichael commented Sep 3, 2019

Uh oh!

estesp left a comment

Choose a reason for hiding this comment

Uh oh!

estesp commented Sep 3, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thaJeztah commented Sep 3, 2019

Uh oh!

crosbymichael commented Sep 4, 2019

Uh oh!

thaJeztah commented Sep 4, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

thaJeztah commented Aug 30, 2019 •

edited

Loading

codecov-io commented Aug 30, 2019 •

edited

Loading

thaJeztah commented Aug 30, 2019 •

edited

Loading

estesp commented Sep 3, 2019 •

edited

Loading