@estesp @stevvooe please see the 2 yaml files ... it just sets up a Zuulv3 Job that runs in openlab. Zuulv3 is the CI system we use in OpenLab.

Codecov Report

Merging #3242 into master will decrease coverage by 4.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #3242      +/-   ##
==========================================
- Coverage   48.65%   44.63%   -4.02%     
==========================================
  Files         102      113      +11     
  Lines        9570    12164    +2594     
==========================================
+ Hits         4656     5430     +774     
- Misses       4088     5899    +1811     
- Partials      826      835       +9

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3a3f0aa...dae0afe. Read the comment docs.

Codecov Report

Merging #3242 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #3242   +/-   ##
=======================================
  Coverage   44.56%   44.56%           
=======================================
  Files         113      113           
  Lines       12194    12194           
=======================================
  Hits         5434     5434           
  Misses       5928     5928           
  Partials      832      832

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d68b593...bdd7dce. Read the comment docs.

This looks reasonable. I went and looked at some of the logs and noticed only make and make test are performed, which is a different (smaller) set of tests than the set which are run in AppVeyor or Travis.

The good part of that is that it looks like CI build/run times are shorter than Travis or AppVeyor, but possibly could miss bugs that would be found via integration, root-enabled or parallel test runs. Any thoughts?

The other piece that comes to mind is that we would then be dependent on OpenLab infrastructure to get "green" PRs? It looks like that is not any more of an issue than the fact we rely (and sometimes are bit) by Travis and AppVeyor downtimes or issues, but any statement on reliability/use by other projects would be good to know.

@estesp Thanks for your reply. : )

The good part of that is that it looks like CI build/run times are shorter than Travis or AppVeyor, but possibly could miss bugs that would be found via integration, root-enabled or parallel test runs. Any thoughts?

Yes, I had same consideration on it. The containerd-arm64-build is a job to invalidate building an available arm64 release, so I only put make and make test in the job. If we want to do more test on containerd, we can also enable the integration, root-enabled or parallel test runs in this job.

Or just make the build job simple, and add more test latter into other separate jobs.

The other piece that comes to mind is that we would then be dependent on OpenLab infrastructure to get "green" PRs? It looks like that is not any more of an issue than the fact we rely (and sometimes are bit) by Travis and AppVeyor downtimes or issues, but any statement on reliability/use by other projects would be good to know.

The OpenLab have been used in many project with stable running everyday, but I still understand your concerns. Fortunately, we also provide the "non-voting" mechanism, we could mark the test job as non-voting, that means the gating will succeeded (it would not be a block things when we add a experimental job) even the job is failed.

You can see theopenlab#4 as the reference:

Once we make sure the job can be executed stable, then we can mark it as normal "voting" job. Hope it's a friendly road to make our containerd running in OpenLab smoothly. : )

I'm fine with this, and it seems to have no impact on the project in any way if we start it in a "non-voting" check mode. Thoughts from other @containerd/containerd-maintainers and @containerd/containerd-reviewers?

I'm fine with this, and it seems to have no impact on the project in any way if we start it in a "non-voting" check mode. Thoughts from other @containerd/containerd-maintainers and @containerd/containerd-reviewers?

SGTM.

SGTM

@estesp @Random-Liu @ehazlett looks like the jobs are still going on fine, so when this gets merged @Yikun can make any necessary changes on the openlab side. So please go ahead and merge this when appropriate

[1] http://status.openlabtesting.org/builds?job_name=containerd-build-arm64

LGTM