Skip to content
This repository was archived by the owner on Nov 17, 2023. It is now read-only.

[CI][1.6.x] fix centos 7 url to unblock centos-cpu & gpu pipeline #18560

Merged
szha merged 3 commits into from
Jun 15, 2020
Merged

[CI][1.6.x] fix centos 7 url to unblock centos-cpu & gpu pipeline #18560

szha merged 3 commits into from
Jun 15, 2020

Conversation

@ChaiBapchya
Copy link
Contributor

@ChaiBapchya ChaiBapchya commented Jun 14, 2020 •
edited
Loading

Currently, 1.6.x branch is failing on multiple pipelines

  1. centos-cpu & centos-gpu pipelines due to
Cannot open: https://centos7.iuscommunity.org/ius-release.rpm. Skipping.

http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Fcentos-cpu/detail/v1.6.x/51/pipeline

  1. Flaky test on unix-cpu mkldnn
  2. Edge pipeline

@mxnet-bot
Copy link

mxnet-bot commented Jun 14, 2020

Hey @ChaiBapchya , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

  • To trigger all jobs: @mxnet-bot run ci [all]
  • To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [clang, website, sanity, unix-cpu, miscellaneous, centos-gpu, windows-cpu, unix-gpu, edge, windows-gpu, centos-cpu]

Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

@waytrue17 @leezu @ChaiBapchya
[v1.7.x] update jetson dockerfile to support CUDA 10.0 (apache#18339)
843c81c 
* update dockerfile for jetson

* add toolchain files

* update build_jetson function

* update ubuntu_julia.sh

* update FindCUDAToolkit.cmake

* Update centos7_python.sh

* revert changes on ubuntu_julia.sh

* disable TVM for gpu build

* Disable TVM_OP on GPU builds

Co-authored-by: Wei Chu <[email protected]>
Co-authored-by: Leonard Lausen <[email protected]>
@ChaiBapchya ChaiBapchya requested a review from szha as a code owner June 14, 2020 22:22
@xinyu-intel @ChaiBapchya
skip quantized conv flaky case (apache#16866)
b426623 
* Fix quantized concat when inputs are mixed int8 and uint8

Change-Id: I4da04bf4502425134a466823fb5f73da2d7a419b

* skip flaky test

* trigger ci
@ChaiBapchya ChaiBapchya changed the title [CI] fix centos 7 url to unblock centos-cpu & gpu pipeline [CI][1.6.x] fix centos 7 url to unblock centos-cpu & gpu pipeline Jun 15, 2020
@szha szha merged commit d271348 into apache:v1.6.x Jun 15, 2020
@ChaiBapchya ChaiBapchya mentioned this pull request Jun 15, 2020
Merged
@ChaiBapchya
Copy link
Contributor Author

ChaiBapchya commented Jun 15, 2020 •
edited
Loading

@szha can we revert this PR? While it fixed centos-cpu & centos-gpu
While trying to fix the edge pipeline, I cherry-picked #18339 into 1.6.x [which helped resolve edge pipeline issue for 1.7.x] but it ended up breaking unix-gpu: http://jenkins.mxnet-ci.amazon-ml.com/job/mxnet-validation/job/unix-gpu/job/v1.6.x/52/display/redirect

So as a result, unix-gpu & edge pipeline are now broken.

Way forward:

  1. Revert this PR
  2. I'll submit another PR just to fix centos-cpu & centos-gpu [by updating the link : https://github.com/[CI][1.6.x] fix centos 7 url to unblock centos-cpu & gpu pipeline #18560/commits/9103f2088890a1190cc2b9865e8c1b3f3fc7c82c ]
  3. I will leave edge pipeline broken & get help from @weichu & @leezu to unblock edge pipeline on 1.6.x

The reason why it showed you green was because it only ran sanity pipeline & didn't run other 10 pipelines. allowing you to merge with just 1 pipeline instead of other 10.
@josephevans Is there a way to prevent merge a branch if only sanity has passed but other 10 pipelines haven't even run/triggered?

@leezu
Copy link
Contributor

leezu commented Jun 15, 2020

@josephevans Is there a way to prevent merge a branch if only sanity has passed but other 10 pipelines haven't even run/triggered?

This needs to be configured in Github Project Settings. There is a setting related to "require checks". It's currently only turned on for the master branch. This may require Apache Infra to change the setting.

@szha
Copy link
Member

szha commented Jun 15, 2020

@ChaiBapchya I think it's ok to fix forward.

@marcoabreu
Copy link
Contributor

marcoabreu commented Jun 15, 2020 •
edited
Loading

We opted to not turn that in for feature branches to not bother infra too much and also allow the release manager to make certain calls without hitting limits. It's the general policy within the project that PRs with failed CI do not get merged.

If a PR was merged, then the committer who merged the PR should be contacted and reminded. But I think sheng is aware of his mistake :)

@szha
Copy link
Member

szha commented Jun 16, 2020

We opted to not turn that in for feature branches to not bother infra too much and also allow the release manager to make certain calls without hitting limits.

I don't think that's the case. We didn't have explicit discussion on this, nor do I think it's the right approach. It doesn't make sense to allow force push to release branches while protecting the development branch. Also, enabling branch protection for release branches doesn't necessarily "bother" apache infra either, as the setup is likely one-time. Branch protection can be turned on for branches that match a pattern, and we do have an explicit pattern for release branches according to the release process.

@ChaiBapchya
Copy link
Contributor Author

ChaiBapchya commented Jun 17, 2020

Makes sense.
Can someone then help create this branch protection ticket to apache-infra? @leezu @marcoabreu
I'm not sure if I as a contributor can do that. Thanks everyone for the advice & clarification.

@marcoabreu
Copy link
Contributor

marcoabreu commented Jun 17, 2020

We'd need a vote on dev@ first

@szha
Copy link
Member

szha commented Jun 18, 2020

we should be able to do so with a lazy consensus.

@marcoabreu
Copy link
Contributor

marcoabreu commented Jun 18, 2020

In any case there's a dev@ vote thread necessary

@ChaiBapchya ChaiBapchya mentioned this pull request Jun 20, 2020
Merged
ChaiBapchya added a commit to ChaiBapchya/mxnet that referenced this pull request Jun 21, 2020
@ChaiBapchya
Revert "[CI][1.6.x] fix centos 7 url to unblock centos-cpu & gpu pipe…
ae187d9 
…line (apache#18560)"

This reverts commit d271348.
sandeep-krishnamurthy pushed a commit that referenced this pull request Jul 1, 2020
@ChaiBapchya @xinyu-intel
@waytrue17 @leezu
[CI][v1.6.x] Fix failing CI pipelines (#18597)
fb3fea4 
* add the missing build_ubuntu_gpu_cuda101_cudnn7_mkldnn_cpp_test in runtime_functions.sh

* Revert "add the missing build_ubuntu_gpu_cuda101_cudnn7_mkldnn_cpp_test in runtime_functions.sh"

This reverts commit de173b0.

* Revert "[CI][1.6.x] fix centos 7 url to unblock centos-cpu & gpu pipeline (#18560)"

This reverts commit d271348.

* fix centos 7 url to unblock centos-cpu & gpu pipeline

* skip quantized conv flaky case (#16866)

* Fix quantized concat when inputs are mixed int8 and uint8

Change-Id: I4da04bf4502425134a466823fb5f73da2d7a419b

* skip flaky test

* trigger ci

* Trigger empty commit

* [v1.7.x] update jetson dockerfile to support CUDA 10.0 (#18339)

* update dockerfile for jetson

* add toolchain files

* update build_jetson function

* update ubuntu_julia.sh

* update FindCUDAToolkit.cmake

* Update centos7_python.sh

* revert changes on ubuntu_julia.sh

* disable TVM for gpu build

* Disable TVM_OP on GPU builds

Co-authored-by: Wei Chu <[email protected]>
Co-authored-by: Leonard Lausen <[email protected]>

* add setuptools to ci/docker/install/requirements

* add missing build_ubuntu_gpu_cuda101_cudnn7_mkldnn_cpp_test

* add setuptool to docker & cpp-test build syntax error

* remove erroneously added cpp tests in 1.6.x

* py3 to p2

Co-authored-by: Xinyu Chen <[email protected]>
Co-authored-by: waytrue17 <[email protected]>
Co-authored-by: Wei Chu <[email protected]>
Co-authored-by: Leonard Lausen <[email protected]>
@ChaiBapchya ChaiBapchya deleted the fix_centos_v16x branch August 11, 2020 08:40
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

@aaronmarkham aaronmarkham Awaiting requested review from aaronmarkham aaronmarkham is a code owner

@szha szha Awaiting requested review from szha szha is a code owner

1 more reviewer

@marcoabreu marcoabreu marcoabreu approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@ChaiBapchya @mxnet-bot @leezu @szha @marcoabreu @waytrue17 @xinyu-intel