Repository navigation
Closed
Labels
area/release-engIssues or PRs related to the Release Engineering subprojectIssues or PRs related to the Release Engineering subprojectkind/kepCategorizes KEP tracking issues and PRs modifying the KEP directoryCategorizes KEP tracking issues and PRs modifying the KEP directorylifecycle/activeIndicates that an issue or PR is actively being worked on by a contributor.Indicates that an issue or PR is actively being worked on by a contributor.priority/important-longtermImportant over the long term, but may not be staffed and/or may need multiple releases to complete.Important over the long term, but may not be staffed and/or may need multiple releases to complete.sig/releaseCategorizes an issue or PR as relevant to SIG Release.Categorizes an issue or PR as relevant to SIG Release.sig/testingCategorizes an issue or PR as relevant to SIG Testing.Categorizes an issue or PR as relevant to SIG Testing.stage/stableDenotes an issue tracking an enhancement targeted for Stable/GA statusDenotes an issue tracking an enhancement targeted for Stable/GA status
Description
Enhancement Description
- One-line enhancement description (can be used as a release note): Reduce Kubernetes build maintenance by moving to a single build system.
- Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/tree/master/keps/sig-testing/2420-reducing-kubernetes-build-maintenance/
- Discussion Link: Please remove bazel based build infrastructure kubernetes#88553 (initial community conversation, will circle back to surface the rest)
- Primary contact (assignee): @BenTheElder
- Responsible SIGs: SIG-Testing, SIG-Release
- Enhancement target (which target equals to which milestone):
- Alpha release target (x.y): 1.21
- Beta release target (x.y): 1.21
- Stable release target (x.y): 1.23
- Alpha
- KEP (
k/enhancements) update PR(s): add KEP 2420: Reducing Kubernetes Build Maintenance #2421 Update Reducing Kubernetes Build Maintenance to implementable #2469 - Code (
k/k) update PR(s): Remove Bazel kubernetes#99561 (main PR, full list in Reducing Kubernetes Build Maintenance #2420 (comment), it's ... long) - Docs (
k/website) update PR(s): N/A? (developer docs will be updated, but not k/website)
- KEP (
- Beta
- KEP (
k/enhancements) update PR(s): update reducing kubernetes build maintenance status #2597 - Code (
k/k) update PR(s): N/A (same as alpha) - Docs (
k/website) update(s): N/A (not user facing)
- KEP (
- Stable
- KEP (
k/enhancements) update PR(s): mark reducing build maintenance implemented #3209 - Code (
k/k) update PR(s): N/A (same as alpha) - Docs (
k/website) update(s): N/A (not user facing)
- KEP (
Additional detailed PR listing below #2420 (comment)
Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently.
/assign
/sig testing
/sig release
Reacted by Phan Văn Hoà - 潘文和 - 반문화
Metadata
Metadata
Assignees
Labels
area/release-engIssues or PRs related to the Release Engineering subprojectIssues or PRs related to the Release Engineering subprojectkind/kepCategorizes KEP tracking issues and PRs modifying the KEP directoryCategorizes KEP tracking issues and PRs modifying the KEP directorylifecycle/activeIndicates that an issue or PR is actively being worked on by a contributor.Indicates that an issue or PR is actively being worked on by a contributor.priority/important-longtermImportant over the long term, but may not be staffed and/or may need multiple releases to complete.Important over the long term, but may not be staffed and/or may need multiple releases to complete.sig/releaseCategorizes an issue or PR as relevant to SIG Release.Categorizes an issue or PR as relevant to SIG Release.sig/testingCategorizes an issue or PR as relevant to SIG Testing.Categorizes an issue or PR as relevant to SIG Testing.stage/stableDenotes an issue tracking an enhancement targeted for Stable/GA statusDenotes an issue tracking an enhancement targeted for Stable/GA status
Activity
[-]Reducing Build Maintenance[/-][+]Reducing Kubernetes Build Maintenance[/+]spiffxp commentedon Feb 8, 2021
Initial draft PR: #2421
BenTheElder commentedon Feb 8, 2021
followup for implementable #2469
BenTheElder commentedon Feb 9, 2021
Discussed today in SIG Testing meeting, no concerns there. (this has also been circulated in SIG Testing and Release previously).
BenTheElder commentedon Feb 25, 2021
UPDATE 02/24/2021
We moved
pull-kubernetes-e2e-kind-ipv6to build without bazel, so far so good. kubernetes/test-infra#21009We're now running
pull-kubernetes-unit(non blocking) which runs ~make testwith the race detector enabled as an approximate equivilant topull-kubernetes-bazel-test(our heaviest bazel dependency in CI). This is working well. kubernetes/test-infra#21008Nominally speaking we are on track to not depend on this build in CI, opening the path to notify and remove.
However there are a handful of test targets in
pull-kubernetes-bazel-testthat are not golang unit tests and therefore will not be covered bymake test.I produced the following list by inspecting the junit output and ignoring
:go_default_testtargets (go unit tests under bazel).//cluster:common_testPer
BUILD:This one I think we can ignore. This doesn't appear to function as intended and we don't normally do this.
EDIT: kubernetes/kubernetes#99486 to remove. See details there.
//cluster:kube-util_testPer
BUILD:This one I think we can ignore. This doesn't appear to function as intended and we don't normally do this.
EDIT: kubernetes/kubernetes#99486 to remove. See details there.
//hack/boilerplate:boilerplate_testIMPORTANTPython script that ensures the boilerplate headers are in place.
This one is already covered by
pull-kubernetes-verify/make verifyso we don't need to do anything.//pkg/util:verify-util-pkgThis is a
sh_testthat basically ensures that ./pkg/util/ only contains code in subdirectories, not ./pkg/util/foo.goI'm not sure why we couldn't just let approvers handle this, it seems a bit heavyweight, but if we still need it we can move it to
pull-kubernetes-verify/make verify.EDIT: Another approach we could take here is to use prow's
blockadeplugin to block^pkg/util/[^/]*\.goby automatically addingdo-not-merge/blocked-pathslabel and posting a configurable explanation comment.EDIT: Fixed by kubernetes/test-infra#21037 (blockade)
EDIT: kubernetes/kubernetes#99486 to remove.
See: kubernetes/kubernetes#15634
//test/conformance:conformance_testIMPORTANTThis is a
sh_testthat ensures the generated conformance.yaml is up to date.We should probably rework this to run under
make verify/pull-kubernetes-verify.EDIT: fixed in kubernetes/kubernetes#99549 to work under
make verifyandhack/update-conformance-yaml.shdims commentedon Feb 25, 2021
cc @navidshaikh
BenTheElder commentedon Feb 26, 2021
Update 02/25/2021
pull-kubernetes-unitis looking pretty good https://prow.k8s.io/?job=pull-kubernetes-unitpkg/controlplanesometimes times out at the current 240s limit we're setting formake test, we could raise it, but we should fix that test Umbrella issue: slow/timing-out unit tests kubernetes#98486pull-kubernetes-e2e-kind-ipv6is working fine building with make with ~30m total runtime https://prow.k8s.io/?job=pull-kubernetes-e2e-kind-ipv6. Writing a PR to switch more jobs currently.Filed add blockade to kubernetes for pkg/util test-infra#21037 for 4) on the list above Reducing Kubernetes Build Maintenance #2420 (comment), checking that one off.
Filed Cleanup old test targets kubernetes#99486 to cleanup 1) 2) 4) from above.
aojea commentedon Feb 27, 2021
/cc
BenTheElder commentedon Feb 28, 2021
Update 02/28/2021
I finished migrating the e2e jobs for the Kubernetes/Kubernetes master branch to make AFAICT.
I've opened conformance.yaml: generate with make kubernetes#99549 to address migrating conformance.yaml / 5) above (Reducing Kubernetes Build Maintenance #2420 (comment))
Once 2) is done, we can flip
pull-kubernetes-bazel-test->pull-kubernetes-unitas blocking (whenconformance.yamlis covered inpull-kubernetes-verify).EDIT: 2) is done. The first follow-up is in kubernetes/test-infra#21070 (block PRs on
pull-kubernetes-unit)After that I'll have a PR out to rewrite
hack/update-bazel.shto drop all BUILD and cleanup any remaining references. We will hold that PR until we notify kubernetes-dev@ etc.When that is done we can drop the
-bazel-buildKubernetes jobs.Finally we can let the remaining release branches age out.
40 remaining items
BenTheElder commentedon Sep 23, 2021
n.b.: #2420 (comment) is a note from the bi-weekly SIG testing meeting, so I agree with my own comment ;-)
just waiting for 1.19 turndown from SIG release
BenTheElder commentedon Nov 4, 2021
More cleanup PRs:
We are on-track to stable with the turn down of 1.19 related Kubernetes project resources. There's a bit more to do but we're nearly done.
Next up: We run a reasonably large custom bazel cache (greenhouse) instance in each CI cluster primarily aimed at Kubernetes/Kubernetes, we should either turn this down or at least reduce the capacity / operational costs significantly.
kevindelgado commentedon Nov 8, 2021
Thanks for the update @BenTheElder! Still gonna give you the copy/pasta with all the deets:
Hi @BenTheElder 👋
Checking in once more as we approach 1.23 code freeze at 6:00 pm PST on Tuesday, November 16.
Please ensure the following items are completed:
As always, we are here to help should questions come up.
Thanks!!
spiffxp commentedon Nov 24, 2021
We're past code freeze and test freeze at this point, I'm hoping / assuming nothing remains to be done in kubernetes/kubernetes itself. What tasks do we need to complete to call this done?
BenTheElder commentedon Nov 24, 2021
There's some pretty minor stuff but we could probably close it out now that we've done the rest.
BenTheElder commentedon Nov 24, 2021
none of those references are in production release binary code or release / build scripts though, just in tests, publishing bot config, that sort of thing.
gracenng commentedon Jan 9, 2022
Hi @BenTheElder ,
1.24 Enhancements Lead here. Could you please update this enhancements's KEP status to
implemented, then close this issue?Thanks :)
BenTheElder commentedon Feb 4, 2022
filed #3209, we can follow up with any additional minor cleanup.
BenTheElder commentedon Feb 4, 2022
@gracenng sorry for the delay, this is done!
drop bazel from kind build node-image