Skip to content

k8s: generate node name if not provided#1673

Merged
tonistiigi merged 1 commit intodocker:masterfrom
crazy-max:fix-k8s-deploy-name
Mar 14, 2023
Merged

k8s: generate node name if not provided#1673
tonistiigi merged 1 commit intodocker:masterfrom
crazy-max:fix-k8s-deploy-name

Conversation

@crazy-max
Copy link
Copy Markdown
Member

@crazy-max crazy-max commented Mar 11, 2023 •
edited
Loading

follow-up docker/setup-buildx-action#215

Appending nodes to a kubernetes builder currently fails if node name is not set because it would be a duplicated endpoint:

Creating a new builder instance
  /usr/bin/docker buildx create --name builder-ec5ab75e-cc9a-4e71-b560-72962afcfd54 --driver kubernetes --driver-opt "namespace=buildkit" --driver-opt "nodeselector=""kubernetes.io/arch=arm64""" --buildkitd-flags --allow-insecure-entitlement security.insecure --allow-insecure-entitlement network.host --platform linux/amd64 --use
  builder-ec5ab75e-cc9a-4e71-b560-72962afcfd54
Appending node(s) to builder
  /usr/bin/docker buildx create --name builder-ec5ab75e-cc9a-4e71-b560-72962afcfd54 --append --driver-opt "namespace=buildkit" --driver-opt "nodeselector=""kubernetes.io/arch=arm64""" --platform linux/arm64
  ERROR: invalid duplicate endpoint kubernetes:///builder-ec5ab75e-cc9a-4e71-b560-72962afcfd54?deployment=&kubeconfig=

This change will generate a random node name if not defined for the kubernetes driver:

$ docker buildx create --config=/tmp/buildkitd.toml --bootstrap --name=buildx-test-d7b70bd217ad8da4932c8ea2c2767014 --node= --platform=linux/amd64 --driver=kubernetes --driver-opt=image=moby/buildkit:master,qemu.install=true
#1 [internal] booting buildkit
#1 waiting for 1 pods to be ready
#1 waiting for 1 pods to be ready 19.0s done
buildx-test-d7b70bd217ad8da4932c8ea2c2767014
#1 DONE 19.0s

$ docker buildx create --config=/tmp/buildkitd.toml --append --bootstrap --name=buildx-test-d7b70bd217ad8da4932c8ea2c2767014 --node= --platform=linux/arm64 --driver=kubernetes --driver-opt=image=moby/buildkit:master,qemu.install=true
#1 [internal] booting buildkit
#1 waiting for 1 pods to be ready
#1 waiting for 1 pods to be ready 2.9s done
#1 DONE 2.9s
buildx-test-d7b70bd217ad8da4932c8ea2c2767014

$ docker buildx inspect --bootstrap --builder=buildx-test-d7b70bd217ad8da4932c8ea2c2767014
Name:          buildx-test-d7b70bd217ad8da4932c8ea2c2767014
Driver:        kubernetes
Last Activity: 2023-03-11 20:25:23 +0000 UTC

Nodes:
Name:           buildx-test-d7b70bd217ad8da4932c8ea2c27670140
Endpoint:       kubernetes:///buildx-test-d7b70bd217ad8da4932c8ea2c2767014?deployment=buildkit-70875667-c592-4a36-b038-58f5f0f3f82e-q42kc&kubeconfig=%2Ftmp%2Fbuildkit-k3s%2Fkubeconfig.yaml
Driver Options: image="moby/buildkit:master" qemu.install="true"
Status:         running
Buildkit:       252e5a6
Platforms:      linux/amd64*, linux/amd64/v2, linux/amd64/v3, linux/amd64/v4, linux/arm64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6

Name:           buildx-test-d7b70bd217ad8da4932c8ea2c27670141
Endpoint:       kubernetes:///buildx-test-d7b70bd217ad8da4932c8ea2c2767014?deployment=buildkit-c28d4390-03a8-4c72-b097-3585adea17b8-hx24n&kubeconfig=%2Ftmp%2Fbuildkit-k3s%2Fkubeconfig.yaml
Driver Options: image="moby/buildkit:master" qemu.install="true"
Status:         running
Buildkit:       252e5a6
Platforms:      linux/arm64*, linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/amd64/v4, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6

@dvdksn Might need docs changes at https://docs.docker.com/build/drivers/kubernetes/#native to remove the need of setting --node (maybe we could just explain setting the node name is optional).

@crazy-max crazy-max force-pushed the fix-k8s-deploy-name branch 2 times, most recently from 0cdfdfa to d8d09d6 Compare March 11, 2023 19:32
@crazy-max crazy-max force-pushed the fix-k8s-deploy-name branch from d8d09d6 to 527d575 Compare March 11, 2023 20:22
@crazy-max crazy-max marked this pull request as ready for review March 13, 2023 09:37
@tonistiigi tonistiigi merged commit 4a73abf into docker:master Mar 14, 2023
@crazy-max crazy-max deleted the fix-k8s-deploy-name branch March 14, 2023 18:52
@dvdksn dvdksn mentioned this pull request Apr 7, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@AkihiroSuda AkihiroSuda AkihiroSuda approved these changes

@morlay morlay Awaiting requested review from morlay

@jedevc jedevc Awaiting requested review from jedevc

@tonistiigi tonistiigi Awaiting requested review from tonistiigi

Assignees

No one assigned

Labels

area/driver/kubernetes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@crazy-max @AkihiroSuda @tonistiigi @dvdksn