Skip to content

[0.13] remote driver with kube-pod URL is stuck at inactive #2324

@chulkilee

Description

@chulkilee

Contributing guidelines

I've found a bug and checked that ...

  • ... the documentation does not mention anything about my problem
  • ... there are no open or closed issues that are related to my problem

Description

docker-buildx 0.13.0 could not detect the buildkit 0.13.0 instance on kubernetes via kube-pod:// URL. It works well if I kubectl port-forward the service into local and use tcp://127.0.0.1:1234.

Expected behaviour

docker buildx 0.13.0 supports remote driver with kube-pod as in 0.12.1

Actual behaviour

Remote builder is stuck at inactive when using kube-pod:// URL

Buildx version

github.com/docker/buildx v0.13.0 Homebrew

Docker info

Client: Docker Engine - Community
 Version:    25.0.3
 Context:    colima
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.13.0
    Path:     /.../.docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  2.24.7
    Path:     /.../.docker/cli-plugins/docker-compose

Server:
 Containers: 9
  Running: 4
  Paused: 0
  Stopped: 5
 Images: 64
 Server Version: 24.0.7
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f
 runc version: v1.1.10-0-g18a0cb0
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.5.0-17-generic
 Operating System: Ubuntu 23.10
 OSType: linux
 Architecture: aarch64
 CPUs: 6
 Total Memory: 7.736GiB
 Name: colima
 ID: 0296b23e-9bd3-4597-918d-0c64e567f14e
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Builders list

NAME/NODE          DRIVER/ENDPOINT                                                                                            STATUS     BUILDKIT               PLATFORMS
kube-chulki        kubernetes
 \_ kube-chulki0    \_ kubernetes:///kube-chulki?deployment=buildkit-4217ad63-0d28-4fdd-8dcf-9bf48ea1bf71-chtgj&kubeconfig=   running    v0.12.5                linux/arm64, linux/arm/v7, linux/arm/v6
 \_ kube-chulki0    \_ kubernetes:///kube-chulki?deployment=buildkit-4217ad63-0d28-4fdd-8dcf-9bf48ea1bf71-chtgj&kubeconfig=   running    v0.12.5                linux/arm64, linux/arm/v7, linux/arm/v6
 \_ kube-chulki0    \_ kubernetes:///kube-chulki?deployment=buildkit-4217ad63-0d28-4fdd-8dcf-9bf48ea1bf71-chtgj&kubeconfig=   running    v0.12.5                linux/arm64, linux/arm/v7, linux/arm/v6
 \_ kube-chulki0    \_ kubernetes:///kube-chulki?deployment=buildkit-4217ad63-0d28-4fdd-8dcf-9bf48ea1bf71-chtgj&kubeconfig=   running    v0.12.5                linux/arm64, linux/arm/v7, linux/arm/v6
remote*            remote
 \_ remote0         \_ kube-pod://buildkitd-amd64-0?namespace=buildkit&context=............                                   inactive
remote-tcp         remote
 \_ remote-tcp0     \_ tcp://127.0.0.1:1234                                                                                   running    v0.13.0                linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/amd64/v4, linux/386
colima             docker
 \_ colima          \_ colima                                                                                                 running    v0.11.7+d3e6c1360f6e   linux/arm64, linux/amd64, linux/amd64/v2
default                                                                                                                       error

Cannot load builder default: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

Configuration

N/A

Build logs

No response

Additional info

Found different behavior with dial-stdio which is apparently used to connect to builder

$ docker-buildx dial-stdio --builder remote-tcp --progress plain
#0 builder: remote-tcp

#1 Proxying to builder
#1 Dialing builder
#1 Dialing builder 1.8s done

$ docker-buildx dial-stdio --builder remote --progress plain
#0 builder: remote

#1 Proxying to builder
#1 Dialing builder
#1 ...

#2 [internal] waiting for connection
#2 ERROR: context deadline exceeded

#1 Proxying to builder
#1 Dialing builder 20.0s done
#1 ERROR: context deadline exceeded
------
 > Proxying to builder:
------
------
 > [internal] waiting for connection:
------
ERROR: context deadline exceeded

I'm not sure it's a buildkit issue or docker-buildx issue though.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions