Description
When using docker-container driver and containerd in an ephemeral CICD environment (CodeBuild) in DinD, I receive the following error. Note, this always occurs, and it only occurs when using containerd. When not using containerd, the build succeeds.
I first attempted this with buildkit 0.17.1 and docker 27.2.1 and thought that upgrading may resolve the issue, but no luck.
Also noteworthy - this configuration works locally on Darwin.
Name: builder_0.19.0_aarch64_from_root
Driver: docker-container
Last Activity: 2025-02-12 01:24:15 +0000 UTC
Nodes:
Name: builder_0.19.0_aarch64_from_root0
Endpoint: unix:///var/run/docker.sock
Driver Options: env.BUILDKIT_STEP_LOG_MAX_SIZE="-1" env.BUILDKIT_STEP_LOG_MAX_SPEED="-1" image="moby/buildkit:v0.19.0"
Status: inactive
BuildKit daemon flags: --debug=true --oci-worker-gc=false --containerd-worker-gc=false --allow-insecure-entitlement=network.host
#0 building with "builder_0.19.0_aarch64_from_root" instance using docker-container driver
#1 [internal] booting buildkit
CICDPipelineEDStackDev: success: Published 6fd0dac7b04a5290b448cbc77202bf1113ca7568fecbdea449c4f7fc6201027b:<redacted>>-<redacted>
#1 pulling image moby/buildkit:v0.19.0
#1 pulling image moby/buildkit:v0.19.0 0.3s done
#1 creating container buildx_buildkit_builder_0.19.0_aarch64_from_root0 done
#1 ERROR: Error response from daemon: No such image: moby/buildkit:v0.19.0
------
> [internal] booting buildkit:
------
ERROR: Error response from daemon: No such image: moby/buildkit:v0.19.0
CICDPipelineEDStackDev: fail: buildx.sh build --build-arg UMBRELLA_IMAGE_TAG=<redacted>>.dkr.ecr.<redacted>.amazonaws.com/<redacted> --tag cdkasset-<redacted> --file Dockerfile.codebuild --platform linux/arm64 . exited with error code 1: #0 building with "builder_0.19.0_aarch64_from_root" instance using docker-container driver
#1 [internal] booting buildkit
#1 pulling image moby/buildkit:v0.19.0
#1 pulling image moby/buildkit:v0.19.0 0.3s done
#1 creating container buildx_buildkit_builder_0.19.0_aarch64_from_root0 done
#1 ERROR: Error response from daemon: No such image: moby/buildkit:v0.19.0
------
> [internal] booting buildkit:
------
ERROR: Error response from daemon: No such image: moby/buildkit:v0.19.0
Similiar to #49236
Reproduce
docker buildx create \
--use \
--name="${DOCKER_BUILDER_NAME}" \
--driver="docker-container" \
--driver-opt="image=moby/buildkit:v${DOCKER_BUILDKIT_VERSION}" \
--driver-opt="env.BUILDKIT_STEP_LOG_MAX_SIZE=-1" `# Remove truncation of logs` \
--driver-opt="env.BUILDKIT_STEP_LOG_MAX_SPEED=-1" `# Remove truncation of logs` \
--buildkitd-flags="--debug=true --oci-worker-gc=false --containerd-worker-gc=false" `# Don't enable GC` \
--bootstrap # Launches builder right after creation for faster builds
docker buildx build \
--output type=docker,compression=zstd,compression-level=22 \
--builder "${DOCKER_BUILDER_NAME}" \
"${@:2}"
Expected behavior
Build should succeed.
docker version
Client:
Version: 27.5.1
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.20.1
Path: /root/.docker/cli-plugins/docker-buildx
Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 27.5.1
Storage Driver: overlay2
driver-type: io.containerd.snapshotter.v1
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
runc version: v1.2.4-0-g6c52b3f
init version: de40ad0
Security Options:
seccomp
Profile: builtin
Kernel Version: 4.14.355-271.569.amzn2.aarch64
Operating System: Amazon Linux 2023.6.20241111 (containerized)
OSType: linux
Architecture: aarch64
CPUs: 8
Total Memory: 15.44GiB
Name: ip-10-0-67-16.ec2.internal
ID: 56cef27a-231c-4f72-a9c1-9b50be891693
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: true
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Product License: Community Enginedocker info
Client:
Version: 27.5.1
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.20.1
Path: /root/.docker/cli-plugins/docker-buildx
Server:
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 27.5.1
Storage Driver: overlay2
driver-type: io.containerd.snapshotter.v1
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 1
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
runc version: v1.2.4-0-g6c52b3f
init version: de40ad0
Security Options:
seccomp
Profile: builtin
Kernel Version: 4.14.355-271.569.amzn2.aarch64
Operating System: Amazon Linux 2023.6.20241111 (containerized)
OSType: linux
Architecture: aarch64
CPUs: 8
Total Memory: 15.44GiB
Name: ip-10-0-67-16.ec2.internal
ID: 56cef27a-231c-4f72-a9c1-9b50be891693
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: true
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Product License: Community EngineAdditional Info
No response
Description
When using docker-container driver and
containerdin an ephemeral CICD environment (CodeBuild) in DinD, I receive the following error. Note, this always occurs, and it only occurs when usingcontainerd. When not usingcontainerd, the build succeeds.I first attempted this with buildkit 0.17.1 and docker 27.2.1 and thought that upgrading may resolve the issue, but no luck.
Also noteworthy - this configuration works locally on Darwin.
Similiar to #49236
Reproduce
Expected behavior
Build should succeed.
docker version
docker info
Additional Info
No response