I've got a bake build that builds around 20 targets concurrently.
All of the targets share the same context, a Go project of around 100Mb.
When I run buildx bake, it's visible that it sends the same context to the buildkit, for each target.
I thought it was the logs that made me believe it was so but that underneath it was all fine and sent only once.
So I created a "minimal" test project with 40 targets building the same
Go sources. I added a 100Mb useless file to the context.
docker buildx bake -f slow.hcl --load takes multiple minutes to complete and we can see that the context is being sent 40 times.
I tested another option where I add a base target to the bakefile, copy the whole context into that image and made the Dockerfile copy from this target instead of directly from .
Now, the build is way less verbose and finished in 19s with:
BASE="target:base" docker buildx bake -f slow.hcl --load
By the way, this workaround requires changing all my Dockerfiles. Every COPY x y becomes COPY --from=base x y. It would be much simpler if we could redefine the "." context and set it to "target:base" using build-contexts flags.
Another thing interesting is how the build takes 19s and if you sum up the build logs, you've got 4.7s.
[+] Building 18.5s (173/173) FINISHED
=> [base internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 59B 0.0s
=> [base internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg26 internal] load build context 0.0s
=> => transferring context: 7.50kB 0.0s
=> CACHED [pkg26 1/1] COPY . . 0.1s
=> [pkg26] exporting to image 0.7s
=> => exporting layers 0.0s
=> => writing image sha256:b028daff45f597ec1bf4a6a090391c5dce0fff854def9057b4ee005ee79ff8ea 0.0s
=> => writing image sha256:82f0fd8aa9e650107ba5a6909752e26afe17d183af92d54ec28396ad44af70e5 0.0s
=> [pkg04 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 360B 0.0s
=> [pkg12 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg32 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg30 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg25 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg05 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg18 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg14 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg15 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg02 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg10 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg38 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg36 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg19 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg23 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg39 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg03 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg28 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg08 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg37 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg21 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg35 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg20 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg06 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg00 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg13 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg33 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg27 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg29 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg24 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg09 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg22 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg16 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg01 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg07 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg11 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg31 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg34 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg17 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg26 internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [pkg04 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg12 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg32 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg30 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg25 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg05 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg18 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg14 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg15 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg02 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg10 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg38 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg36 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg19 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg23 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg39 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg03 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg28 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg08 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg37 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg21 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg35 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg20 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg06 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg00 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg13 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg33 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg27 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg29 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg24 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg09 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg22 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg16 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg01 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg07 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg11 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg31 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg34 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg17 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg26 internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [pkg26] resolve image config for docker.io/docker/dockerfile@sha256:9ba7531bd80fb0a858632727cf7a112fbfd19b17e94 0.0s
=> CACHED [pkg26] docker-image://docker.io/docker/dockerfile@sha256:9ba7531bd80fb0a858632727cf7a112fbfd19b17e94c4e 0.0s
=> [pkg04 internal] load .dockerignore 0.0s
=> [pkg04 internal] load build definition from Dockerfile 0.0s
=> [pkg26 internal] load metadata for docker.io/library/golang:1.19.2-alpine3.16@sha256:e4dcdac3ed37d8c2b3b8bcef29 0.0s
=> [pkg26 build 1/4] FROM docker.io/library/golang:1.19.2-alpine3.16@sha256:e4dcdac3ed37d8c2b3b8bcef2909573b2ad9c2 0.0s
=> CACHED [pkg26 build 2/4] WORKDIR /src 0.0s
=> CACHED [pkg26 build 3/4] COPY --from=base . ./ 0.9s
=> CACHED [pkg26 build 4/4] RUN go build -o main ./pkg1/ 3.0s
=> [pkg12 internal] load build definition from Dockerfile 0.0s
=> [pkg12 internal] load .dockerignore 0.0s
=> [pkg32 internal] load build definition from Dockerfile 0.0s
=> [pkg32 internal] load .dockerignore 0.0s
=> [pkg30 internal] load build definition from Dockerfile 0.0s
=> [pkg30 internal] load .dockerignore 0.0s
=> [pkg25 internal] load .dockerignore 0.0s
=> [pkg25 internal] load build definition from Dockerfile 0.0s
=> [pkg05 internal] load build definition from Dockerfile 0.0s
=> [pkg05 internal] load .dockerignore 0.0s
=> [pkg18 internal] load .dockerignore 0.0s
=> [pkg18 internal] load build definition from Dockerfile 0.0s
=> [pkg14 internal] load build definition from Dockerfile 0.0s
=> [pkg14 internal] load .dockerignore 0.0s
=> [pkg15 internal] load build definition from Dockerfile 0.0s
=> [pkg15 internal] load .dockerignore 0.0s
=> [pkg02 internal] load build definition from Dockerfile 0.0s
=> [pkg02 internal] load .dockerignore 0.0s
=> [pkg10 internal] load .dockerignore 0.0s
=> [pkg10 internal] load build definition from Dockerfile 0.0s
=> [pkg38 internal] load build definition from Dockerfile 0.0s
=> [pkg38 internal] load .dockerignore 0.0s
=> [pkg36 internal] load .dockerignore 0.0s
=> [pkg36 internal] load build definition from Dockerfile 0.0s
=> [pkg19 internal] load .dockerignore 0.0s
=> [pkg19 internal] load build definition from Dockerfile 0.0s
=> [pkg23 internal] load build definition from Dockerfile 0.0s
=> [pkg23 internal] load .dockerignore 0.0s
=> [pkg39 internal] load .dockerignore 0.0s
=> [pkg39 internal] load build definition from Dockerfile 0.0s
=> [pkg03 internal] load .dockerignore 0.0s
=> [pkg03 internal] load build definition from Dockerfile 0.0s
=> [pkg28 internal] load build definition from Dockerfile 0.0s
=> [pkg28 internal] load .dockerignore 0.0s
=> [pkg08 internal] load build definition from Dockerfile 0.0s
=> [pkg08 internal] load .dockerignore 0.0s
=> [pkg37 internal] load .dockerignore 0.0s
=> [pkg37 internal] load build definition from Dockerfile 0.0s
=> [pkg21 internal] load build definition from Dockerfile 0.0s
=> [pkg21 internal] load .dockerignore 0.0s
=> [pkg35 internal] load build definition from Dockerfile 0.0s
=> [pkg35 internal] load .dockerignore 0.0s
=> [pkg20 internal] load build definition from Dockerfile 0.0s
=> [pkg20 internal] load .dockerignore 0.0s
=> [pkg06 internal] load build definition from Dockerfile 0.0s
=> [pkg06 internal] load .dockerignore 0.0s
=> [pkg00 internal] load build definition from Dockerfile 0.0s
=> [pkg00 internal] load .dockerignore 0.0s
=> [pkg13 internal] load build definition from Dockerfile 0.0s
=> [pkg13 internal] load .dockerignore 0.0s
=> CACHED [pkg26 stage-1 1/1] COPY --from=build /src/main /main 0.0s
=> [pkg33 internal] load .dockerignore 0.0s
=> [pkg33 internal] load build definition from Dockerfile 0.0s
=> [pkg29 internal] load build definition from Dockerfile 0.0s
=> [pkg29 internal] load .dockerignore 0.0s
=> [pkg27 internal] load build definition from Dockerfile 0.0s
=> [pkg27 internal] load .dockerignore 0.0s
=> [pkg24 internal] load .dockerignore 0.0s
=> [pkg24 internal] load build definition from Dockerfile 0.0s
=> [pkg09 internal] load .dockerignore 0.0s
=> [pkg09 internal] load build definition from Dockerfile 0.0s
=> [pkg22 internal] load .dockerignore 0.0s
=> [pkg22 internal] load build definition from Dockerfile 0.0s
=> [pkg16 internal] load .dockerignore 0.0s
=> [pkg16 internal] load build definition from Dockerfile 0.0s
=> [pkg01 internal] load .dockerignore 0.0s
=> [pkg01 internal] load build definition from Dockerfile 0.0s
=> [pkg07 internal] load .dockerignore 0.0s
=> [pkg07 internal] load build definition from Dockerfile 0.0s
=> [pkg11 internal] load build definition from Dockerfile 0.0s
=> [pkg11 internal] load .dockerignore 0.0s
=> [pkg31 internal] load .dockerignore 0.0s
=> [pkg31 internal] load build definition from Dockerfile 0.0s
=> [pkg34 internal] load build definition from Dockerfile 0.0s
=> [pkg34 internal] load .dockerignore 0.0s
=> [pkg17 internal] load build definition from Dockerfile 0.0s
=> [pkg17 internal] load .dockerignore 0.0s
=> [pkg26 internal] load .dockerignore 0.0s
=> [pkg26 internal] load build definition from Dockerfile 0.0s
Versions used:
❯ docker version
Client:
Cloud integration: v1.0.29
Version: 20.10.20
API version: 1.41
Go version: go1.18.7
Git commit: 9fdeb9c
Built: Tue Oct 18 18:20:35 2022
OS/Arch: darwin/arm64
Context: default
Experimental: true
Server: Docker Desktop 4.13.0 (89412)
Engine:
Version: 20.10.20
API version: 1.41 (minimum version 1.12)
Go version: go1.18.7
Git commit: 03df974
Built: Tue Oct 18 18:18:16 2022
OS/Arch: linux/arm64
Experimental: false
containerd:
Version: 1.6.8
GitCommit: 9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
runc:
Version: 1.1.4
GitCommit: v1.1.4-0-g5fd4c4d
docker-init:
Version: 0.19.0
GitCommit: de40ad0
❯ docker buildx version
github.com/docker/buildx v0.9.1 ed00243a0ce2a0aee75311b06e32d33b44729689
❯ docker buildx ls
NAME/NODE DRIVER/ENDPOINT STATUS BUILDKIT PLATFORMS
default * docker
default default running 20.10.20 linux/arm64, linux/amd64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6
desktop-linux docker
desktop-linux desktop-linux running 20.10.20 linux/arm64, linux/amd64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/arm/v7, linux/arm/v6
I've got a bake build that builds around 20 targets concurrently.
All of the targets share the same context, a Go project of around 100Mb.
When I run buildx bake, it's visible that it sends the same context to the buildkit, for each target.
I thought it was the logs that made me believe it was so but that underneath it was all fine and sent only once.
So I created a "minimal" test project with 40 targets building the same
Go sources. I added a 100Mb useless file to the context.
docker buildx bake -f slow.hcl --loadtakes multiple minutes to complete and we can see that the context is being sent 40 times.I tested another option where I add a
basetarget to the bakefile, copy the whole context into that image and made the Dockerfile copy from this target instead of directly from .Now, the build is way less verbose and finished in 19s with:
BASE="target:base" docker buildx bake -f slow.hcl --loadBy the way, this workaround requires changing all my Dockerfiles. Every
COPY x ybecomesCOPY --from=base x y. It would be much simpler if we could redefine the "." context and set it to "target:base" using build-contexts flags.Another thing interesting is how the build takes 19s and if you sum up the build logs, you've got 4.7s.
Versions used: