Description
We have a very large generated Dockerfile (15k+ lines), from which we build specific targets.
since docker-engine v27 the builds are failing with
=> [internal] load build definition from Dockerfile
=> => transferring dockerfile: 896.44kB
ERROR: failed to receive status: rpc error: code = ResourceExhausted desc = grpc: received message larger than max (44865299 vs. 16777216)
with version 26.1.4 it works fine.
I might be wrong here but I think it might be caused by this and introduced with this commit
Will it be possible to increase the MaxCallRecvMsgSize/MaxCallSendMsgSize to math.MaxInt32 ?
Reproduce
Use this simple python script to generate large DockerFile
stages = 300
lines_per_stage = 50
with open("Dockerfile", "w") as f:
for i in range(stages):
f.write(f"FROM ubuntu:20.04 as stage{i}\n")
for j in range(lines_per_stage):
f.write(f"ENV VAR{i}_{j}=\"This is a large environment variable to increase the size of the Dockerfile.\"\n")
f.write(f"RUN echo \"This is a large echo statement to increase the size of the Dockerfile.\"\n")
f.write(f"LABEL key{i}_{j}=\"This is a large label to increase the size of the Dockerfile.\"\n")
f.write("\n")
f.write("FROM ubuntu:20.04 as final\n")
for i in range(stages):
f.write(f"COPY --from=stage{i} /tmp/stage{i} /tmp/stage{i}\n")
f.write("RUN echo \"Final stage\"\n")
f.write("LABEL final_label=\"Final stage label\"\n")
generate the file and try to build a stage
python generate.py
sudo docker build --target stage37 .
[+] Building 1.1s (1/1) FINISHED
=> [internal] load build definition from Dockerfile
=> => transferring dockerfile: 4.47MB
ERROR: failed to receive status: rpc error: code = ResourceExhausted desc = grpc: received message larger than max (35785741 vs. 16777216)
Expected behavior
Try the same with docker v26.1.4 and the build passes.
docker version
Client:
Version: 27.0.3
API version: 1.46
Go version: go1.21.11
Git commit: 7d4bcd8
Built: Fri Jun 28 23:59:41 2024
OS/Arch: darwin/arm64
Context: desktop-linux
Server: Docker Desktop 4.32.0 (157355)
Engine:
Version: 27.0.3
API version: 1.46 (minimum version 1.24)
Go version: go1.21.11
Git commit: 662f78c
Built: Sat Jun 29 00:02:44 2024
OS/Arch: linux/arm64
Experimental: false
containerd:
Version: 1.7.18
GitCommit: ae71819c4f5e67bb4d5ae76a6b735f29cc25774e
runc:
Version: 1.7.18
GitCommit: v1.1.13-0-g58aa920
docker-init:
Version: 0.19.0
GitCommit: de40ad0
docker info
Client:
Version: 27.0.3
Context: desktop-linux
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.15.1-desktop.1
Path: /Users/user/.docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.28.1-desktop.1
Path: /Users/alonz/.docker/cli-plugins/docker-compose
debug: Get a shell into any image or container (Docker Inc.)
Version: 0.0.32
Path: /Users/alonz/.docker/cli-plugins/docker-debug
desktop: Docker Desktop commands (Alpha) (Docker Inc.)
Version: v0.0.14
Path: /Users/alonz/.docker/cli-plugins/docker-desktop
dev: Docker Dev Environments (Docker Inc.)
Version: v0.1.2
Path: /Users/alonz/.docker/cli-plugins/docker-dev
extension: Manages Docker extensions (Docker Inc.)
Version: v0.2.25
Path: /Users/alonz/.docker/cli-plugins/docker-extension
feedback: Provide feedback, right in your terminal! (Docker Inc.)
Version: v1.0.5
Path: /Users/alonz/.docker/cli-plugins/docker-feedback
init: Creates Docker-related starter files for your project (Docker Inc.)
Version: v1.3.0
Path: /Users/alonz/.docker/cli-plugins/docker-init
sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
Version: 0.6.0
Path: /Users/alonz/.docker/cli-plugins/docker-sbom
scout: Docker Scout (Docker Inc.)
Version: v1.10.0
Path: /Users/alonz/.docker/cli-plugins/docker-scout
Server:
Containers: 4
Running: 1
Paused: 0
Stopped: 3
Images: 18
Server Version: 27.0.3
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: ae71819c4f5e67bb4d5ae76a6b735f29cc25774e
runc version: v1.1.13-0-g58aa920
init version: de40ad0
Security Options:
seccomp
Profile: unconfined
cgroupns
Kernel Version: 6.6.32-linuxkit
Operating System: Docker Desktop
OSType: linux
Architecture: aarch64
CPUs: 11
Total Memory: 7.754GiB
Name: docker-desktop
ID: 535b8cb9-8d2e-41e4-ab95-1e61d5480f46
Docker Root Dir: /var/lib/docker
Debug Mode: false
HTTP Proxy: http.docker.internal:3128
HTTPS Proxy: http.docker.internal:3128
No Proxy: hubproxy.docker.internal
Labels:
com.docker.desktop.address=unix:///Users/alonz/Library/Containers/com.docker.docker/Data/docker-cli.sock
Experimental: false
Insecure Registries:
hubproxy.docker.internal:5555
127.0.0.0/8
Live Restore Enabled: falseAdditional Info
No response
Description
We have a very large generated Dockerfile (15k+ lines), from which we build specific targets.
since docker-engine v27 the builds are failing with
with version 26.1.4 it works fine.
I might be wrong here but I think it might be caused by this and introduced with this commit
Will it be possible to increase the
MaxCallRecvMsgSize/MaxCallSendMsgSizetomath.MaxInt32?Reproduce
Use this simple python script to generate large DockerFile
generate the file and try to build a stage
Expected behavior
Try the same with docker v26.1.4 and the build passes.
docker version
Client: Version: 27.0.3 API version: 1.46 Go version: go1.21.11 Git commit: 7d4bcd8 Built: Fri Jun 28 23:59:41 2024 OS/Arch: darwin/arm64 Context: desktop-linux Server: Docker Desktop 4.32.0 (157355) Engine: Version: 27.0.3 API version: 1.46 (minimum version 1.24) Go version: go1.21.11 Git commit: 662f78c Built: Sat Jun 29 00:02:44 2024 OS/Arch: linux/arm64 Experimental: false containerd: Version: 1.7.18 GitCommit: ae71819c4f5e67bb4d5ae76a6b735f29cc25774e runc: Version: 1.7.18 GitCommit: v1.1.13-0-g58aa920 docker-init: Version: 0.19.0 GitCommit: de40ad0docker info
Additional Info
No response