--cache-from and Multi Stage: Pre-Stages are not cached #34715
Description
Description
If you want to use a Multi Stage Build together with --cache-from it's very hard and complicated to load the Cache of Pre Stages, as --cache-from disables the lookup in the local cache (see #32612). The only way is to tag the pre-stage images as well and add them to the --cache-from, which is very complicated.
Steps to reproduce the issue:
- Assuming we have a Multistage Dockerfile like:
FROM busybox as builder
RUN echo "hello" > test
FROM busybox
COPY --from=builder test test
RUN echo test
- Building it the first time:
$ docker build -t test:latest .
Sending build context to Docker daemon 2.048kB
Step 1/5 : FROM busybox as builder
---> d20ae45477cb
Step 2/5 : RUN echo "hello" > test
---> Running in b5e871ebd251
---> 6889762613a0
Removing intermediate container b5e871ebd251
Step 3/5 : FROM busybox
---> d20ae45477cb
Step 4/5 : COPY --from=builder test test
---> f9ee9cc534a7
Removing intermediate container 8d76fd7eb6be
Step 5/5 : RUN echo test
---> Running in 5b768ed39212
test
---> b4a81a0e7c96
Removing intermediate container 5b768ed39212
Successfully built b4a81a0e7c96
Successfully tagged test:latest
So far all good.
3. Now running it a second time, see how all layers are fully cached:
$ docker build -t test:latest .
Sending build context to Docker daemon 2.048kB
Step 1/5 : FROM busybox as builder
---> d20ae45477cb
Step 2/5 : RUN echo "hello" > test
---> Using cache
---> 6889762613a0
Step 3/5 : FROM busybox
---> d20ae45477cb
Step 4/5 : COPY --from=builder test test
---> Using cache
---> f9ee9cc534a7
Step 5/5 : RUN echo test
---> Using cache
---> b4a81a0e7c96
Successfully built b4a81a0e7c96
Successfully tagged test:latest
- Now running it with
--cache-from test:latest:
$ docker build -t test:latest --cache-from test:latest .
Sending build context to Docker daemon 2.048kB
Step 1/5 : FROM busybox as builder
---> d20ae45477cb
Step 2/5 : RUN echo "hello" > test
---> Running in 89d43713b017
---> 18e01d7690cb
Removing intermediate container 89d43713b017
Step 3/5 : FROM busybox
---> d20ae45477cb
Step 4/5 : COPY --from=builder test test
---> Using cache
---> f9ee9cc534a7
Step 5/5 : RUN echo test
---> Using cache
---> b4a81a0e7c96
Successfully built b4a81a0e7c96
Successfully tagged test:latest
See how Step 2/5 : RUN echo "hello" > test is not using any cache.
Interestingly Step 4 is using the cache again, as it finds that cache within the test:latest image.
So it actually builds the first stage image but never uses it. A lot of time the first stages are very heavy computations, like installing packages, building stuff etc. So we almost loose the niceness of Multi Stage Build.
There is a way to fix this, with tagging the first stage image via --target builder:
$ docker build -t test-builder:latest --target builder .
Sending build context to Docker daemon 2.048kB
Step 1/5 : FROM busybox as builder
---> d20ae45477cb
Step 2/5 : RUN echo "hello" > test
---> Using cache
---> 18e01d7690cb
Successfully built 18e01d7690cb
Successfully tagged test-builder:latest
and then using both images for --cache-from:
$ docker build -t test:latest --cache-from test:latest --cache-from test-builder:latest .
Sending build context to Docker daemon 2.048kB
Step 1/5 : FROM busybox as builder
---> d20ae45477cb
Step 2/5 : RUN echo "hello" > test
---> Using cache
---> 18e01d7690cb
Step 3/5 : FROM busybox
---> d20ae45477cb
Step 4/5 : COPY --from=builder test test
---> Using cache
---> f9ee9cc534a7
Step 5/5 : RUN echo test
---> Using cache
---> b4a81a0e7c96
Successfully built b4a81a0e7c96
Successfully tagged test:latest
but IMHO that is super complicated and confusing.
I'm not 100% sure how we could fix this. Implementing --cache-from to also use the local cache as a secondary cache lookup would solve the problem (see #32612)
Output of docker version:
$ docker version
Client:
Version: 17.06.1-ce
API version: 1.30
Go version: go1.8.3
Git commit: 874a737
Built: Thu Aug 17 22:53:38 2017
OS/Arch: darwin/amd64
Server:
Version: 17.06.1-ce
API version: 1.30 (minimum version 1.12)
Go version: go1.8.3
Git commit: 874a737
Built: Thu Aug 17 22:54:55 2017
OS/Arch: linux/amd64
Experimental: true
Output of docker info:
$ docker info
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 1059
Server Version: 17.06.1-ce
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 6e23458c129b551d5c9871e5174f6b1b7f6d1170
runc version: 810190ceaa507aa2727d7ae6f4790c76ec150bd2
init version: 949e6fa
Security Options:
seccomp
Profile: default
Kernel Version: 4.9.41-moby
Operating System: Alpine Linux v3.5
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 1.952GiB
Name: moby
ID: FHCJ:CF22:VRF6:Y4HR:BM3W:ATJ3:3QGW:AGO5:OTKL:W2ES:OM6Q:WZ5Y
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: 18
Goroutines: 31
System Time: 2017-09-03T19:59:13.529192672Z
EventsListeners: 1
No Proxy: *.local, 169.254/16
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false