We're having an issue where a Dockerfile that used to build with overlay FS fails to build in overlay2. After some investigation I've cut it down to a very simple Dockerfile that reproduces the issue:
FROM node:4.4.7
RUN npm install -g npm
RUN npm install -g jshint
RUN npm install -g mocha
Building this with overlay works as expected, however building it with overlay2 fails on the final step with:
/bin/sh: 1: npm: not found
Looking at the individual layers reveals that this is because the /usr/local/lib/node_modules/npm directory has disappeared during the second RUN command. In our real Dockerfile, there are several npm packages installed in that directory which all disappear. The only package left is the new one in that command (jshint in the above example)
Note that running a container from node:4.4.7 and running those commands manually works perfectly fine, with both overlay and overlay2.
Output of docker version:
Client:
Version: 1.12.0
API version: 1.24
Go version: go1.6.3
Git commit: 8eab29e
Built: Thu Jul 28 22:00:36 2016
OS/Arch: linux/amd64
Server:
Version: 1.12.0
API version: 1.24
Go version: go1.6.3
Git commit: 8eab29e
Built: Thu Jul 28 22:00:36 2016
OS/Arch: linux/amd64
Output of docker info:
Containers: 1
Running: 0
Paused: 0
Stopped: 1
Images: 3
Server Version: 1.12.0
Storage Driver: overlay2
Backing Filesystem: extfs
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: null overlay host bridge
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: apparmor
Kernel Version: 4.4.0-31-generic
Operating System: Ubuntu 14.04.4 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 15.35 GiB
Name: lp02488
ID: 6KEQ:SM2A:GBFZ:NIW4:72MA:LEN5:QR6J:QMSU:Y4M6:6DKA:ITNK:EE75
Docker Root Dir: /space/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Insecure Registries:
127.0.0.0/8
Additional environment details (AWS, VirtualBox, physical, etc.):
The node:4.4.7 image uses debian 8.5, however I've tried similar steps with an ubuntu-based container with npm installed and have had the same problem.
Steps to reproduce the issue:
- Run
docker build --no-cache -t test . using the above Dockerfile
- If successful, run
docker run --rm -ti test npm list -g --depth 0 to list the installed packages
Describe the results you received:
Using overlay:
The Dockerfile builds as expected. The npm command lists all three installed packages.
Using overlay2:
The Dockerfile fails at the last RUN command with:
/bin/sh: 1: npm: not found
Investigation shows that the npm global install folder has been replaced with just the most recent package (which has therefore removed the globally-installed npm)
Describe the results you expected:
The Dockerfile should build properly on both overlay and overlay2
Additional information you deem important (e.g. issue happens only occasionally):
This seems to occur regardless of what the second two packages are, however it only goes wrong if the command npm install -g npm is first, in which case it goes wrong in the next command that plays with /usr/local/lib/node_modules , e.g.:
This fails on RUN npm install -g mocha:
FROM node:4.4.7
RUN npm install -g npm
RUN npm install -g jshint
RUN npm install -g mocha
RUN npm install -g supertest
RUN npm install -g chai
This fails on RUN npm install -g jshint:
FROM node:4.4.7
RUN npm install -g npm
RUN touch /usr/local/lib/node_modules/test
RUN npm install -g jshint
RUN npm install -g mocha
RUN npm install -g supertest
RUN npm install -g chai
This fails on RUN npm install -g mocha again:
FROM node:4.4.7
RUN npm install -g npm
RUN touch /test.txt
RUN npm install -g jshint
RUN npm install -g mocha
RUN npm install -g supertest
RUN npm install -g chai
However this completes successfully:
FROM node:4.4.7
RUN npm install -g jshint
RUN npm install -g npm
RUN npm install -g mocha
RUN npm install -g supertest
RUN npm install -g chai
As does this (though obviously the npm list doesn't work because of the touched file):
FROM node:4.4.7
RUN touch /usr/local/lib/node_modules/test
RUN npm install -g npm
RUN npm install -g jshint
RUN npm install -g mocha
RUN npm install -g supertest
RUN npm install -g chai
But this fails on RUN npm install -g mocha again:
FROM node:4.4.7
RUN touch /usr/local/lib/node_modules/test && rm -f /usr/local/lib/node_modules/test
RUN npm install -g npm
RUN npm install -g jshint
RUN npm install -g mocha
RUN npm install -g supertest
RUN npm install -g chai
All the above were built with docker build --no-cache -t test .. When I say "fails on", I mean that that step reported npm as not found (so it had disappeared by then). They all work perfectly fine (as in, they don't accidentally delete npm) in overlay.
We're having an issue where a Dockerfile that used to build with overlay FS fails to build in overlay2. After some investigation I've cut it down to a very simple Dockerfile that reproduces the issue:
Building this with overlay works as expected, however building it with overlay2 fails on the final step with:
/bin/sh: 1: npm: not foundLooking at the individual layers reveals that this is because the /usr/local/lib/node_modules/npm directory has disappeared during the second RUN command. In our real Dockerfile, there are several npm packages installed in that directory which all disappear. The only package left is the new one in that command (jshint in the above example)
Note that running a container from node:4.4.7 and running those commands manually works perfectly fine, with both overlay and overlay2.
Output of
docker version:Output of
docker info:Additional environment details (AWS, VirtualBox, physical, etc.):
The node:4.4.7 image uses debian 8.5, however I've tried similar steps with an ubuntu-based container with npm installed and have had the same problem.
Steps to reproduce the issue:
docker build --no-cache -t test .using the above Dockerfiledocker run --rm -ti test npm list -g --depth 0to list the installed packagesDescribe the results you received:
Using overlay:
The Dockerfile builds as expected. The npm command lists all three installed packages.
Using overlay2:
The Dockerfile fails at the last RUN command with:
/bin/sh: 1: npm: not foundInvestigation shows that the npm global install folder has been replaced with just the most recent package (which has therefore removed the globally-installed npm)
Describe the results you expected:
The Dockerfile should build properly on both overlay and overlay2
Additional information you deem important (e.g. issue happens only occasionally):
This seems to occur regardless of what the second two packages are, however it only goes wrong if the command
npm install -g npmis first, in which case it goes wrong in the next command that plays with /usr/local/lib/node_modules , e.g.:This fails on
RUN npm install -g mocha:This fails on
RUN npm install -g jshint:This fails on
RUN npm install -g mochaagain:However this completes successfully:
As does this (though obviously the npm list doesn't work because of the touched file):
But this fails on
RUN npm install -g mochaagain:All the above were built with
docker build --no-cache -t test .. When I say "fails on", I mean that that step reported npm as not found (so it had disappeared by then). They all work perfectly fine (as in, they don't accidentally delete npm) in overlay.