Skip to content

Disappearing npm packages when building with overlay2 #25244

@adaphi

Description

@adaphi

We're having an issue where a Dockerfile that used to build with overlay FS fails to build in overlay2. After some investigation I've cut it down to a very simple Dockerfile that reproduces the issue:

FROM node:4.4.7

RUN npm install -g npm

RUN npm install -g jshint

RUN npm install -g mocha

Building this with overlay works as expected, however building it with overlay2 fails on the final step with:
/bin/sh: 1: npm: not found
Looking at the individual layers reveals that this is because the /usr/local/lib/node_modules/npm directory has disappeared during the second RUN command. In our real Dockerfile, there are several npm packages installed in that directory which all disappear. The only package left is the new one in that command (jshint in the above example)
Note that running a container from node:4.4.7 and running those commands manually works perfectly fine, with both overlay and overlay2.

Output of docker version:

Client:
 Version:      1.12.0
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   8eab29e
 Built:        Thu Jul 28 22:00:36 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.12.0
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   8eab29e
 Built:        Thu Jul 28 22:00:36 2016
 OS/Arch:      linux/amd64

Output of docker info:

Containers: 1
 Running: 0
 Paused: 0
 Stopped: 1
Images: 3
Server Version: 1.12.0
Storage Driver: overlay2
 Backing Filesystem: extfs
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: null overlay host bridge
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: apparmor
Kernel Version: 4.4.0-31-generic
Operating System: Ubuntu 14.04.4 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 15.35 GiB
Name: lp02488
ID: 6KEQ:SM2A:GBFZ:NIW4:72MA:LEN5:QR6J:QMSU:Y4M6:6DKA:ITNK:EE75
Docker Root Dir: /space/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Insecure Registries:
 127.0.0.0/8

Additional environment details (AWS, VirtualBox, physical, etc.):
The node:4.4.7 image uses debian 8.5, however I've tried similar steps with an ubuntu-based container with npm installed and have had the same problem.

Steps to reproduce the issue:

  1. Run docker build --no-cache -t test . using the above Dockerfile
  2. If successful, run docker run --rm -ti test npm list -g --depth 0 to list the installed packages

Describe the results you received:
Using overlay:
The Dockerfile builds as expected. The npm command lists all three installed packages.
Using overlay2:
The Dockerfile fails at the last RUN command with:
/bin/sh: 1: npm: not found
Investigation shows that the npm global install folder has been replaced with just the most recent package (which has therefore removed the globally-installed npm)

Describe the results you expected:
The Dockerfile should build properly on both overlay and overlay2

Additional information you deem important (e.g. issue happens only occasionally):
This seems to occur regardless of what the second two packages are, however it only goes wrong if the command npm install -g npm is first, in which case it goes wrong in the next command that plays with /usr/local/lib/node_modules , e.g.:

This fails on RUN npm install -g mocha:

FROM node:4.4.7

RUN npm install -g npm

RUN npm install -g jshint

RUN npm install -g mocha

RUN npm install -g supertest

RUN npm install -g chai

This fails on RUN npm install -g jshint:

FROM node:4.4.7

RUN npm install -g npm

RUN touch /usr/local/lib/node_modules/test

RUN npm install -g jshint

RUN npm install -g mocha

RUN npm install -g supertest

RUN npm install -g chai

This fails on RUN npm install -g mocha again:

FROM node:4.4.7

RUN npm install -g npm

RUN touch /test.txt

RUN npm install -g jshint

RUN npm install -g mocha

RUN npm install -g supertest

RUN npm install -g chai

However this completes successfully:

FROM node:4.4.7

RUN npm install -g jshint

RUN npm install -g npm

RUN npm install -g mocha

RUN npm install -g supertest

RUN npm install -g chai

As does this (though obviously the npm list doesn't work because of the touched file):

FROM node:4.4.7

RUN touch /usr/local/lib/node_modules/test

RUN npm install -g npm

RUN npm install -g jshint

RUN npm install -g mocha

RUN npm install -g supertest

RUN npm install -g chai

But this fails on RUN npm install -g mocha again:

FROM node:4.4.7

RUN touch /usr/local/lib/node_modules/test && rm -f /usr/local/lib/node_modules/test

RUN npm install -g npm

RUN npm install -g jshint

RUN npm install -g mocha

RUN npm install -g supertest

RUN npm install -g chai

All the above were built with docker build --no-cache -t test .. When I say "fails on", I mean that that step reported npm as not found (so it had disappeared by then). They all work perfectly fine (as in, they don't accidentally delete npm) in overlay.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions