This is a placeholder issue for discussing improvements to the way docker volumes keeps track of (named) volumes created through plugins.

This issue came up in a discussion on IRC (logs can be found here on botbot.me)

I'm writing down some thoughts; there may be technical inaccuracies (please comment and I'll correct them 😄)

Description

Docker volume can keep track of volumes in a number of ways;

• Volumes created with the 'local' driver are stored inside /var/lib/docker, and can be resolved through the filesystem
• Volumes created through the volumes API (both 'local' and 'plugin') are kept in-memory
• Volumes that are currently used by containers can be resolved through the information stored in the container's JSON

Apart from the above, Docker does not persist any information about volumes. This means that, after a daemon restart, information about unused volumes, created through plugins is no longer available. The volumes are still "there", but Docker does not have any notion of their existence.

Problem

Not keeping track of this information can lead to a number of problems. Some examples;

docker volume create --name foobar
docker volume create --driver somedriver --name foobar2
docker volume create --driver somedriver --name foobar3

docker run -d -v foobar3:/foo nginx

docker volume

DRIVER              VOLUME NAME
local               foobar
somedriver          foobar2
somedriver          foobar3

restart docker


docker volume

DRIVER              VOLUME NAME
local               foobar
somedriver          foobar3

After restarting the daemon, volume foobar2 is no longer visible. (note: foobar3 actually becomes visible as a "local" volume, which is caused by #15994)

Trying to use foobar2 for a container (but omitting the --volume-driver),
will now create a new volume, using the default ('local') driver;

docker run -d -v foobar2:/foo nginx

DRIVER              VOLUME NAME
local               foobar
local               foobar2
somedriver          foobar3

Possible solutions

1. Persist information about all volumes in /var/lib/docker/volume/<volume-name>.

For volumes created through plug-ins, this could simply mean to keep a JSON file, containing the minimum required information, probably only "driver" (possibly "Source") is needed at this moment;

{
    "Driver": "somedriver",
    "Source": "/some/path/foobar3/_data",
}

advantages:

• Simple solution
• Consistent with the 'local' driver
• Guarantees "globally" unique volume names (/var/lib/docker/volume/<volume-name>)

disadvantages:

• Who is "responsible" for keeping track of volumes; the driver or docker?
• Plugin's may have other ways to create / destroy volumes; information can get out of sync.

2. Extend the volume-plugins interface

Extend the volume-plugins interface to have plugins return a list of volumes they have present.

advantages:

• The plugin is responsibility for providing and maintaining the volume information
• Information doesn't get out of sync (if a plugin creates / destroys a volume outside of Docker)

disadvantages:

• More complicated / moving parts
• No guarantee that volume-names are "globally" unique

3. Combination of the above

Store information about volumes in /var/lib/docker, and synchronize this information
by requesting the actual list of volumes from the plugins.