Proposal: Allow specification of LABEL Name/Value pairs in image json content by rhatdan · Pull Request #8955 · moby/moby

rhatdan · 2014-11-04T20:28:48Z

Save "UserData" field in Dockerfile into image content.

This will allow a user to save user data into an image, which
can later be retrieved using:

docker inspect IMAGEID

I have copied this from the "Comment" handling in docker images.

We want to be able to add Name/Value data to an image to describe the image,
and then be able to use other tools to look at this data, to be able to do
security checks based on this data.

We are thinking about adding version names,
Perhaps listing the content of the dockerfile.
Descriptions of where the code came from etc.

This USERDATA field should also be allowed to be specified in the
docker import --change USERDATA:Name=Value
docker commit --change USERDATA:Name=Value

Docker-DCO-1.1-Signed-off-by: Dan Walsh <[email protected]> (github: rhatdan)

rhatdan · 2014-11-04T20:29:11Z

This pull request replaces the Meta pull

#7470

rhatdan · 2014-11-04T20:29:52Z

To give you an idea of what kind of data we would see being stored in META data fields. Look at what rpm ships
rpm -qi httpd

Name : httpd
Version : 2.4.10
Release : 2.fc22
Architecture: x86_64
Install Date: Mon 04 Aug 2014 09:45:02 AM EDT
Group : System Environment/Daemons
Size : 3967053
License : ASL 2.0
Signature : (none)
Source RPM : httpd-2.4.10-2.fc22.src.rpm
Build Date : Thu 31 Jul 2014 07:25:12 AM EDT
Build Host : buildvm-03.phx2.fedoraproject.org
Relocations : (not relocatable)
Packager : Fedora Project
Vendor : Fedora Project
URL : http://httpd.apache.org/
Summary : Apache HTTP Server
Description : The Apache HTTP Server is a powerful, efficient, and extensible web server.

erikh · 2014-11-04T20:45:40Z

Adding more JSON to the dockerfile seems like a mistake, as we're looking to deprecate that.

Perhaps you'd like the syntax in #8251 instead? I would greatly prefer using that.

rhatdan · 2014-11-04T21:14:11Z

This is not about putting data into the container, it is about adding data to help Vendors describe their images.

Think of the images as being applications. Vendors want to describe their applications based on things like licenses, versions, Short and long description, build dates, build servers ...

This data should be included in the json data related to the image, and should be flexible, since it is impossible to describe all potential fields a vendor would like to have.

erikh · 2014-11-04T21:38:52Z

We don’t want JSON in the Dockerfile. It’s already polluted and complicated a number of issues as is, and we’re looking to deprecate the existing syntax. Look strongly at the ENV pull request mentioned in my previous comment, and see if the parser it uses can be leveraged for your purposes.

On Nov 4, 2014, at 1:14 PM, rhatdan [email protected] wrote:

This is not about putting data into the container, it is about adding data to help Vendors describe their images.

Think of the images as being applications. Vendors want to describe their applications based on things like licenses, versions, Short and long description, build dates, build servers ...

This data should be included in the json data related to the image, and should be flexible, since it is impossible to describe all potential fields a vendor would like to have.

—
Reply to this email directly or view it on GitHub #8955 (comment).

jessfraz · 2014-11-04T21:39:48Z

I think it's also more readable from a user standpoint in the env variable syntax, also +1 for no more json

sghosh151 · 2014-11-05T02:29:06Z

json allows for embedded key value pairs whereas the ENV pull referenced is limiting to a flat key value set.

rhatdan · 2014-11-05T11:26:10Z

@jfrazelle @erikh So you would like the syntax to be

USERDATA Description="This is a tool for reading docker images" Version=1.0 License=GPL

erikh · 2014-11-05T12:42:34Z

Yes.

On Nov 5, 2014, at 3:26 AM, rhatdan [email protected] wrote:

@jfrazelle https://github.com/jfrazelle @erikh https://github.com/erikh So you would like the syntax to be

USERDATA Description="This is a tool for reading docker images" Version=1.0 License=GPL

—
Reply to this email directly or view it on GitHub #8955 (comment).

rhatdan · 2014-11-05T13:10:50Z

Changed the proposal to use Name/Value Pairs rather then JSON.

jessfraz · 2014-11-05T18:28:59Z

thanks! :)

fsamin · 2014-11-05T19:44:15Z

Hi there,

From my point of view, starting points about userdata are:

For example, Name/Value pairs in the npm's package format (package.json)
The common information described on the docker registry

Here are some of the datas that I would like to keep in the Dockerfile :

short description (as set in the settings/description of the docker registry)
description (as a .md file ?)
keywords
repository information (type=git, url=https://github.com/xxx/xxx.git)
license

By the way, there also some specific datas that may be usefull to formalize : the recommanded way to use start the container :

as a daemon or not ? (-d option of the docker run command)
ports to map (-p option of the docker run command)
volumes to map (-v option of the docker run command)
environment variables to use (-e option of the docker run)

All of these could be used in the registry to improve and standardize the images information. External tools could also instrumentalize the way to start containers (maybe like github.com/yesnault/docktor try to do).

erikh · 2014-11-05T19:48:07Z

I do like the idea of setting “preferences” but I think this is out of scope for this PR. Please feel free to submit a proposal if you have strong feelings about where this should go in the future.

On Nov 5, 2014, at 11:44 AM, François Samin [email protected] wrote:

Hi there,

From my point of view, starting points about userdata are:

For example, Name/Value pairs in the npm's package format (package.json)
The common information described on the docker registry
Here are some of the datas that I would like to keep in the Dockerfile :

short description (as set in the settings/description of the docker registry)
description (as a .md file ?)
keywords
repository information (type=git, url=https://github.com/xxx/xxx.git https://github.com/xxx/xxx.git)
license
By the way, there also some specific datas that may be usefull to formalize : the recommanded way to use start the container :

as a daemon or not ? (-d option of the docker run command)
ports to map (-p option of the docker run command)
volumes to map (-v option of the docker run command)
environment variables to use (-e option of the docker run)
All of these could be used in the registry to improve and standardize the images information. External tools could also instrumentalize the way to start containers (maybe like github.com/yesnault/docktor try to do).

—
Reply to this email directly or view it on GitHub #8955 (comment).

thaJeztah · 2014-11-05T22:16:43Z

USERDATA Description="This is a tool for reading docker images" Version=1.0 License=GPL

Are newlines supported with this notation? Because this becomes very un-readable if a lot of key/value pairs have to be set. To illustrate, this would be the line for the properties in the PR description;

USERDATA Name=httpd Version=2.4.10 Release=2.fc22 Architecture=x86_64 Install Date="Mon 04 Aug 2014 09:45:02 AM EDT" Group="System Environment/Daemons" Size="3967053" License="ASL 2.0" Signature="(none)" Source RPM="httpd-2.4.10-2.fc22.src.rpm" Build Date="Thu 31 Jul 2014 07:25:12 AM EDT" Build Host=buildvm-03.phx2.fedoraproject.org Relocations="(not relocatable)" Packager="Fedora Project" Vendor="Fedora Project" URL="http://httpd.apache.org/" Summary="Apache HTTP Server" Description="The Apache HTTP Server is a powerful, efficient, and extensible web server."

Additionally, with this notation;

How should USERDATA keys containing spaces be set? For example Source RPM - should the key be quotes as well? ("Source RPM"="value")
How to explicitly set values to NULL / empty? (mykey=???)

Not debating the env-style notation, just trying to get the practical implications, because I foresee that the USERDATA can contain quite lot of data.

rhatdan · 2014-11-06T00:37:08Z

I expect to be able to support this syntax.
FROM fedora:20
USERDATA Description="This image is used to start the foo executable" Products" Version="1.0"
USERDATA Vendor="Red Hat" Buildhost=builder.redhat.com
USERDATA "RPM Version"=rpm-4.12.0.1-4.fc22.x86_64
CMD ["/bin/sh"]

The name value pairs will get wrapped in Quotes (") if they do not contain them and then converted internally to json format. A null string will end up being "".

erikh · 2014-11-06T00:38:25Z

This sounds great to me.

On Nov 5, 2014, at 4:37 PM, rhatdan [email protected] wrote:

I expect to be able to support this syntax.
FROM fedora:20
USERDATA Description="This image is used to start the foo executable" Products" Version="1.0"
USERDATA Vendor="Red Hat" Buildhost=builder.redhat.com
USERDATA "RPM Version"=rpm-4.12.0.1-4.fc22.x86_64
CMD ["/bin/sh"]

The name value pairs will get wrapped in Quotes (") if they do not contain them and then converted internally to json format. A null string will end up being "".

—
Reply to this email directly or view it on GitHub #8955 (comment).

tiborvass · 2014-11-06T00:40:27Z

@rhatdan this line:
USERDATA Description="This image is used to start the foo executable" Products" Version="1.0" has a quoting problem around Products, is it intentional?

tiborvass · 2014-11-06T00:56:11Z

Multiline could be supported by:

USERDATA Description="This image is\
used to start the foo executable"

I also expect USERDATA Version 1.0 to work.

Unfortunately this proposal touches quoting issues (like #8251). Should we first accept quoting for USERDATA and ENV, and then eventually bring it to other instructions? Or should we bring quoting as a whole to the Dockerfile, and therefore quoting issues can be left out from this proposal?

erikh · 2014-11-06T01:00:12Z

I would like to restrict it to userdata and env for now, seeing as there are optional backwards-compat methods of dealing with this in both places.

On Nov 5, 2014, at 4:56 PM, Tibor Vass [email protected] wrote:

Multiline could be supported by:

USERDATA Description="This image is
used to start the foo executable"
I also expect USERDATA Version 1.0 to work.

Unfortunately this proposal touches quoting issues (like #8251 #8251). Should we first accept quoting for USERDATA and ENV, and then eventually bring it to other instructions? Or should we bring quoting as a whole to the Dockerfile, and therefore quoting issues can be left out from this proposal?

—
Reply to this email directly or view it on GitHub #8955 (comment).

thaJeztah · 2014-11-06T06:52:47Z

I expect to be able to support this syntax

The problem I see there, is that we're probable running out of layers very quickly. So, if possible, being able to define multiple key/value pairs per USERDATA instruction would resolve that.

Multiline could be supported by:

that example is to use new lines inside a value; would it be possible somehow to (eg) define one key/value pair per line (without creating new layers)?

rhatdan · 2014-11-06T10:06:54Z

@tiborvass

USERDATA Description="This image is used to start the foo executable" Products" Version="1.0" has a quoting problem around Products, is it intentional?
This is a typo.

rhatdan · 2014-11-06T10:15:12Z

I think it should support whatever we add to ENV except for the space separation. I think that should be deprecated. I think it confuses the usability rather then enhances it.

I would like to see repeated
ENV lines

Or USERDATA lines combined into a single layer. Maybe only have layers for commands that add lots of content like ADD and RUN.

tiborvass · 2014-11-06T15:36:44Z

@jlhawn is the layer problem being worked on by someone, or do you think we should start squashing things like ENV and this proposed USERDATA? I feel like it's a separate issue that should be worked on, and is out of scope for this proposal.

ibuildthecloud · 2015-01-16T01:22:54Z

docs/sources/reference/api/docker_remote_api.md

Can you move this to the v1.17 section

ibuildthecloud · 2015-01-16T15:51:09Z

runconfig/config.go

I feel Labels would be more consistent with other fields such as ExposedPorts and Volumes. I know env is not plural, but I think thats just because "envs" is weird.

ibuildthecloud · 2015-01-21T15:47:08Z

docs/sources/reference/api/docker_remote_api_v1.17.md

Typo, should be { ... } not [ ... ]

Save "LABEL" field in Dockerfile into image content. This will allow a user to save user data into an image, which can later be retrieved using: docker inspect IMAGEID I have copied this from the "Comment" handling in docker images. We want to be able to add Name/Value data to an image to describe the image, and then be able to use other tools to look at this data, to be able to do security checks based on this data. We are thinking about adding version names, Perhaps listing the content of the dockerfile. Descriptions of where the code came from etc. This LABEL field should also be allowed to be specified in the docker import --change LABEL:Name=Value docker commit --change LABEL:Name=Value Docker-DCO-1.1-Signed-off-by: Dan Walsh <[email protected]> (github: rhatdan)

icecrime · 2015-02-28T02:12:59Z

@rhatdan I'm trying to push #9882 forward: are you ok with me closing that one?

rhatdan · 2015-03-02T17:51:23Z

@icecrime

rhatdan · 2015-03-02T17:51:35Z

@icecrime yes.

jamtur01 added the /project/doc label Nov 5, 2014

rhatdan force-pushed the userdata branch from 35e10ba to 08d9152 Compare November 5, 2014 13:06

rhatdan changed the title ~~Allow the specification of UserData Json content to be saved in an image~~ Allow specification of UserData Name/Value pairs in image json content Nov 5, 2014

rhatdan force-pushed the userdata branch from 08d9152 to dd0eb49 Compare November 5, 2014 13:09

rhatdan force-pushed the userdata branch from dd0eb49 to e60a943 Compare November 5, 2014 13:55

tonistiigi mentioned this pull request Nov 5, 2014

Proposal: Docker Plugins #8968

Closed

erikh mentioned this pull request Nov 6, 2014

Add support for ENV of the form: ENV name=value ... #8251

Merged

rhatdan force-pushed the userdata branch from cdea82f to 3d30295 Compare January 15, 2015 12:53

ibuildthecloud reviewed Jan 16, 2015
View reviewed changes

docs/sources/reference/api/docker_remote_api.md Outdated

Copy link

Contributor

ibuildthecloud Jan 16, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you move this to the v1.17 section

rhatdan force-pushed the userdata branch 2 times, most recently from ccbb533 to 8f0b391 Compare January 16, 2015 13:45

ibuildthecloud reviewed Jan 16, 2015
View reviewed changes

rhatdan force-pushed the userdata branch 5 times, most recently from 1211af0 to 698693c Compare January 21, 2015 13:13

ibuildthecloud reviewed Jan 21, 2015
View reviewed changes

docs/sources/reference/api/docker_remote_api_v1.17.md Outdated

Copy link

Contributor

ibuildthecloud Jan 21, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo, should be { ... } not [ ... ]

rhatdan force-pushed the userdata branch from 698693c to 201d9ae Compare January 21, 2015 20:22

rhatdan changed the title ~~Proposal: Allow specification of UserData Name/Value pairs in image json content~~ Proposal: Allow specification of LABEL Name/Value pairs in image json content Jan 21, 2015

rhatdan force-pushed the userdata branch 2 times, most recently from f625f59 to 0fa2834 Compare February 5, 2015 11:34

jessfraz added the status/1-design-review label Feb 10, 2015

rhatdan force-pushed the userdata branch from 0fa2834 to c22c608 Compare February 10, 2015 08:33

philips mentioned this pull request Feb 13, 2015

Proposal: App Container (appc) as a Runtime Option in the Docker Engine #10777

Closed

rhatdan force-pushed the userdata branch from c22c608 to a695e32 Compare February 17, 2015 15:22

jessfraz mentioned this pull request Feb 28, 2015

Mark container as needing to run with --privileged #4852

Closed

icecrime closed this Mar 2, 2015

dmp42 removed the status/1-design-review label Mar 2, 2015

ibuildthecloud mentioned this pull request Sep 8, 2015

Revert "Container don't inherit from image labels" #15503

Merged

Conversation

rhatdan commented Nov 4, 2014

Uh oh!

rhatdan commented Nov 4, 2014

Uh oh!

rhatdan commented Nov 4, 2014

Uh oh!

erikh commented Nov 4, 2014

Uh oh!

rhatdan commented Nov 4, 2014

Uh oh!

erikh commented Nov 4, 2014

Uh oh!

jessfraz commented Nov 4, 2014

Uh oh!

sghosh151 commented Nov 5, 2014

Uh oh!

rhatdan commented Nov 5, 2014

Uh oh!

erikh commented Nov 5, 2014

Uh oh!

rhatdan commented Nov 5, 2014

Uh oh!

jessfraz commented Nov 5, 2014

Uh oh!

fsamin commented Nov 5, 2014

Uh oh!

erikh commented Nov 5, 2014

Uh oh!

thaJeztah commented Nov 5, 2014

Uh oh!

rhatdan commented Nov 6, 2014

Uh oh!

erikh commented Nov 6, 2014

Uh oh!

tiborvass commented Nov 6, 2014

Uh oh!

tiborvass commented Nov 6, 2014

Uh oh!

erikh commented Nov 6, 2014

Uh oh!

thaJeztah commented Nov 6, 2014

Uh oh!

rhatdan commented Nov 6, 2014

Uh oh!

rhatdan commented Nov 6, 2014

Uh oh!

tiborvass commented Nov 6, 2014

Uh oh!

ibuildthecloud Jan 16, 2015

Choose a reason for hiding this comment

Uh oh!

ibuildthecloud Jan 16, 2015

Choose a reason for hiding this comment

Uh oh!

ibuildthecloud Jan 21, 2015

Choose a reason for hiding this comment

Uh oh!

icecrime commented Feb 28, 2015

Uh oh!

rhatdan commented Mar 2, 2015

Uh oh!

rhatdan commented Mar 2, 2015

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

15 participants