Skip to content

Replace 'commit --run' with 'commit --change'#5105

Closed
shykes wants to merge 7 commits intomoby:masterfrom
shykes:commit-v2
Closed

Replace 'commit --run' with 'commit --change'#5105
shykes wants to merge 7 commits intomoby:masterfrom
shykes:commit-v2

Conversation

@shykes
Copy link
Copy Markdown
Contributor

@shykes shykes commented Apr 9, 2014

docker commit --change allows specifying standard changes to be applied to the new image. It deprecates the less user-friendly and less stable commit --run.

Changes are expressed in the Dockerfile syntax, and can be used to modify the image metadata. For example:

docker commit --change 'WORKDIR /home/bob' will commit the image with /home/bob as the default working directory.

docker commit --change 'EXPOSE 80 53/udp' will commit the image with network ports 80/tcp and 53/udp marked as exposed.

docker commit --change 'VOLUME /var/data' will commit the image with /var/data marked as a volume.

And so on.

The following Dockerfile instructions can be used as a change:

  • ENTRYPOINT
  • CMD
  • USER
  • WORKDIR
  • ENV
  • VOLUME
  • EXPOSE
  • ONBUILD

The syntax for these commands is identical in --change and in a
Dockerfile. See the Dockerfile reference for a complete syntax reference.

Note that some fields which could be changed directly with the deprecated --run can no longer be changed with the new --change. This is by design, for 2 reasons:

  1. Not every field in the underlying Config object should be automatically exposed for modification. This gives us a stable indirection to control the exact set of allowed modifications.

  2. If the Dockerfile syntax does not allow a legitimate modification, we can simply enrich the syntax, fixing the problem in a unified way both in Dockerfile and docker commit.

@shykes
Copy link
Copy Markdown
Contributor Author

shykes commented Apr 9, 2014

This depends on #5102

@tianon
Copy link
Copy Markdown
Member

tianon commented Apr 9, 2014

Legit! 👍 Seems like a nice clean solution to the problem IMO, and it encourages the right things.

I wonder if maybe in the future we could consider a --context flag to allow "ADD" here too? Maybe even just parse the "ADD" line and infer the context from the ADD argument? I can see why ADD isn't included in this PR for sure (too many hard questions around ADD that could bite later). :)

@creack
Copy link
Copy Markdown
Contributor

creack commented Apr 9, 2014

code lgtm

@shykes
Copy link
Copy Markdown
Contributor Author

shykes commented Apr 9, 2014

@tianon yes this is a small first step towards a unified "instruction set" across all docker commands which involve modifying a container or image.

Solomon Hykes added 7 commits April 9, 2014 16:54
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <[email protected]> (github: shykes)
This allows using the Dockerfile syntax as a common "scripting" layer to express changes to an image configuration.

Docker-DCO-1.1-Signed-off-by: Solomon Hykes <[email protected]> (github: shykes)
…string

Docker-DCO-1.1-Signed-off-by: Solomon Hykes <[email protected]> (github: shykes)
Docker-DCO-1.1-Signed-off-by: Solomon Hykes <[email protected]> (github: shykes)
…ns (and print a warning)

This makes pkg/dockerfile usable as the reference parser for 'docker build'.

Docker-DCO-1.1-Signed-off-by: Solomon Hykes <[email protected]> (github: shykes)
… in the Dockerfile syntax

This allows easy conversion of a Config object to and from the Dockerfile syntax.

Docker-DCO-1.1-Signed-off-by: Solomon Hykes <[email protected]> (github: shykes)
```docker commit --change``` allows specifying standard changes to be applied to the
new image. It deprecates the less user-friendly and less stable *commit --run*.

Changes are expressed in the Dockerfile syntax, and can be used  to modify the image
metadata. For example:

```docker commit --change 'WORKDIR /home/bob'``` will commit the image
with */home/bob* as the default working directory.

```docker commit --change 'EXPOSE 80 53/udp'``` will commit the image
with network ports *80/tcp* and *53/udp* marked as exposed.

```docker commit --change 'VOLUME /var/data'``` will commit the image
with */var/data* marked as a volume.

The following Dockerfile instructions can be used as a change:

* ENTRYPOINT
* CMD
* USER
* WORKDIR
* ENV
* VOLUME
* EXPOSE
* ONBUILD

The syntax for these commands is identical in *--change* and in a
Dockerfile. See the Dockerfile reference for a complete syntax reference.

Note that some fields which could be changed directly with the
deprecated ```--run``` can no longer be changed with the new
```--change```. This is by design, for 2 reasons:

1) Not every field in the underlying Config object should be
automatically exposed for modification. This gives us a stable
indirection to control the exact set of allowed modifications.

2) If the Dockerfile syntax does not allow a legitimate modification,
we can simply enrich the syntax, fixing the problem in a unified way
both in Dockerfile and *docker commit*.

Docker-DCO-1.1-Signed-off-by: Solomon Hykes <[email protected]> (github: shykes)
@shykes
Copy link
Copy Markdown
Contributor Author

shykes commented Apr 9, 2014

Rebased. Ping @creack @crosbymichael @vieux @unclejack

@creack
Copy link
Copy Markdown
Contributor

creack commented Apr 10, 2014

@shykes This disable almost all output for docker build and breaks the cache of the docker image at some point (no output, don't know what step).
On master: make is 100% cached, on this branch, it has only ~10 layer cached.

@creack
Copy link
Copy Markdown
Contributor

creack commented Apr 10, 2014

master: 100% cached, "old" output.
this branch:

$> make test                                                                                                                                                   docker build -t "docker:shykes-commit-v2" .
Uploading context 128.1 MB
Uploading context
# Skipping unknown instruction DOCKER-VERSION
 ---> Using cache
 ---> Using cache
 ---> Using cache
 ---> Using cache
 ---> Using cache
 ---> Using cache
 ---> Using cache
 ---> Using cache
 ---> Using cache
 ---> Using cache
 ---> Using cache
 ---> Running in 3a3c86a95bf1
 ---> Running in 5705ee4068e2
+ for platform in '$DOCKER_CROSSPLATFORMS'
+ GOOS=linux
+ GOARCH=386
+ ./make.bash --no-clean
# Building C bootstrap tool.
cmd/dist

# Building compilers and Go bootstrap tool for host, linux/amd64.
lib9
libbio
libmach
misc/pprof
cmd/addr2line
cmd/nm
cmd/objdump
cmd/pack
cmd/prof
cmd/cc
cmd/gc
cmd/6l
cmd/8l
^Cmake: *** [build] Interrupt
fish: Job 1, “make test” terminated by signal SIGINT (Quit request from job control (^C))

@mkscrg
Copy link
Copy Markdown

mkscrg commented May 20, 2014

The current state of the docs is not great for users that want these capabilities. Info about --run was removed six weeks ago with 168f8ab. Since then there's no "advertised" way to update a container on commit, AFAICT.

Is this considered a minor feature? I've heard of a few build pipelines that require it (including my own), mostly applications with dependencies managed by something like Bundler / Maven / etc.

@pda
Copy link
Copy Markdown
Contributor

pda commented May 24, 2014

I've heard of a few build pipelines that require it (including my own)

Yes — we use a pipeline which injects code and runs build steps using something like:

docker run --volume=…:/src image/name bash -c "cp -a /src /dst && cd /dst && make"
docker commit …

Currently the run/commit would clobber the original CMD specified in the Dockerfile, so we have to docker commit --run=… to put it back.

In #4885 I proposed docker commit merge config from the image not the container, so that the run step doesn't destroy the config established by the Dockerfile. The comments suggested my use case wasn't understood. But I'm sure others are running into this issue.

@progrium
Copy link
Copy Markdown
Contributor

progrium commented Jun 2, 2014

Here is a data point on how something like this would want to be used: https://github.com/progrium/docker-releasetag

@progrium
Copy link
Copy Markdown
Contributor

progrium commented Jun 2, 2014

Not an uncommon pattern: https://github.com/deis/deis/blob/deis-build/controller/registry/docker.py#L7-L55

Though ideally it would be with images not a container, which is why I proposed it for docker tag.

@erikh
Copy link
Copy Markdown
Contributor

erikh commented Aug 1, 2014

@shykes can we revisit this from scratch after I get the new Dockerfile handler working? It seems like we're in serious conflict here. :)

@rhatdan
Copy link
Copy Markdown
Contributor

rhatdan commented Sep 17, 2014

Since I want this functionality for docker import. I have patches for COMMENT, and ENV, not to mention a separate new command META, @erikh Is this something you have looked at since August?

@erikh
Copy link
Copy Markdown
Contributor

erikh commented Sep 18, 2014

Hi Dan,

Just a little. I have been very busy adding new features to Docker and the parser took a while to shake most of the bugs out.

What I’d like to do is defer to solomon (Who is on vacation, so I’m trying to avoid pinging him) on these issues -- especially dockerfile commands -- because they require us to support them indefinitely.

If these features absolutely cannot wait, I will discuss them with the team and hopefully we can work something out.

Work for you?

-Erik

@unclejack
Copy link
Copy Markdown
Contributor

@erikh @rhatdan We've discussed this on IRC, but didn't comment on this issue. The conclusion was that we need to change --run to --change and add --change to import.

The plan would be to:

  1. get this PR to master
  2. add --change to import

What do you think?

@rhatdan
Copy link
Copy Markdown
Contributor

rhatdan commented Sep 18, 2014

Sounds good to me. I am going to have one of my people work on converting my patches to use --change

@crosbymichael
Copy link
Copy Markdown
Contributor

Please reopen when you have time to finish this PR and have it rebased.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.