Skip to content

Update image export to support Docker format#3172

Merged
fuweid merged 1 commit intocontainerd:masterfrom
dmcgowan:export-docker-compatibility
May 17, 2019
Merged

Update image export to support Docker format#3172
fuweid merged 1 commit intocontainerd:masterfrom
dmcgowan:export-docker-compatibility

Conversation

@dmcgowan
Copy link
Copy Markdown
Member

@dmcgowan dmcgowan commented Apr 3, 2019

Add manifest.json file which is used by Docker to import images.

This change adds the manifest.json file by default as the size of it is small. Containerd will still always prefer OCI layout when available on the newest version of Docker will do the same. Added a flag to disable adding this files.

Additionally updated the exporter to exclude adding manifests and their subobjects when --all-platforms is not provided. This more closely matches the behavior of pull, so pull/fetch + export will work, and pull/fetch --all-platforms + export --all-platforms will work.

@dmcgowan dmcgowan added this to the 1.3-beta.0 milestone Apr 3, 2019
@dmcgowan dmcgowan force-pushed the export-docker-compatibility branch from 63da395 to 28c3ff8 Compare April 3, 2019 22:45
Comment thread images/archive/exporter.go Outdated
Comment thread images/archive/exporter.go Outdated
Comment thread images/archive/exporter.go Outdated
Comment thread images/archive/exporter.go Outdated
@AkihiroSuda
Copy link
Copy Markdown
Member

This seems causing two issues when the image name lacks tag

  1. the image can't be loaded to Docker
$ sudo buildctl b -o type=image,name=foobar --frontend dockerfile.v0 --local dockerfile=. --local context=. 
$ sudo ctr --namespace=buildkit i export - foobar | docker load
bc6050364fad: Loading layer [==================================================>]  33.68MB/33.68MB
1f45365f628e: Loading layer [==================================================>]     169B/169B
e4007c134428: Loading layer [==================================================>]  7.768MB/7.768MB
d1b98eeb2284: Loading layer [==================================================>]     186B/186B
715db8ecd1a2: Loading layer [==================================================>]     188B/188B
invalid tag "foobar"
  1. The value of org.opencontainers.image.ref.name is set to the image name without tag (foobar), but other OCI implementations are likely to treat this value as the tag https://github.com/opencontainers/image-spec/blob/master/image-layout.md#indexjson-file

A workaround would be to append :latest when the tag part is not present.

@dmcgowan
Copy link
Copy Markdown
Member Author

dmcgowan commented Apr 4, 2019

The intent is to normalize the names that go into the manifest.json file, so if there is no tag then I agree the default should be added. Normally tags such as foobar are not used within containerd but we need to work around different workflows so we can leave image name format as undefined as possible.

@dmcgowan dmcgowan force-pushed the export-docker-compatibility branch 2 times, most recently from cb12b2d to 9e18e5c Compare April 4, 2019 17:37
Add manifest.json file which is used by Docker
to import images.

Signed-off-by: Derek McGowan <[email protected]>
@dmcgowan dmcgowan force-pushed the export-docker-compatibility branch from 9e18e5c to 4754d2a Compare April 4, 2019 22:23
@codecov-io
Copy link
Copy Markdown

Codecov Report

Merging #3172 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #3172   +/-   ##
=======================================
  Coverage   45.22%   45.22%           
=======================================
  Files         111      111           
  Lines       11975    11975           
=======================================
  Hits         5416     5416           
  Misses       5725     5725           
  Partials      834      834
Flag Coverage Δ
#linux 49.26% <ø> (ø) ⬆️
#windows 40.58% <ø> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a7e30fa...4754d2a. Read the comment docs.

1 similar comment
@codecov-io
Copy link
Copy Markdown

Codecov Report

Merging #3172 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master    #3172   +/-   ##
=======================================
  Coverage   45.22%   45.22%           
=======================================
  Files         111      111           
  Lines       11975    11975           
=======================================
  Hits         5416     5416           
  Misses       5725     5725           
  Partials      834      834
Flag Coverage Δ
#linux 49.26% <ø> (ø) ⬆️
#windows 40.58% <ø> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a7e30fa...4754d2a. Read the comment docs.

Copy link
Copy Markdown
Member

@AkihiroSuda AkihiroSuda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but it would be better if we can manually set org.opencontainers.image.ref.name via a CLI flag. Can be a follow-up PR.

cli.StringFlag{
Name: "oci-ref-name",
Value: "",
Usage: "override org.opencontainers.image.ref.name annotation",
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not keep this

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem is this annotation is so poorly defined that is unclear what the correct behavior is when outputting multiple images. I agree we should solve this in a follow up to make sure we come up with a good solution for generating OCI indexes in a sane manner when multiple images are involved.

@crosbymichael
Copy link
Copy Markdown
Member

LGTM

Copy link
Copy Markdown
Member

@fuweid fuweid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@fuweid fuweid merged commit e61f7f4 into containerd:master May 17, 2019
@dmcgowan dmcgowan deleted the export-docker-compatibility branch September 10, 2019 17:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants