Skip to content

contrib: Docker v1 image importer#1602

Closed
AkihiroSuda wants to merge 1 commit intocontainerd:masterfrom
AkihiroSuda:dockersaveload
Closed

contrib: Docker v1 image importer#1602
AkihiroSuda wants to merge 1 commit intocontainerd:masterfrom
AkihiroSuda:dockersaveload

Conversation

@AkihiroSuda
Copy link
Copy Markdown
Member

@AkihiroSuda AkihiroSuda commented Oct 5, 2017

EDIT (Apr 10, 2018): TODO: cherry-pick containerd/cri#727
EDIT (Dec 5, 2017): Now OCI fixes were merged separately via #1880, and almost whole this PR consists of just copy-pasted code from containerd/cri#360 . So now this PR should be easy to review.


This PR adds an importer for Docker v1 images.

ctr help

$ ctr images import --help            
NAME:                             
   ctr images import - import images                                

USAGE:                            
   ctr images import [command options] [flags] <in>                 

DESCRIPTION:                      
   Import images from a tar stream.                                 
Implemented formats:              
- oci.v1     (default)            
- docker.v1                       


For oci.v1 format, you need to specify --oci-name because an OCI archive contains image refs (tags)                                     
but does not contain the base image name.                           

e.g.                              
  $ ctr images import --format oci.v1 --oci-name foo/bar foobar.tar 

If foobar.tar contains an OCI ref named "latest" and anonymous ref "sha256:deadbeef", the command will create                           
"foo/bar:latest" and "foo/bar@sha256:deadbeef" images in the containerd store.                                                          


For docker.v1 format, you don't need to specify the image name because it is automatically generated from the                           
RepoTags included in the archive, as in Docker.                     
RepoTags are normalized in the following manner:                    
  busybox         --> docker.io/library/busybox:latest              
  busybox:latest  --> docker.io/library/busybox:latest              
  library/busybox --> docker.io/library/busybox:latest              
  gcr.io/library/busybox:latest@sha256:deadbeef --> gcr.io/library/busybox@sha256:deadbeef                                              


OPTIONS:                          
   --format value       image format. See DESCRIPTION. (default: "oci.v1")                                                              
   --oci-name value     prefix added to either oci.v1 ref annotation or digest (default: "unknown/unknown")                             
   --snapshotter value  snapshotter name. Empty value stands for the daemon default value. (default: "overlayfs")

Importing Docker images

You can import multiple images on a single archive at once.

$ docker save busybox alpine:latest docker.io/library/hello-world | sudo ctr images import --format docker.v1 -
unpacking docker.io/library/hello-world:latest (sha256:fb1bc3b85163b220a42eb1901b62d45e6f6c95440a25804ffaa7dc47322aebc3)...done
unpacking docker.io/library/busybox:1.25.0 (sha256:1f0a80413771fd34977e18025bcbf07b56c0512592d614745ea8f71046a69c33)...done
unpacking docker.io/library/busybox:latest (sha256:18d0ac541d4b86286a16dcf92967b74aed97fb584f0cc6f7c30add824352ca32)...done
unpacking docker.io/library/alpine:latest (sha256:a2c81ecd8c707b2f1864b5c30a1a36260a5eb8ed74ef6091b0aadfa43864e9f8)...done

RepoTags are normalized in docker.io/library/foo:bar form:

$ sudo ctr images ls
REF                                  TYPE                                                 DIGEST                                                                  SIZE    PLATFORMS   LABELS 
docker.io/library/alpine:latest      application/vnd.docker.distribution.manifest.v2+json sha256:a2c81ecd8c707b2f1864b5c30a1a36260a5eb8ed74ef6091b0aadfa43864e9f8 4.0 MiB linux/amd64 -      
docker.io/library/busybox:1.25.0     application/vnd.docker.distribution.manifest.v2+json sha256:1f0a80413771fd34977e18025bcbf07b56c0512592d614745ea8f71046a69c33 1.2 MiB linux/amd64 -      
docker.io/library/busybox:latest     application/vnd.docker.distribution.manifest.v2+json sha256:18d0ac541d4b86286a16dcf92967b74aed97fb584f0cc6f7c30add824352ca32 1.3 MiB linux/amd64 -      
docker.io/library/hello-world:latest application/vnd.docker.distribution.manifest.v2+json sha256:fb1bc3b85163b220a42eb1901b62d45e6f6c95440a25804ffaa7dc47322aebc3 5.3 KiB linux/amd64 -      

GC labels:

$ sudo ctr content ls
DIGEST                                                                  SIZE            AGE             LABELS
sha256:18d0ac541d4b86286a16dcf92967b74aed97fb584f0cc6f7c30add824352ca32 356 B           28 seconds      containerd.io/gc.ref.content.1=sha256:6a749002dd6a65988a6696ca4d0c4cbe87145df74e3bf6feae4025ab28f420f2,containerd.io/gc.ref.content.0=sha256:54511612f1c4d97e93430fc3d5dc2f05dfbe8fb7e6259b7351deeca95eaf2971
sha256:1f0a80413771fd34977e18025bcbf07b56c0512592d614745ea8f71046a69c33 356 B           28 seconds      containerd.io/gc.ref.content.0=sha256:2b8fd9751c4c0f5dd266fcae00707e67a2545ef34f9a29354585f93dac906749,containerd.io/gc.ref.content.1=sha256:8ac8bfaff55af948c796026ee867448c5b5b5d9dd3549f4006d9759b25d4a893
sha256:2b8fd9751c4c0f5dd266fcae00707e67a2545ef34f9a29354585f93dac906749 1.459 kB        28 seconds      containerd.io/gc.ref.snapshot.overlayfs=sha256:8ac8bfaff55af948c796026ee867448c5b5b5d9dd3549f4006d9759b25d4a893
sha256:51d9ee0d3e49cf0d51e2149c89b0c428ed1151a7daecdcfaba8a3fc71ef3e8d0 3.584 kB        28 seconds      -
sha256:54511612f1c4d97e93430fc3d5dc2f05dfbe8fb7e6259b7351deeca95eaf2971 1.497 kB        28 seconds      containerd.io/gc.ref.snapshot.overlayfs=sha256:6a749002dd6a65988a6696ca4d0c4cbe87145df74e3bf6feae4025ab28f420f2
sha256:5bef08742407efd622d243692b79ba0055383bbce12900324f75e56f589aedb0 4.221 MB        28 seconds      -
sha256:6a749002dd6a65988a6696ca4d0c4cbe87145df74e3bf6feae4025ab28f420f2 1.338 MB        28 seconds      -
sha256:725dcfab7d63ec87fa6fc5162ca0a36f67ad89cdfd7f9a156957b79d8b855368 1.51 kB         28 seconds      containerd.io/gc.ref.snapshot.overlayfs=sha256:51d9ee0d3e49cf0d51e2149c89b0c428ed1151a7daecdcfaba8a3fc71ef3e8d0
sha256:7328f6f8b41890597575cbaadc884e7386ae0acc53b747401ebce5cf0d624560 1.52 kB         28 seconds      containerd.io/gc.ref.snapshot.overlayfs=sha256:5bef08742407efd622d243692b79ba0055383bbce12900324f75e56f589aedb0
sha256:8ac8bfaff55af948c796026ee867448c5b5b5d9dd3549f4006d9759b25d4a893 1.293 MB        28 seconds      -
sha256:a2c81ecd8c707b2f1864b5c30a1a36260a5eb8ed74ef6091b0aadfa43864e9f8 356 B           28 seconds      containerd.io/gc.ref.content.0=sha256:7328f6f8b41890597575cbaadc884e7386ae0acc53b747401ebce5cf0d624560,containerd.io/gc.ref.content.1=sha256:5bef08742407efd622d243692b79ba0055383bbce12900324f75e56f589aedb0
sha256:fb1bc3b85163b220a42eb1901b62d45e6f6c95440a25804ffaa7dc47322aebc3 353 B           28 seconds      containerd.io/gc.ref.content.1=sha256:51d9ee0d3e49cf0d51e2149c89b0c428ed1151a7daecdcfaba8a3fc71ef3e8d0,containerd.io/gc.ref.content.0=sha256:725dcfab7d63ec87fa6fc5162ca0a36f67ad89cdfd7f9a156957b79d8b855368

Importing OCI images

$ skopeo copy docker://busybox:latest oci-archive:a.tar:latest
$ sudo ctr images import --format oci.v1 --oci-name example.com/foo/busybox a.tar 
unpacking example.com/foo/busybox:latest (sha256:13dfd2f55e8948d1f1f1f5a834e146f42e3bb0b1750f06dd6865b950d1d3ec9e)...done
$ sudo ctr images ls
REF                            TYPE                                       DIGEST                                                                  SIZE      PLATFORMS   LABELS 
example.com/foo/busybox:latest application/vnd.oci.image.manifest.v1+json sha256:13dfd2f55e8948d1f1f1f5a834e146f42e3bb0b1750f06dd6865b950d1d3ec9e 699.4 KiB linux/amd64 -      
$ sudo ctr content ls
DIGEST                                                                  SIZE            AGE             LABELS
sha256:0ffadd58f2a61468f527cc4f0fc45272ee4a1a428abe014546c89de2aa6a0eb5 715.3 kB        30 seconds      -
sha256:13dfd2f55e8948d1f1f1f5a834e146f42e3bb0b1750f06dd6865b950d1d3ec9e 347 B           30 seconds      containerd.io/gc.ref.content.1=sha256:0ffadd58f2a61468f527cc4f0fc45272ee4a1a428abe014546c89de2aa6a0eb5,containerd.io/gc.ref.content.0=sha256:f9a6f01030e69f43f260597505b4d667503a35518db1f03e660b19793039fbef
sha256:f9a6f01030e69f43f260597505b4d667503a35518db1f03e660b19793039fbef 575 B           30 seconds      containerd.io/gc.ref.snapshot.overlayfs=sha256:0271b8eebde3fa9a6126b1f2335e170f902731ab4942f9f1914e77016540c7bb

In this example, skopeo is used for preparing an OCI image to import, but it is also possible to create an OCI image from the containerd store: ctr images export a.tar docker.io/library/busybox:latest

About this PR

  • Name normalization (busybox -> docker.io/library/busybox:latest) is based on @Random-Liu 's CRI-containerd implementation Add image load cri#360 , so the design should be now OK.
  • This PR "vendor"s github.com/docker/distribution/reference as github.com/containerd/containerd/contrib/docker1/reference. So please skip reviewing this pkg. This pkg should soon go away.
  • For reviewing, please focus on GC stuff. (This PR also contains fix for OCI importer, which were not updated for GC OCI fix was merged in importer: refactor and fix GC #1880 )

Update #1229
@Random-Liu @stevvooe

@codecov-io
Copy link
Copy Markdown

codecov-io commented Oct 5, 2017

Codecov Report

Merging #1602 into master will increase coverage by 1.36%.
The diff coverage is 57.91%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1602      +/-   ##
==========================================
+ Coverage   41.06%   42.43%   +1.36%     
==========================================
  Files          66       73       +7     
  Lines        7748     8430     +682     
==========================================
+ Hits         3182     3577     +395     
- Misses       4064     4331     +267     
- Partials      502      522      +20
Flag Coverage Δ
#windows 42.43% <57.91%> (+1.36%) ⬆️
Impacted Files Coverage Δ
contrib/docker1/importer.go 0% <0%> (ø)
contrib/docker1/normalize.go 55% <55%> (ø)
contrib/docker1/reference/helpers.go 58.33% <58.33%> (ø)
contrib/docker1/reference/normalize.go 78.35% <78.35%> (ø)
contrib/docker1/reference/reference.go 79.08% <79.08%> (ø)
contrib/docker1/reference/digestset/set.go 79.86% <79.86%> (ø)
contrib/docker1/reference/regexp.go 90.9% <90.9%> (ø)
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9304193...b163f65. Read the comment docs.

Comment thread cmd/ctr/import.go Outdated

The default selector string is <ref>, but it is not guaranteed to be a valid selector string, depending on the importer implementation.
`,
Flags: []cli.Flag{
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This selector interface might be too much overloaded. RFC.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you use the filter package?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, I'll try

@Random-Liu
Copy link
Copy Markdown
Member

@AkihiroSuda Thanks for working on this!

@AkihiroSuda AkihiroSuda changed the title [WIP] contrib: Docker v1 image importer contrib: Docker v1 image importer Oct 9, 2017
@AkihiroSuda
Copy link
Copy Markdown
Member Author

updated PR

Comment thread client.go Outdated
// OCI format is assumed by default.
// Filter strings are format-specific, but needs to match exactly one item.
//
// TODO(AkihiroSuda): support importing multiple images from a single archive at once? (map[ref]filter?)
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RFC

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think this is a must.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For us, we only need loading one image from tarball for now. We only build one image into each tar in the Kubernetes building process.

Comment thread client.go Outdated
func (c *Client) Import(ctx context.Context, ref string, reader io.Reader, opts ...ImportOpt) (Image, error) {
iopts, err := resolveImportOpt(ref, opts...)
// Note that unreferenced blobs may be imported to the content store as well.
func (c *Client) Import(ctx context.Context, ref string, reader io.Reader, filter string, opts ...ImportOpt) (Image, error) {
Copy link
Copy Markdown
Member Author

@AkihiroSuda AkihiroSuda Oct 9, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On second thought, should I remove filters, and change this function definition to Import (ctx context.Context, reader io.Reader, opts ...ImportOpt)(map[string]Image, error)?

The map key for map[string]Image is reference string, which is determined by the importer implementation.
(However, it may override existing refs... should we let importer add some prefix to refs?)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would the prefix represent? Various importers?

Copy link
Copy Markdown
Member Author

@AkihiroSuda AkihiroSuda Oct 11, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suppose we are importing Docker v1 Image archive which has RepoTags: [docker.io/library/busybox:latest, busybox:latest] and RepoTags: [docker.io/library/nginx:latest]. (Note that a single archive contain multiple different images)

prefix := "tmp-42424242.local/"
importer := &docker1.Importer{Prefix: prefix}
res, err := Import (ctx, tarReader, WithImporter(importer))

Then the result map keys will be tmp-424242.local/docker.io/library/busybox:latest, tmp-424242.local/busybox:latest, and tmp-424242.local/docker.io/library/nginx:latest.

The caller can rename these images later by itself.


For OCI images, which has tags (e.g. latest) but no repotags (e.g. busybox:latest), prefix might be like tmp-424242.local/busybox:.

cc @stevvooe @dmcgowan @Random-Liu RFC

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AkihiroSuda What is this filter used for? Selecting an image from the tarball when there are multiple images inside the tar? I didn't quite get it.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm suggesting remove filters here to support loading multiple images from a single archive at once. #1602 (comment)

Copy link
Copy Markdown
Member

@Random-Liu Random-Liu Oct 11, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, the prefix part seems confusing to me. :p

Are you trying to avoid conflict refs inside the tarball? Or conflict ref with existing ref in containerd?

If it's the former, is that going to happen?
If it's the later, how does Docker solve it today?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Latter and it is not resolved in Docker (just override).

For importing v1 image, you could omit prefix with risk of overriding.

But for importing OCI image, we somehow need to pass extra prefix (and default tag?), because OCI images lack RepoTags-equivalent.

@jessvalarezo
Copy link
Copy Markdown
Contributor

jessvalarezo commented Oct 10, 2017

Hi @AkihiroSuda! From just reading the description I'm not sure what <ref> or <in> are, or what they should look like.

Comment thread client.go Outdated
if c.format != "" {
return errors.New("format already set")
if c.importer != nil {
return errors.New("impoter already set")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: s/impoter/importer

Comment thread client.go Outdated
return c.importFromOCITar(ctx, ref, reader, iopts)
default:
return nil, errors.Errorf("unsupported format: %s", iopts.format)
imgrec := images.Image{
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what does rec mean here?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably "record" (copy-pasted from Pull())

@Random-Liu
Copy link
Copy Markdown
Member

@stevvooe I've tried this, and this does work. Can we get this in anytime soon? This is blocking our work to integrate cri-containerd with kube-up.sh.

Thanks!

@AkihiroSuda
Copy link
Copy Markdown
Member Author

@Random-Liu I think this PR still needs discussion about how to import multiple images in a single archive at once.

Since we are not going to stabilize the client library in v1.0, IIUC it is not impossible to merge this PR to the upstream right now, but could you consider copy-pasting the code of this PR to CRI-containerd side?
After we find and implement the correct importer interface, we can deduplicate the code.

@Random-Liu
Copy link
Copy Markdown
Member

Random-Liu commented Oct 13, 2017

@AkihiroSuda The problem is that we need a command line tool to load image during Kubernetes bootstrap. We do have our own command line tool crictl, but it is designed to be CRI compatible and mainly for debug purpose, we can't add load in it.

The only command line tool we could use for load is ctr. :)

One option is to cherypick this commit, and maintain our own temporary containerd branch for us to use for now. I'm fine with that if we think this PR may take longer to merge.

@AkihiroSuda
Copy link
Copy Markdown
Member Author

@Random-Liu
ctr is just a dev tool, and we are not going to keep compatibility of ctr CLI across different versions.
So I'm not sure it is a correct usecase of ctr.

How about adding some CRI-containerd-specific utility command to CRI-containerd repo, which CRI-containerd maintainers can keep compatibility as they want?
e.g. cri-containerdutil import.

@Random-Liu
Copy link
Copy Markdown
Member

@AkihiroSuda I just checked, we do use docker load for system containers in production today, so we do need a reliable replacement. Ideally we should move to private registry, but I doubt that is going to happen anytime soon.

Since we control containerd version in our production environment today, as long as the implementation is stable, I'm fine with ctr to be changed in the future, although I prefer no significant change :)

If we do find that ctr implementation doesn't meet the requirement, we'll have to build another tool.

@AkihiroSuda
Copy link
Copy Markdown
Member Author

@stevvooe @Random-Liu shall we discuss this in person tomorrow?

@AkihiroSuda
Copy link
Copy Markdown
Member Author

As discussed in Moby Summit:

  • Client library in this repo should support "prefix" for importing multiple images from a single archive
  • ctr-containerd should have its own importer CLI

I'll try to update this PR next week (if WiFi at the hotel is fine😅)

@AkihiroSuda
Copy link
Copy Markdown
Member Author

s/ctr-containerd/cri-containerd/

@Random-Liu
Copy link
Copy Markdown
Member

Random-Liu commented Oct 22, 2017

cri-containerd should have its own importer CLI

I'll use ctr for now, and will add another command or cli on our side, if this doesn't meet the requirement. Actually our requirement is pretty simple. :)

@stevvooe @AkihiroSuda

  1. Can we make <ref> the second arg so that it could be optional? For docker, it's not required, right?
  2. @AkihiroSuda Can we file a simple PR making minimal change that could be merged sooner?
    Probably without changing the OCI image import/export logic, just add the docker image support? We really need this now, and it's blocking our side. Kubernetes code freeze is 3rd week of Nov., I need to get Kubernetes side change in before that time. :p

I'll make change 1) based on this PR, and maintain our own fork for now.

@AkihiroSuda
Copy link
Copy Markdown
Member Author

@Random-Liu @stevvooe

Updated PR.

Please review the CLI design.

Fixes for client API, minor bugs, and typos will be follow-up PR if needed.

@AkihiroSuda
Copy link
Copy Markdown
Member Author

@Random-Liu @dmcgowan @stevvooe PTAL?

I think the GC&interface fix for OCI importer should be merged before v1.0 GA.
(Let me know if I should split this PR)

For Docker v1 importer, no need to hurry, but it is placed on contrib directory and safe to merge, I think.

Comment thread contrib/docker1/normalize_test.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't look right: the normalization should ignore the tag.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You meant it should NOT ignore the tag?

This code is copypasted from CRI-containerd.
Does CRI-containerd need to be modified?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps. It shouldn't be dropping information from the image reference.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member

@Random-Liu Random-Liu Mar 13, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is specific to cri-containerd I think. The reason is that in CRI there is only repoDigest and repoTag. I remember I tried docker, and if we pull a image which has both tag and digest, it only keep the digest and ignore the tag. Let me try again.

In containerd, we should do whatever is right. :)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$ docker pull busybox:latest@sha256:2107a35b58593c58ec5f4e8f2c4a70d195321078aebfadfbfb223a2ff4a4ed21
sha256:2107a35b58593c58ec5f4e8f2c4a70d195321078aebfadfbfb223a2ff4a4ed21: Pulling from library/busybox
d070b8ef96fc: Pull complete 
Digest: sha256:2107a35b58593c58ec5f4e8f2c4a70d195321078aebfadfbfb223a2ff4a4ed21
Status: Downloaded newer image for busybox@sha256:2107a35b58593c58ec5f4e8f2c4a70d195321078aebfadfbfb223a2ff4a4ed21
$ docker inspect sha256:f6e427c148a766d2d6c117d67359a0aa7d133b5bc05830a7ff6e8b64ff6b1d1d
[
    {
        "Id": "sha256:f6e427c148a766d2d6c117d67359a0aa7d133b5bc05830a7ff6e8b64ff6b1d1d",
        "RepoTags": [],
        "RepoDigests": [
            "busybox@sha256:2107a35b58593c58ec5f4e8f2c4a70d195321078aebfadfbfb223a2ff4a4ed21"
        ],
        "Parent": "",
        "Comment": "",
        "Created": "2018-02-28T22:14:49.023807051Z",
        "Container": "8d2c840a1a9b2544fe713c2e24b6757d52328f09bdfc9c2ef6219afbf7ae6b59",
        "ContainerConfig": {
            "Hostname": "8d2c840a1a9b",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": [
                "/bin/sh",
                "-c",
                "#(nop) ",
                "CMD [\"sh\"]"
            ],
            "ArgsEscaped": true,
            "Image": "sha256:8cae5980d887cc55ba2f978ae99c662007ee06d79881678d57f33f0473fe0736",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": {}
        },
        "DockerVersion": "17.06.2-ce",
        "Author": "",
        "Config": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
            ],
            "Cmd": [
                "sh"
            ],
            "ArgsEscaped": true,
            "Image": "sha256:8cae5980d887cc55ba2f978ae99c662007ee06d79881678d57f33f0473fe0736",
            "Volumes": null,
            "WorkingDir": "",
            "Entrypoint": null,
            "OnBuild": null,
            "Labels": null
        },
        "Architecture": "amd64",
        "Os": "linux",
        "Size": 1146369,
        "VirtualSize": 1146369,
        "GraphDriver": {
            "Name": "aufs",
            "Data": null
        },
        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:c5183829c43c4698634093dc38f9bee26d1b931dedeba71dbee984f42fe1270d"
            ]
        }
    }
]

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added Conservative bool for switching the behavior.
If true, it removes tag as in CRI implementation.

@AkihiroSuda AkihiroSuda changed the title contrib: Docker v1 image importer (also contains GC fix for OCI importer) contrib: Docker v1 image importer (also contains GC fix for OCI importer #1880) Dec 5, 2017
@AkihiroSuda AkihiroSuda added this to the 1.0.0 post release milestone Dec 5, 2017
@AkihiroSuda AkihiroSuda changed the title contrib: Docker v1 image importer (also contains GC fix for OCI importer #1880) contrib: Docker v1 image importer Dec 5, 2017
@AkihiroSuda AkihiroSuda modified the milestones: 1.0.0 post release, 1.1 Jan 30, 2018
@stevvooe
Copy link
Copy Markdown
Member

Where does this PR sit against #2175?

@Random-Liu
Copy link
Copy Markdown
Member

@stevvooe For CRI, we do still need a load command, until we remove the internal in-memory cache for images, and always list images from containerd.

Since we've removed grpc, it makes sense to always list images from containerd. We can:

  • Either keep both ctr import and ctr cri load;
  • Or only have ctr import, and still have a separate ctrcri binary for this release. And we can get rid of the extra in-memory image cache in the future.

@AkihiroSuda
Copy link
Copy Markdown
Member Author

Updated, but feel free to postpone this to v1.2 if there is a problem.

Comment thread contrib/docker1/reference/doc.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed the comment

@stevvooe stevvooe modified the milestones: 1.1, 1.2 Mar 21, 2018
For usage, please refer to `ctr images import --help`.

Signed-off-by: Akihiro Suda <[email protected]>
/*
Copyright The docker/distribution Authors.

Licensed under the Apache License, Version 2.0 (the "License");
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This redundancy is for making ltag happy.

I think ideally we should just list a single Apache boilerplate with Copyright The docker/distribution Authors and containerd Authors.

@kunalkushwaha

@tonistiigi
Copy link
Copy Markdown
Member

@AkihiroSuda The 1.1 spec defines that the layout that Docker produces is only for backward compatibility. The actual layout is determined from the manifest.json file that contains the full paths to all files the manifest uses. The importer should handle any path in manifest.json like Docker implementation does.

@AkihiroSuda
Copy link
Copy Markdown
Member Author

temporarily closing for now

@Random-Liu
Copy link
Copy Markdown
Member

Random-Liu commented Sep 11, 2018

containerd/cri#865 is merged. We should be able to see images loaded/pulled into containerd with ctr now.

If we can get this in containerd, we can deprecate the ctr cri load command.

@dmcgowan
Copy link
Copy Markdown
Member

I am working on that now

@Random-Liu
Copy link
Copy Markdown
Member

@dmcgowan Cool! :D

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants