Skip to content

add Image content converter#4881

Merged
dmcgowan merged 1 commit intocontainerd:masterfrom
AkihiroSuda:converter
Jan 22, 2021
Merged

add Image content converter#4881
dmcgowan merged 1 commit intocontainerd:masterfrom
AkihiroSuda:converter

Conversation

@AkihiroSuda
Copy link
Copy Markdown
Member

@AkihiroSuda AkihiroSuda commented Dec 24, 2020

Go example:

opts := []converter.Opt{
  // convert Docker media types to OCI ones
  converter.WithDocker2OCI(true),
  // convert tar.gz layers to uncompressed tar layers
  converter.WithLayerConvertFunc(uncompress.LayerConvertFunc),
}
srcRef := "example.com/foo:orig"
dstRef := "example.com/foo:converted"
dstImg, err = converter.Convert(ctx, client, dstRef, srcRef, opts...)
fmt.Println(dstImg.Target)

ctr example: ctr images convert --oci --uncompress example.com/foo:orig example.com/foo:converted

Go test: go test -exec sudo -test.root -test.run TestConvert

The implementation is from containerd/stargz-snapshotter#224, but eStargz-specific functions are not included in this PR.

eStargz converter can be specified by importing estargz package and using WithLayerConvertFunc(estargz.LayerConvertFunc) option.

This converter interface will be potentially useful for converting zstd and ocicrypt layers as well.

@AkihiroSuda
Copy link
Copy Markdown
Member Author

cc @ktock

@AkihiroSuda AkihiroSuda force-pushed the converter branch 2 times, most recently from cbe4dbf to e7f8c59 Compare December 24, 2020 09:38
@containerd containerd deleted a comment from theopenlab-ci Bot Dec 24, 2020
@containerd containerd deleted a comment from theopenlab-ci Bot Dec 24, 2020
@containerd containerd deleted a comment from theopenlab-ci Bot Dec 24, 2020
@theopenlab-ci
Copy link
Copy Markdown

theopenlab-ci Bot commented Dec 31, 2020

Build succeeded.

@AkihiroSuda
Copy link
Copy Markdown
Member Author

CI failure is unrelated

--- FAIL: TestLosetup (0.20s)
    --- FAIL: TestLosetup/RemoveLoopDevicesAssociatedWithImage (0.01s)
Error:         losetup_test.go:96: assertion failed: expected [/dev/loop1 /dev/loop4] (length 2) to have length 0
FAIL
FAIL	github.com/containerd/containerd/snapshots/devmapper/losetup	0.219s

Comment thread convert.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The package comment should be updated.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see a need for this to be added in the top level package. It is an image utility function which can operate entirely from a client instance. I realize there are a few other functions which we could categorize like that, but don't see that as a reason to continue to add more.

Copy link
Copy Markdown
Member Author

@AkihiroSuda AkihiroSuda Jan 12, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved the function to images/converter pkg.

convert_test.go still remains in the top level pkg as it depends on the private functions in the top pkg.

@AkihiroSuda AkihiroSuda force-pushed the converter branch 3 times, most recently from 3576a07 to b8c590b Compare January 12, 2021 05:59
@theopenlab-ci
Copy link
Copy Markdown

theopenlab-ci Bot commented Jan 12, 2021

Build succeeded.

  • containerd-build-arm64 : RETRY_LIMIT in 1m 00s (non-voting)

Comment thread cmd/ctr/commands/images/convert.go Outdated
@AkihiroSuda
Copy link
Copy Markdown
Member Author

@dmcgowan PTAL

@containerd containerd deleted a comment from theopenlab-ci Bot Jan 15, 2021
@containerd containerd deleted a comment from theopenlab-ci Bot Jan 15, 2021
@AkihiroSuda
Copy link
Copy Markdown
Member Author

rebased

@theopenlab-ci
Copy link
Copy Markdown

theopenlab-ci Bot commented Jan 21, 2021

Build succeeded.

Comment thread images/mediatypes.go Outdated
Comment thread images/mediatypes.go Outdated
@dmcgowan
Copy link
Copy Markdown
Member

Two small changes then looks good

Go example:
```go
opts := []converter.Opt{
  // convert Docker media types to OCI ones
  converter.WithDocker2OCI(true),
  // convert tar.gz layers to uncompressed tar layers
  converter.WithLayerConvertFunc(uncompress.LayerConvertFunc),
}
srcRef := "example.com/foo:orig"
dstRef := "example.com/foo:converted"
dstImg, err = converter.Convert(ctx, client, dstRef, srcRef, opts...)
fmt.Println(dstImg.Target)
```

ctr example: `ctr images convert --oci --uncompress example.com/foo:orig example.com/foo:converted`

Go test: `go test -exec sudo -test.root -test.run TestConvert`

The implementation is from containerd/stargz-snapshotter#224,
but eStargz-specific functions are not included in this PR.

eStargz converter can be specified by importing `estargz` package and using `WithLayerConvertFunc(estargz.LayerConvertFunc)` option.

This converter interface will be potentially useful for converting zstd and ocicrypt layers as well.

Signed-off-by: Akihiro Suda <[email protected]>
@AkihiroSuda
Copy link
Copy Markdown
Member Author

Updated to apply containerd/stargz-snapshotter#246
cc @ktock

@theopenlab-ci
Copy link
Copy Markdown

theopenlab-ci Bot commented Jan 22, 2021

Build succeeded.

@dmcgowan dmcgowan merged commit 5a66c2a into containerd:master Jan 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants