Automatically decompress archives for transfer service import#9864
Automatically decompress archives for transfer service import#9864mxpv merged 1 commit intocontainerd:mainfrom
Conversation
|
Hi @hinshun. Thanks for your PR. I'm waiting for a containerd member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
| } | ||
| return archive.ImportIndex(ctx, store, iis.stream, opts...) | ||
|
|
||
| r, err := compression.DecompressStream(iis.stream) |
There was a problem hiding this comment.
Should we be making use of the media type here? Possibly only running through this logic if the media type indicates compression or is empty.
There was a problem hiding this comment.
We can check for empty media type, but here the input is a compressed OCI tarball (with possibly uncompressed layers). Typically the media type here is ocispec.MediaTypeImageIndex and there isn't a compressed version afaik.
Signed-off-by: Edgar Lee <[email protected]>
7dd6887 to
34c5458
Compare
Automatically decompress archives for transfer service import
Currently,
ctr image importas well as the transfer service import withNewImageImportStreamdoes not support archives that are compressed (gzip, zstd).I've tried passing a decompressed stream like so:
However, I believe there is a bug in the proxy transfer service because it fails during
untarBlobwithfailed to read expected number of bytes: unexpected EOF. Patching containerd, I found it errors in this line with the following parameters:copied 34860544, size 34877440, ws.Offset 0, size-ws.Offset 34877440:https://github.com/containerd/containerd/blob/v1.7.13/content/helpers.go#L184
When doing the decompression either with a local store
local.NewStoreor on the server-side (this patch), then it works perfectly fine.