c8d/inspect: Add Manifests field#48264
Conversation
Manifests fieldManifests field
013c580 to
61758d1
Compare
54c91a4 to
49956b1
Compare
laurazard
left a comment
There was a problem hiding this comment.
LGTM (took my brain a sec to wrap around the changes 😅 probably because I haven't looked at this in a while)
7cd9db0 to
8f3682a
Compare
|
Rebased on top of #48330 (as both touch the same code). |
8f3682a to
a2945a1
Compare
Perhaps less relevant (but perhaps it is?) I was curious if there's a performance impact for getting all the info; but it's very much possible we would need (most of) the data in either case (that said; perhaps there's parts we could keep cached somewhere to save on creating the summary). But regardless, depending on the number of architectures in an image, and depending on whether there's attestations, the Manifests still produce a lot of extra information. Here's docker pull --platform=linux/amd64 alpine
Using default tag: latest
latest: Pulling from library/alpine
c6a83fedfae6: Download complete
Digest: sha256:0a4eaa0eecf5f8c050e5bba433f58c052be7587ee8af3e8b3910ef9ab5fbe9f5
Status: Downloaded newer image for alpine:latest
docker.io/library/alpine:latestdocker image inspect alpineThis outputs;
[
{
"Id": "sha256:0a4eaa0eecf5f8c050e5bba433f58c052be7587ee8af3e8b3910ef9ab5fbe9f5",
"RepoTags": [
"alpine:latest"
],
"RepoDigests": [
"alpine@sha256:0a4eaa0eecf5f8c050e5bba433f58c052be7587ee8af3e8b3910ef9ab5fbe9f5"
],
"Parent": "",
"Comment": "",
"Created": "2024-07-22T22:26:43.778747613Z",
"DockerVersion": "",
"Author": "",
"Config": {
"Hostname": "",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": [
"/bin/sh"
],
"Image": "",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": null,
"OnBuild": null,
"Labels": null
},
"Architecture": "amd64",
"Os": "linux",
"Size": 2381,
"GraphDriver": {
"Data": null,
"Name": "overlayfs"
},
"RootFS": {
"Type": "layers",
"Layers": [
"sha256:78561cef0761903dd2f7d09856150a6d4fb48967a8f113f3e33d79effbf59a07"
]
},
"Metadata": {
"LastTagTime": "2024-08-15T09:43:26.820642347Z"
},
"Manifests": [
{
"ID": "sha256:eddacbc7e24bf8799a4ed3cdcfa50d4b88a323695ad80f317b6629883b2c2a78",
"Descriptor": {
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"digest": "sha256:eddacbc7e24bf8799a4ed3cdcfa50d4b88a323695ad80f317b6629883b2c2a78",
"size": 528,
"platform": {
"architecture": "amd64",
"os": "linux"
}
},
"Available": true,
"Size": {
"Content": 3624891,
"Total": 12083131
},
"Kind": "image",
"ImageData": {
"Platform": {
"architecture": "amd64",
"os": "linux"
},
"Size": {
"Unpacked": 8458240
},
"Containers": null
}
},
{
"ID": "sha256:5c7e326e3c8a8c51654a6c5d94dac98d7f6fc4b2a762d86aaf67b7e76a6aee46",
"Descriptor": {
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"digest": "sha256:5c7e326e3c8a8c51654a6c5d94dac98d7f6fc4b2a762d86aaf67b7e76a6aee46",
"size": 528,
"platform": {
"architecture": "arm",
"os": "linux",
"variant": "v6"
}
},
"Available": false,
"Size": {
"Content": 0,
"Total": 0
},
"Kind": "image",
"ImageData": {
"Platform": {
"architecture": "arm",
"os": "linux",
"variant": "v6"
},
"Size": {
"Unpacked": 0
},
"Containers": null
}
},
{
"ID": "sha256:fda9b1b812b25c68f94da5b719039bfa9a3b76e167a8f87e7fc62cb159d21ca1",
"Descriptor": {
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"digest": "sha256:fda9b1b812b25c68f94da5b719039bfa9a3b76e167a8f87e7fc62cb159d21ca1",
"size": 528,
"platform": {
"architecture": "arm",
"os": "linux",
"variant": "v7"
}
},
"Available": false,
"Size": {
"Content": 0,
"Total": 0
},
"Kind": "image",
"ImageData": {
"Platform": {
"architecture": "arm",
"os": "linux",
"variant": "v7"
},
"Size": {
"Unpacked": 0
},
"Containers": null
}
},
{
"ID": "sha256:24ba417e25e780ff13c888ccb1badec5b027944666ff695681909bafe09a3944",
"Descriptor": {
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"digest": "sha256:24ba417e25e780ff13c888ccb1badec5b027944666ff695681909bafe09a3944",
"size": 528,
"platform": {
"architecture": "arm64",
"os": "linux",
"variant": "v8"
}
},
"Available": false,
"Size": {
"Content": 0,
"Total": 0
},
"Kind": "image",
"ImageData": {
"Platform": {
"architecture": "arm64",
"os": "linux",
"variant": "v8"
},
"Size": {
"Unpacked": 0
},
"Containers": null
}
},
{
"ID": "sha256:fa66aa594ffa884dff44f4a97821756545834505df611c375a30c45926402f89",
"Descriptor": {
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"digest": "sha256:fa66aa594ffa884dff44f4a97821756545834505df611c375a30c45926402f89",
"size": 528,
"platform": {
"architecture": "386",
"os": "linux"
}
},
"Available": false,
"Size": {
"Content": 0,
"Total": 0
},
"Kind": "image",
"ImageData": {
"Platform": {
"architecture": "386",
"os": "linux"
},
"Size": {
"Unpacked": 0
},
"Containers": null
}
},
{
"ID": "sha256:a01843eb870e11bb20c78a9068269c810f14dd5c49364064fa3f9cf798f666dd",
"Descriptor": {
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"digest": "sha256:a01843eb870e11bb20c78a9068269c810f14dd5c49364064fa3f9cf798f666dd",
"size": 528,
"platform": {
"architecture": "ppc64le",
"os": "linux"
}
},
"Available": false,
"Size": {
"Content": 0,
"Total": 0
},
"Kind": "image",
"ImageData": {
"Platform": {
"architecture": "ppc64le",
"os": "linux"
},
"Size": {
"Unpacked": 0
},
"Containers": null
}
},
{
"ID": "sha256:e99a4d9aa9f905cee4171c6d616e4008fa32202fa8aa8aa65efcafbc3a0f5fa5",
"Descriptor": {
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"digest": "sha256:e99a4d9aa9f905cee4171c6d616e4008fa32202fa8aa8aa65efcafbc3a0f5fa5",
"size": 528,
"platform": {
"architecture": "riscv64",
"os": "linux"
}
},
"Available": false,
"Size": {
"Content": 0,
"Total": 0
},
"Kind": "image",
"ImageData": {
"Platform": {
"architecture": "riscv64",
"os": "linux"
},
"Size": {
"Unpacked": 0
},
"Containers": null
}
},
{
"ID": "sha256:14da06d3a8959002fd621dd3994a254e2126d239f2fe69e829fd95d16ce81dea",
"Descriptor": {
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"digest": "sha256:14da06d3a8959002fd621dd3994a254e2126d239f2fe69e829fd95d16ce81dea",
"size": 528,
"platform": {
"architecture": "s390x",
"os": "linux"
}
},
"Available": false,
"Size": {
"Content": 0,
"Total": 0
},
"Kind": "image",
"ImageData": {
"Platform": {
"architecture": "s390x",
"os": "linux"
},
"Size": {
"Unpacked": 0
},
"Containers": null
}
}
]
}
]That data is "per request", but it's possible to inspect multiple images; that results in multiple requests, but the output on the client side will be a combination of all of those; docker tag alpine blpine
docker image inspect alpine blpineSome of that for sure is a limitation on the client-side; dumping the JSON output will always be verbose, and not very user-friendly (we need to start working on the "human friendly" versions of So I'm wondering if we need to make these "opt-in" to have more granularity;
We also need to look at a
|
We're already traversing all of these (to pick the "best representative" variant and calculate the size), so the only performance impact here is just storing the data in the JSON.
IMO, opt-in for
Also,
And this is the direction I'd like to focus next - we really need that (and not just for images, but all "objects" in general). That's where we can have all the granularity we want and provide a better human-readable output, while |
| if err != nil { | ||
| log.G(ctx).WithField("name", name).WithError(err).Error("failed to parse image name as reference") | ||
| // Include the malformed name in RepoTags to be consistent with `docker image ls`. | ||
| repoTags = append(repoTags, img.Name) |
There was a problem hiding this comment.
Pretty unrelated; I thought for a bit we still had a bug to fix around this;
But we fixed that "in post";
Wondering now if we should look at handling de-duplicating here (in a follow-up); i.e., if it would be more efficient to do it right at the start, WDYT?
There was a problem hiding this comment.
Right, let me update
There was a problem hiding this comment.
It's fine for a follow up as it's not directly related! Was reviewing this PR, and I think (almost done) it looks good otherwise
There was a problem hiding this comment.
Was wondering about the performance difference between collecting them to map & then copying keys to slices between collecting to slice and sorting it afterwards.
Results:
BenchmarkCollectRepoTagsAndDigests
BenchmarkCollectRepoTagsAndDigests/slice
BenchmarkCollectRepoTagsAndDigests/slice-8 627352 18742 ns/op 5644 B/op 118 allocs/op
BenchmarkCollectRepoTagsAndDigests/map
BenchmarkCollectRepoTagsAndDigests/map-8 635817 18702 ns/op 5397 B/op 116 allocs/op
PASS
Benchmark code
package containerd
import (
"context"
"testing"
c8dimages "github.com/containerd/containerd/v2/core/images"
"github.com/containerd/log"
"github.com/distribution/reference"
"github.com/docker/docker/internal/sliceutil"
ocispec "github.com/opencontainers/image-spec/specs-go/v1"
)
func BenchmarkCollectRepoTagsAndDigests(b *testing.B) {
ctx := context.Background()
tagged := []c8dimages.Image{
{Name: "docker.io/library/alpine:latest", Target: ocispec.Descriptor{Digest: "sha256:20e6ec8bc89f4bd24469283fe6c147cfed1ad837e435386ed050d8c57c58eb91"}},
{Name: "docker.io/library/alpine:3.14", Target: ocispec.Descriptor{Digest: "sha256:30e6ec8bc89f4bd24469283fe6c147cfed1ad837e435386ed050d8c57c58eb92"}},
{Name: "docker.io/library/alpine:3.13", Target: ocispec.Descriptor{Digest: "sha256:40e6ec8bc89f4bd24469283fe6c147cfed1ad837e435386ed050d8c57c58eb93"}},
{Name: "docker.io/library/alpine:3.13", Target: ocispec.Descriptor{Digest: "sha256:40e6ec8bc89f4bd24469283fe6c147cfed1ad837e435386ed050d8c57c58eb93"}},
{Name: "docker.io/library/alpine@sha256:20e6ec8bc89f4bd24469283fe6c147cfed1ad837e435386ed050d8c57c58eb91", Target: ocispec.Descriptor{Digest: "sha256:20e6ec8bc89f4bd24469283fe6c147cfed1ad837e435386ed050d8c57c58eb91"}},
{Name: "docker.io/library/alpine@sha256:21a3deaa0d32a8057914f36584b5288d2e5ecc984380bc0118285c70fa8c9300", Target: ocispec.Descriptor{Digest: "sha256:21a3deaa0d32a8057914f36584b5288d2e5ecc984380bc0118285c70fa8c9300"}},
}
b.Run("slice", func(b *testing.B) {
b.ResetTimer()
for i := 0; i < b.N; i++ {
collectRepoTagsAndDigests(ctx, tagged)
}
b.ReportAllocs()
})
b.Run("map", func(b *testing.B) {
b.ResetTimer()
for i := 0; i < b.N; i++ {
collectRepoTagsAndDigestsWithMap(ctx, tagged)
}
b.ReportAllocs()
})
}
func collectRepoTagsAndDigests(ctx context.Context, tagged []c8dimages.Image) (repoTags []string, repoDigests []string) {
repoTags = make([]string, 0, len(tagged))
repoDigests = make([]string, 0, len(tagged))
for _, img := range tagged {
if isDanglingImage(img) {
if len(tagged) > 1 {
// This is unexpected - dangling image should be deleted
// as soon as another image with the same target is created.
// Log a warning, but don't error out the whole operation.
log.G(ctx).WithField("refs", tagged).Warn("multiple images have the same target, but one of them is still dangling")
}
continue
}
name, err := reference.ParseNamed(img.Name)
if err != nil {
log.G(ctx).WithField("name", name).WithError(err).Error("failed to parse image name as reference")
// Include the malformed name in RepoTags to be consistent with `docker image ls`.
repoTags = append(repoTags, img.Name)
continue
}
repoTags = append(repoTags, reference.FamiliarString(name))
if _, ok := name.(reference.Digested); ok {
repoDigests = append(repoDigests, reference.FamiliarString(name))
// Image name is a digested reference already, so no need to create a digested reference.
continue
}
digested, err := reference.WithDigest(reference.TrimNamed(name), img.Target.Digest)
if err != nil {
// This could only happen if digest is invalid, but considering that
// we get it from the Descriptor it's highly unlikely.
// Log error just in case.
log.G(ctx).WithError(err).Error("failed to create digested reference")
continue
}
repoDigests = append(repoDigests, reference.FamiliarString(digested))
}
return sliceutil.Dedup(repoTags), sliceutil.Dedup(repoDigests)
}
// collectRepoTagsAndDigestsWithMap returns repoTags and repoDigests for images with the same target using maps for deduplication.
func collectRepoTagsAndDigestsWithMap(ctx context.Context, tagged []c8dimages.Image) (repoTags []string, repoDigests []string) {
repoTagsMap := make(map[string]struct{})
repoDigestsMap := make(map[string]struct{})
for _, img := range tagged {
if isDanglingImage(img) {
if len(tagged) > 1 {
// This is unexpected - dangling image should be deleted
// as soon as another image with the same target is created.
// Log a warning, but don't error out the whole operation.
log.G(ctx).WithField("refs", tagged).Warn("multiple images have the same target, but one of them is still dangling")
}
continue
}
name, err := reference.ParseNamed(img.Name)
if err != nil {
log.G(ctx).WithField("name", name).WithError(err).Error("failed to parse image name as reference")
// Include the malformed name in RepoTags to be consistent with `docker image ls`.
repoTagsMap[img.Name] = struct{}{}
continue
}
repoTagsMap[reference.FamiliarString(name)] = struct{}{}
if _, ok := name.(reference.Digested); ok {
repoDigestsMap[reference.FamiliarString(name)] = struct{}{}
// Image name is a digested reference already, so no need to create a digested reference.
continue
}
digested, err := reference.WithDigest(reference.TrimNamed(name), img.Target.Digest)
if err != nil {
// This could only happen if digest is invalid, but considering that
// we get it from the Descriptor it's highly unlikely.
// Log error just in case.
log.G(ctx).WithError(err).Error("failed to create digested reference")
continue
}
repoDigestsMap[reference.FamiliarString(digested)] = struct{}{}
}
repoTags = make([]string, 0, len(repoTagsMap))
repoDigests = make([]string, 0, len(repoDigestsMap))
// Convert maps to slices
for tag := range repoTagsMap {
repoTags = append(repoTags, tag)
}
for digest := range repoDigestsMap {
repoDigests = append(repoDigests, digest)
}
return repoTags, repoDigests
}So not a big difference, slice is slightly faster but does more a bit more allocs than the map.
I'll go with map.
thaJeztah
left a comment
There was a problem hiding this comment.
LGTM
one question on the "raw" custom reader passed, but maybe I'm thinking wrong there (otherwise, we could consider un-exporting it in the meantime)
client/image_inspect.go
Outdated
| type imageInspectClientOpt struct { | ||
| raw *bytes.Buffer | ||
| apiOptions image.InspectOptions | ||
| } | ||
|
|
||
| type ImageInspectOpt func(*imageInspectClientOpt) |
There was a problem hiding this comment.
This part is OK for now, and I like the idea of non-exported fields for "client" and "api" options for clearer separation.
The only thing here that I want to look into a bit (not for this PR) is what design pattern is most logical; the confusing bit here is that ImageInspectOpt is an exported type, but it includes a non-exported type in its signature. That means that no implementations can exist outside of those implemented in this package (which currently is fine?).
Mostly wondering if the non-exported type in its signature could be confusing, and it it would make sense to (at some point) export the imageInspectClientOpt type itself, but keep its fields (those that we don't want to be set through other means than functional arguments) un-exported.
LOL; hope I'm making sense here!
There was a problem hiding this comment.
Yeah so actually I wanted to go one step further and make the ImageInspectOpt itself unexported, because with all imageInspectClientOpt fields being private, there's no benefit for the user to be able to construct its own opt func.
That didn't work though, because linter was complaining about the exported functions using the unexported type.
Alternatively.. we could drop the func opts and just have ImageInspectOpt that would embed image.InspectOptions and exported Raw.
There was a problem hiding this comment.
Added a commit with alternative approach: 51bfa52
Thoughts? @thaJeztah @laurazard
There was a problem hiding this comment.
Buildkit uses interfaces for option types which makes it much easier to discover what can go there.
There was a problem hiding this comment.
So it could be something like
type ImageOption interface {
SetImageOption(*ImageOpt)
}Then you can do something like
type ImageOptFunc func(*ImageOpt)
func(f ImageOptFunc) SetImageOption(o *ImageOpt) {
f(o)
}There was a problem hiding this comment.
Is this what you had in mind 6d852cc?
Tbh, I don't see how it makes the options more discoverable, but maybe it's a matter of the IDE, so I don't mind it either. (or I just got your suggestion wrong 😅)
There was a problem hiding this comment.
(Ugh, I thought I posted this comment earlier, but looks like I didn't)
I'm also slightly confused; can you explain more?
Looking at these minimal examples;
Details
package godoc
type ImageOpt struct {
something, otherThing string
feature bool
}
type ImageOption func(*ImageOpt)
func WithSomething(value string) ImageOption {
return func(opt *ImageOpt) {
opt.something = value
}
}
func WithOtherThing(value string) ImageOption {
return func(opt *ImageOpt) {
opt.otherThing = value
}
}
func WithEnableFeature() ImageOption {
return func(opt *ImageOpt) {
opt.feature = true
}
}Details
package godoc2
type ImageOpt struct {
something, otherThing string
feature bool
}
type ImageOption interface {
Apply(*ImageOpt)
}
type ImageOptFunc func(*ImageOpt)
func (f ImageOptFunc) Apply(o *ImageOpt) {
f(o)
}
func WithSomething(value string) ImageOption {
return ImageOptFunc(func(opt *ImageOpt) {
opt.something = value
})
}
func WithOtherThing(value string) ImageOption {
return ImageOptFunc(func(opt *ImageOpt) {
opt.otherThing = value
})
}
func WithEnableFeature() ImageOption {
return ImageOptFunc(func(opt *ImageOpt) {
opt.feature = true
})
}Then GoDoc at least shows all options under the ImageInspectOpt type, but I think both do that the same?
One exception would be, when changing any of those to return func(*ImageOpt) (instead of explictly ImageOption), which would still satisfy the type, but won't show it as such in GoDoc, and I think that would be the same for both?
f756c6f to
51bfa52
Compare
be68fa9 to
350370b
Compare
350370b to
e80050e
Compare
|
Rebased |
ad4b050 to
afaf6e3
Compare
thaJeztah
left a comment
There was a problem hiding this comment.
couple of small comments, but LGTM otherwise
perhaps pending the discussion on the options signature, but I think we can still changes that one as the way they're used won't change.
afaf6e3 to
536ec57
Compare
|
oh! probably new tests were added in the "mount from image" PR; |
Signed-off-by: Paweł Gronowski <[email protected]>
Don't use the `GetImage` call which returns a "best-effort" view of the image that is compatible with the old images.Image response. Instead, use the multi-platform view of the image to construct the inspect response. Signed-off-by: Paweł Gronowski <[email protected]>
Deprecate ImageInspectWithRaw and add a simpler ImageInspect function which takes optional options. Signed-off-by: Paweł Gronowski <[email protected]>
Add `Manifests` field to image inspect (`/images/{name}/json`) response.
This is the same as in `/images/json`.
Signed-off-by: Paweł Gronowski <[email protected]>
Signed-off-by: Paweł Gronowski <[email protected]>
536ec57 to
a096045
Compare
|
Right! Fixed |
|
Green, I'm going to get this one in. We can still change the interface before the v28 release if we need. |
Totalsize calculation #48330Add
Manifestsfield to image inspect (/images/{name}/json) response.This is the same as in
/images/json.- How to verify it
- Description for the changelog
- A picture of a cute animal (not mandatory but encouraged)