Skip to content

distribution: match manifest list resolution with containerd#43675

Merged
thaJeztah merged 1 commit into
moby:masterfrom
tonistiigi:make-pull-match-containerd
Jun 2, 2022
Merged

distribution: match manifest list resolution with containerd#43675
thaJeztah merged 1 commit into
moby:masterfrom
tonistiigi:make-pull-match-containerd

Conversation

@tonistiigi
Copy link
Copy Markdown
Member

There are slight differences on how Docker and containerd find a specific image config/rootfs for running a container from a multi-platform image index.

Containerd logic for these edge cases seems to be according to spec https://github.com/opencontainers/image-spec although some cases are a bit vague there.

This PR matches Docker logic with what containerd is doing. With containerd storage integration, the Moby implementation will be replaced with containerd's anyway in the next release and this would be the future behavior. With this PR we can move to correct logic sooner and avoid behavior changes with containerd integration.

There is also a discussion going on at OCI Reference Types working group opencontainers/wg-reference-types#50 that may choose to use nested Image indexes as a way to attach signatures to images. As nested indexes are currently handled differently in Moby we should get to matching implementation sooner to avoid breakage.

Changes:

  • Manifest list is allowed to contain manifest lists.

Also, implementations SHOULD support the following media types:
application/vnd.oci.image.index.v1+json (nested index)
https://github.com/opencontainers/image-spec/blob/main/image-index.md

  • Unknown mediatype inside manifest list is skipped instead of causing an error

An encountered mediaType that is unknown to the implementation MUST be ignored.
https://github.com/opencontainers/image-spec/blob/main/image-index.md

  • Platform in descriptor is optional

This OPTIONAL property describes the minimum runtime requirements of the image.
https://github.com/opencontainers/image-spec/blob/main/image-index.md

The algorithm for finding the best match is as follows:

  • From the image index all descriptors matching the minimum platform requirements are filtered out
  • Matched descriptors are sorted from most specific to least specific (arm/v7 is more specific than arm/v6 is more specific than "")
  • If matched descriptor points to the image manifest it is returned
  • If it points to an unknown object it is skipped
  • If it points to another image index then the same algorithm is performed there. If no matches were in the inner manifest list then the descriptor is skipped.

OCI mediatypes and Docker mediatypes work the same way. Obviously, the specs are slightly different but the intention in here is to replicate containerd's behavior not to redefine new behavior for the loosely defined areas.

Signed-off-by: Tonis Tiigi [email protected]

Comment thread distribution/pull_v2.go
id, _, err = p.pullSchema1(ctx, manifestRef, v, &platform)
if err != nil {
for _, match := range manifestMatches {
if err := checkImageCompatibility(match.Platform.OS, match.Platform.OSVersion); err != nil {
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a windows-only check. Logically it should probably also just continue rather than error. I left it as-is for now as it seems to be a very specific old case that is intentionally set to error.

Comment thread distribution/pull_v2.go Outdated
Comment thread distribution/pull_v2.go
platform specs.Platform
}

func (e noMatchesErr) Error() string {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For a follow-up, we should check what status code this produces at the API level (and if needed, use a suitable errdef

Copy link
Copy Markdown
Member

@thaJeztah thaJeztah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Copy Markdown
Member

@cpuguy83 cpuguy83 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Copy Markdown
Member

@cpuguy83 cpuguy83 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

distribution/pull_v2.go:1068:66: packge is a misspelling of package (misspell)

Make finding the correct runtime image from image index
more compliant with OCI spec and match containerd implementation.

Changes:
- Manifest list is allowed to contain manifest lists
- Unknown mediatype inside manifest list is skipped instead of causing an error
- Platform in descriptor is optional 

Signed-off-by: Tonis Tiigi <[email protected]>
@tonistiigi tonistiigi force-pushed the make-pull-match-containerd branch from 0c15e10 to 9adad26 Compare June 2, 2022 18:21
@thaJeztah
Copy link
Copy Markdown
Member

All green; only failure is a known flaky test TestNetworkDBCRUDTableEntries (#42739);

=== FAIL: github.com/docker/docker/libnetwork/networkdb TestNetworkDBCRUDTableEntries (7.62s)
2022/06/02 18:34:34 Closing DB instances...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants