Skip to content

Make docker pull detect plugin content and error out.#25582

Merged
aaronlehmann merged 1 commit intomoby:masterfrom
anusha-ragunathan:detect-plugin-mediatype
Aug 22, 2016
Merged

Make docker pull detect plugin content and error out.#25582
aaronlehmann merged 1 commit intomoby:masterfrom
anusha-ragunathan:detect-plugin-mediatype

Conversation

@anusha-ragunathan
Copy link
Copy Markdown
Contributor

@anusha-ragunathan anusha-ragunathan commented Aug 10, 2016

- What I did
Make docker pull detect plugin mediatype and error out.
In this process, also add the plugin media type to distribution package.

- How I did it
Detect using manifest config's mediatype.

- How to verify it
docker pull tiborvass/no-remove fails; before this change it panics.

Signed-off-by: Anusha Ragunathan [email protected]

Fixes #23752

@anusha-ragunathan anusha-ragunathan added this to the 1.13.0 milestone Aug 10, 2016
@thaJeztah thaJeztah added status/2-code-review status/failing-ci Indicates that the PR in its current state fails the test suite and removed status/0-triage labels Aug 10, 2016
@thaJeztah
Copy link
Copy Markdown
Member

Got a golint failure;

19:16:12 ---> Making bundle: validate-lint (in bundles/1.13.0-dev/validate-lint)
19:16:12 Errors from golint:
19:16:12 plugin/distribution/types.go:13:7: exported const DefaultTag should have comment or be unexpected

Comment thread distribution/pull_v2.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bit unsure about "cannot be pulled in this manner", perhaps

invalid manifest type for image. Use `docker plugin install`

but not really satisfied with that one @sfsmithcha perhaps you have a suggestion?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about

target is a plugin manifest. Use `docker plugin install` to install plugins.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, that looks good, thanks

@thaJeztah thaJeztah removed the status/failing-ci Indicates that the PR in its current state fails the test suite label Aug 10, 2016
@thaJeztah
Copy link
Copy Markdown
Member

Tested this and it resolves the issue I saw, so LGTM (pending vendoring, and possibly better suggestions for the error)

Comment thread distribution/pull_v2.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if m, ok := manifest.(*schema2.DeserializedManifest); ok { m.Manifest.Config.MediaType } ?

@tonistiigi
Copy link
Copy Markdown
Member

before this change it panics.

We should add some general purpose protection for this as well. Thankfully it doesn't panic the daemon but only the HTTP request. But we should consider anything returned by registry as potentially malicious and always validate it.

@anusha-ragunathan
Copy link
Copy Markdown
Contributor Author

Vendoring change at distribution/distribution#1901

@anusha-ragunathan anusha-ragunathan modified the milestones: 1.12.1, 1.13.0 Aug 15, 2016
Comment thread distribution/pull_v2.go Outdated
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe only schema2.MediaTypeConfig is expected value here. Wondering if we could just throw an error of an expected configuration type if that is not seen. Ping @aaronlehmann

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the plugin stuff extends this with an additional media type?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does, but it has it's own pull method. It is invalid to pull a plugin type here. My preference for the code would be rather than having to have this part of the code know about plugins, just have it check for the configuration types it expects, in this case schema2.MediaTypeConfig

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, SGTM

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried this approach before blacklisting plugins. The problem is that docker pull official Hub images such busybox, python, ubuntu has mediatype set to "application/octet-stream" and not the expected schema2.MediaTypeConfig ("application/vnd.docker.container.image.v1+json").

distribution/distribution#1621 seems relevant.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That we know about. At some point we will have to be restrictive in the supported values. If it fails then we can assess whether it is legitimate and we should make a code change or whether the manifest needs to be recreated. We accept the application/octet-stream because we know there were many manifests mistakenly created with it and that at the time when they were created the only supported value is application/vnd.docker.container.image.v1+json.

For this change in particular, since now there are multiple possible types, I think it makes sense to define what is supported and have a special case for that one distribution bug.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@anusha-ragunathan to me it more about separation of the code. This code should not be pulling plugins, and it really doesn't even need to know about anything that is a non-image. Having the whitelist would also negate the need of having the v0 version defined in distribution since the code change can be isolated to the image pull logic.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can make these sort of updates later, I am fine with it like this fo rnow

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dmcgowan: distribution knowing about the plugin mediatype should be okay. Are you concerned purely because it is in v0 and will be okay to include it in distribution once we have a stable v1 manifest spec? Also, I agree that code separation is ideal but the above workaround to detecting the container image mediatype feels hackish. What if there are other mediatypes that we are not aware of, but qualify as a good container image today. Then this change will break a lot of users. That's why blacklisting is safe, since it wont break a thing.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to add something in distribution to handle this sort of filtering and offering a "media type" separate from the version, so we can test to see if the media type is indeed an image rather than something else. I will create an issue in distribution to track this.

@thaJeztah
Copy link
Copy Markdown
Member

ping @aaronlehmann @dmcgowan what's the status here?

@dmcgowan
Copy link
Copy Markdown
Member

Upstream merged

@tiborvass
Copy link
Copy Markdown
Contributor

@dmcgowan My ¢0.02, i don't quite see why the plugin media type has to be in the distribution repo. It should be orthogonal.

@dmcgowan
Copy link
Copy Markdown
Member

dmcgowan commented Aug 16, 2016

@tiborvass I agree, but we would like to have the definition for all the known media types for the schema2 manifest in the schema2 package. Didn't have to be done right now but it should be followed up with the documentation next to the other schema2 documentation. The distribution package certainly doesn't have to be the gatekeeper for these media types but we would like them to flow back in once going out in a release.

@tiborvass tiborvass removed this from the 1.12.1 milestone Aug 18, 2016
@anusha-ragunathan
Copy link
Copy Markdown
Contributor Author

Distribution vendor at #25841

@anusha-ragunathan
Copy link
Copy Markdown
Contributor Author

@dmcgowan Distribution changes are vendored in. PTAL.

@dmcgowan
Copy link
Copy Markdown
Member

LGTM

@thaJeztah
Copy link
Copy Markdown
Member

re-LGTM, ping @aaronlehmann @tiborvass

Comment thread plugin/distribution/types.go Outdated
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm surprised golint allowed these not to be commented individually. golint generally requires exported constants to have comments. Are we somehow running golint incorrectly?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

golint allows to put one comment on top of const if we want to be lazy (and if there is no comment on any constant).

@aaronlehmann
Copy link
Copy Markdown

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants