Support runtime specific configurations. #943

Random-Liu · 2018-10-08T21:07:45Z

Support per-runtime configuration in the cri config file. The newly introduced Options field can be any type. Currently, only supported types are options.Options for shim v2, and runctypes.RuncOptions. runctypes.RuncOptions is the fallback for unknown runtime types.

Change getRuntimeType to support new option types.

Here is an example config which uses runc with shim v1 as the default runtime, runc with shim v2 as the runc runtime, and kata as the kata runtime:

[plugins.cri.containerd.default_runtime]                                                                                           
  runtime_type = "io.containerd.runtime.v1.linux"
# https://github.com/containerd/containerd/blob/v1.2.0-rc.1/runtime/linux/runctypes/runc.pb.go#L40
[plugins.cri.containerd.default_runtime.options]
  Runtime = "runc"
  RuntimeRoot = "/run/containerd/default"

[plugins.cri.containerd.runtimes.runc]
  runtime_type = "io.containerd.runc.v1"
# https://github.com/containerd/containerd/blob/v1.2.0-rc.1/runtime/v2/runc/options/oci.pb.go#L39
[plugins.cri.containerd.runtimes.runc.options]
  BinaryName = "runc"
  Root = "/run/containerd/runc"

[plugins.cri.containerd.runtimes.kata]
  runtime_type = "io.containerd.runtime.kata.v2"

Note that the deprecated daemon level config systemd_cgroup, no_pivot, and runtime level config runtime_engine, runtime_root still work for the legacy runtime io.containerd.runtime.v1.linux. However, 1) they won't work for shim v2 runtimes; 2) they'll be deprecated when shim v1 is deprecated.

It is recommended to use the new Options field for all runtimes.

/cc @yujuhong @crosbymichael @tallclair
Signed-off-by: Lantao Liu [email protected]

k8s-ci-robot · 2018-10-08T21:07:47Z

@Random-Liu: GitHub didn't allow me to request PR reviews from the following users: tallclair.

Note that only containerd members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

Support per-runtime configuration in cri config file.

The newly introduced Options field can accept any types.

Here is an example config which uses runc with shim v1 as the default runtime, runc with shim v2 as the runc runtime, and kata as the kata runtime:
[plugins.cri.containerd.default_runtime]                                                                                           
 runtime_type = "io.containerd.runtime.v1.linux"
[plugins.cri.containerd.default_runtime.options]
 Runtime = "runc"
 RuntimeRoot = "/run/containerd/default"

[plugins.cri.containerd.runtimes.runc]
 runtime_type = "io.containerd.runc.v1"
[plugins.cri.containerd.runtimes.runc.options]
 BinaryName = "runc"
 Root = "/run/containerd/runc"

[plugins.cri.containerd.runtimes.kata]
 runtime_type = "io.containerd.runtime.kata.v2"
Note that the deprecated daemon level config systemd_cgroup, no_pivot, and runtime level config runtime_engine, runtime_root still work for the legacy runtime io.containerd.runtime.v1.linux. However, 1) they won't work for shim v2 runtimes; 2) they'll be deprecated when shim v1 is deprecated.

It is recommended to use the new Options field for all runtimes.

/cc @yujuhong @crosbymichael @tallclair
Signed-off-by: Lantao Liu [email protected]

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Random-Liu · 2018-10-08T22:15:24Z

/test pull-cri-containerd-verify

mikebrow

See comments. Other than what to do with the docs for new fields the code looks very good!

mikebrow · 2018-10-08T23:11:34Z

docs/config.md

+      # "io.containerd.runc.v1". See:
+      #   https://github.com/containerd/containerd/blob/v1.2.0-rc.1/runtime/v2/runc/options/oci.pb.go#L39.
+      [plugins.cri.containerd.runtimes.runc.options]
+        NoPivotRoot = false


let's go ahead and document these here. or generate a runc_options.md file and link to that

also we're missing:

// create a new keyring for the container NoNewKeyring bool // place the shim in a cgroup ShimCgroup string // set the I/O's pipes uid IoUid uint32 // set the I/O's pipes gid IoGid uint32 // criu binary path CriuPath string

We should list them and if necessary say that they are not supported, yet, and link to an issue for tracking. or you could even have a section for fields not supported. If not supported we need to test for and flag as an error so the user does not try them out. This because the fields are actually there.

All of them are supported now actually. We just pass through whatever configurations from the config file. That is why I just linked to the code, because everything is supported there. I can add that "all fields" are supported.

Granted these .md's could use some more content but probably should document the fields here vs say go read the code :-)

Will document them then. :)

criu supported ? FYI that's checkpoint and restore..

The option is supported, means that you can config it. We are just not using it - there is no code path that checkpoint or restore will be called.

We now support whatever configurations supported by the runtime, we directly pass through the config, and don't do anything special in the cri plugin.

For example, if we have a KataOptions type, and contains all kinds of VM specific options, we can support it now with shim v2.

A little surprised but ok, new meta! :-)

Yeah, our goal is to support multiple runtimes now. If not necessary, we may not want to add runtime specific logic, just pass through. :)

mikebrow · 2018-10-08T23:23:49Z

pkg/server/helpers.go

+	if err := toml.PrimitiveDecode(*r.Options, options); err != nil {
+		return nil, err
+	}
+	return options, nil


I suppose if we're not supporting those fancy new options we'd fail them here if set.

ok... fail not needed, pass through is the new meta :-)

Signed-off-by: Lantao Liu <[email protected]>

mikebrow

/LGTM

Cherrypick #943 to release/1.2

k8s-ci-robot requested review from yujuhong and crosbymichael October 8, 2018 21:07

Random-Liu assigned yujuhong and mikebrow Oct 8, 2018

k8s-ci-robot added the size/L label Oct 8, 2018

Random-Liu added this to the v1.2 milestone Oct 8, 2018

mikebrow reviewed Oct 8, 2018

View reviewed changes

Random-Liu force-pushed the support-per-runtime-config branch 4 times, most recently from 2b9c6c7 to 2c2aacc Compare October 9, 2018 00:15

Support runtime specific configurations.

1442425

Signed-off-by: Lantao Liu <[email protected]>

Random-Liu force-pushed the support-per-runtime-config branch from 2c2aacc to 1442425 Compare October 9, 2018 00:17

Random-Liu mentioned this pull request Oct 9, 2018

Cleanup after shim v1 is deprecated. #946

Closed

mikebrow approved these changes Oct 9, 2018

View reviewed changes

k8s-ci-robot added the lgtm label Oct 9, 2018

Random-Liu merged commit 70da14e into containerd:master Oct 9, 2018

Random-Liu deleted the support-per-runtime-config branch October 9, 2018 15:09

Random-Liu mentioned this pull request Oct 9, 2018

"io.containerd.runc.v1" is not usable via CRI? containerd/containerd#2701

Closed

Random-Liu added a commit that referenced this pull request Oct 9, 2018

Merge pull request #947 from Random-Liu/cherrypick-#943-release-1.2

4f939fc

Cherrypick #943 to release/1.2

mikebrow mentioned this pull request Nov 9, 2018

support DisableCgroup, DisableApparmor, RestrictOOMScoreAdj #970

Merged

Random-Liu mentioned this pull request Jan 15, 2019

Add config_path proto options for shim v2 containerd/containerd#2916

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support runtime specific configurations. #943

Support runtime specific configurations. #943

Random-Liu commented Oct 8, 2018 •

edited

Loading

k8s-ci-robot commented Oct 8, 2018

Random-Liu commented Oct 8, 2018

mikebrow left a comment

mikebrow Oct 8, 2018

mikebrow Oct 8, 2018

Random-Liu Oct 8, 2018 •

edited

Loading

mikebrow Oct 8, 2018

Random-Liu Oct 8, 2018

mikebrow Oct 8, 2018

Random-Liu Oct 9, 2018 •

edited

Loading

Random-Liu Oct 9, 2018 •

edited

Loading

mikebrow Oct 9, 2018 •

edited

Loading

Random-Liu Oct 9, 2018 •

edited

Loading

mikebrow Oct 8, 2018

mikebrow Oct 9, 2018 •

edited

Loading

mikebrow left a comment

Support runtime specific configurations. #943

Support runtime specific configurations. #943

Conversation

Random-Liu commented Oct 8, 2018 • edited Loading

k8s-ci-robot commented Oct 8, 2018

Random-Liu commented Oct 8, 2018

mikebrow left a comment

Choose a reason for hiding this comment

mikebrow Oct 8, 2018

Choose a reason for hiding this comment

mikebrow Oct 8, 2018

Choose a reason for hiding this comment

Random-Liu Oct 8, 2018 • edited Loading

Choose a reason for hiding this comment

mikebrow Oct 8, 2018

Choose a reason for hiding this comment

Random-Liu Oct 8, 2018

Choose a reason for hiding this comment

mikebrow Oct 8, 2018

Choose a reason for hiding this comment

Random-Liu Oct 9, 2018 • edited Loading

Choose a reason for hiding this comment

Random-Liu Oct 9, 2018 • edited Loading

Choose a reason for hiding this comment

mikebrow Oct 9, 2018 • edited Loading

Choose a reason for hiding this comment

Random-Liu Oct 9, 2018 • edited Loading

Choose a reason for hiding this comment

mikebrow Oct 8, 2018

Choose a reason for hiding this comment

mikebrow Oct 9, 2018 • edited Loading

Choose a reason for hiding this comment

mikebrow left a comment

Choose a reason for hiding this comment

Random-Liu commented Oct 8, 2018 •

edited

Loading

Random-Liu Oct 8, 2018 •

edited

Loading

Random-Liu Oct 9, 2018 •

edited

Loading

Random-Liu Oct 9, 2018 •

edited

Loading

mikebrow Oct 9, 2018 •

edited

Loading

Random-Liu Oct 9, 2018 •

edited

Loading

mikebrow Oct 9, 2018 •

edited

Loading