Skip to content

[release/1.7] Fix CRI snapshotter root path when not under containerd root#10096

Merged
dmcgowan merged 4 commits intocontainerd:release/1.7from
Kern--:cri-remote-snapshotter-stats
Apr 23, 2024
Merged

[release/1.7] Fix CRI snapshotter root path when not under containerd root#10096
dmcgowan merged 4 commits intocontainerd:release/1.7from
Kern--:cri-remote-snapshotter-stats

Conversation

@Kern--
Copy link
Copy Markdown

@Kern-- Kern-- commented Apr 19, 2024 •
edited
Loading

Fixes #10095 in 1.7

This PR backports 4 changes:

  1. Add platform config to proxy plugins #8417 - Allows proxy plugins to have platforms (not strictly necessary to fix the issue, but I thought it would be less weird to keep this than to take a newer change to ProxyPlugins without the older one)
  2. Add exports to proxy plugin config #9253 - Allows proxy plugins to have exports (e.g. root)
  3. Snapshotters: Export the root path #10073 - Exports root paths for a number of snapshotters that didn't already have them
  4. CRI: "Fix" imageFSPath behavior #9216 - Uses exported root key for snapshotter root if present.

The result of this is that you can tell containerd about a remote snapshotter's root like:

[proxy_plugins.soci]
type = "snapshot"
address = "/run/soci-snapshotter-grpc/soci-snapshotter-grpc.sock"
[proxy_plugins.soci.exports]
root = "/var/lib/soci-snapshotter-grpc"

which will be correctly reported by CRI which unblocks the kubelet from enforcing ephemeral-storage limits and doing image garbage collection.

dmcgowan and others added 4 commits April 17, 2024 18:07
@dmcgowan
Add platform config to proxy plugins
8916e2c 
Signed-off-by: Derek McGowan <[email protected]>
(cherry picked from commit 4e56939)
Signed-off-by: Kern Walster <[email protected]>
@dmcgowan
Add exports to proxy plugin config
8fb6bfa 
Allows external plugins to define exports.

Signed-off-by: Derek McGowan <[email protected]>
(cherry picked from commit e4639ad)
Signed-off-by: Kern Walster <[email protected]>
@dcantah
Snapshotters: Export the root path
bd423bf 
Some of the snapshotters that allow you to change their root location
were already doing this, this just makes all of them follow the same
pattern.

Signed-off-by: Danny Canter <[email protected]>
(cherry picked from commit 32caaee)
Signed-off-by: Kern Walster <[email protected]>
@dcantah
CRI: "Fix" imageFSPath behavior
a8ebceb 
Currently it didn't take into account that certain snapshots can explicitly
have their root directories placed at a different location. This changes
it to use the RootPath method of the snapshotter if it implements it.

Without this change, cadvisor is not able to get filesystem usage
information, which prevents the kubelet from doing image garbage
collection and enforcing ephemeral storage limits.

Signed-off-by: Danny Canter <[email protected]>
(cherry picked from commit 6aeec45)
Signed-off-by: Kern Walster <[email protected]>
@k8s-ci-robot
Copy link
Copy Markdown

k8s-ci-robot commented Apr 19, 2024

Hi @Kern--. Thanks for your PR.

I'm waiting for a containerd member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@Kern-- Kern-- mentioned this pull request Apr 19, 2024
Closed
@ruiwen-zhao
Copy link
Copy Markdown
Member

ruiwen-zhao commented Apr 19, 2024

LGTM

btw just to make sure, this PR doesn't backport #9152, but rather cherrypicks the closed PR #9216.

henry118
henry118 reviewed Apr 19, 2024
View reviewed changes
Comment thread services/server/config/config.go
Comment on lines +169 to +170
Platform string `toml:"platform"`
Exports map[string]string `toml:"exports"`
Copy link
Copy Markdown
Member

@henry118 henry118 Apr 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need these? It's better to keep the existing configs for compatibility.

Copy link
Copy Markdown
Author

@Kern-- Kern-- Apr 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do need Exports, but we don't need Platform.

I kept the Platform change because it was implemented before Exports and so I thought it would be cleaner (for a vague sense of the word "clean") to take both rather than just the newer piece that I needed.

I can remove the Platform backport if needed.

@Kern--
Copy link
Copy Markdown
Author

Kern-- commented Apr 19, 2024

btw just to make sure, this PR doesn't backport #9152, but rather cherrypicks the closed PR #9216.

Yes, that's right. I updated the PR description to reflect that.

@dcantah
Copy link
Copy Markdown
Member

dcantah commented Apr 23, 2024

Thanks!

@dmcgowan dmcgowan merged commit c4a8642 into containerd:release/1.7 Apr 23, 2024
@kiashok kiashok mentioned this pull request Apr 24, 2024
Merged
@Kern-- Kern-- mentioned this pull request Apr 24, 2024
Merged
@dmcgowan dmcgowan added the area/cri Container Runtime Interface (CRI) label Apr 24, 2024
@henry118 henry118 mentioned this pull request Oct 4, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dmcgowan dmcgowan dmcgowan approved these changes

@henry118 henry118 henry118 approved these changes

@ruiwen-zhao ruiwen-zhao ruiwen-zhao approved these changes

@dcantah dcantah dcantah approved these changes

Assignees

No one assigned

Labels

area/cri Container Runtime Interface (CRI) impact/changelog needs-ok-to-test size/M

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@Kern-- @k8s-ci-robot @ruiwen-zhao @dcantah @dmcgowan @henry118