Skip to content

Conversation

@klihub
Copy link
Member

@klihub klihub commented Jan 11, 2023

Split out the NRI integration bits ('nri-api') from pkg/cri/server to pkg/cri/nri, allowing a single implementation to be shared between the server and sbserver variants of the CRI plugin. Rework it to not assume access to unexported internals of server or sbserver. Add code to hook the NRI service plugin and NRI processing into the 'sbserver' CRI plugin variant.

@k8s-ci-robot
Copy link

Hi @klihub. Thanks for your PR.

I'm waiting for a containerd member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@klihub klihub force-pushed the devel/sbserver-nri-integration branch 3 times, most recently from f7377f9 to d0ca371 Compare January 11, 2023 17:31
@klihub klihub force-pushed the devel/sbserver-nri-integration branch 6 times, most recently from 06e4121 to 6de3983 Compare January 17, 2023 07:41
@klihub klihub requested a review from mxpv January 17, 2023 13:29
@klihub klihub force-pushed the devel/sbserver-nri-integration branch from 6de3983 to bc00906 Compare January 18, 2023 19:17
@klihub klihub requested a review from mxpv January 18, 2023 19:18
@klihub klihub force-pushed the devel/sbserver-nri-integration branch from bc00906 to 2bede55 Compare January 18, 2023 20:57
opts = append(opts, containerd.WithSandbox(sandboxID))
}

if c.nri.IsEnabled() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the caller responsible for checking IsEnabled?

nri.API sometimes checks IsEnabled inside its associated functions, sometimes doesn't and we have nri.API's no-op implementation for non-Linux platforms.

Is it possible to use the no-op implementation for this disabled case?

Copy link
Member Author

@klihub klihub Jan 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the caller responsible for checking IsEnabled?

It should not be strictly necessary, as there is a test later in the call chain that checks and short-circuits the call at that level. Here I wanted to avoid scheduling the deferred function for possible failure cleanup and avoid adding the extra option which is unnecessary if NRI is disabled (and now that I look at it that option is ignorant about being disabled and relies on the other infra to properly not do anything then).

nri.API sometimes checks IsEnabled inside its associated functions, sometimes doesn't and we have nri.API's no-op implementation for non-Linux platforms.

Is it possible to use the no-op implementation for this disabled case?

Do you mean something like this ?

  1. change API to be a interface not a struct
  2. have a common no-op implementation of API (without any platform-specific build flags)
  3. have platform-specific real implementations of API
  4. if NRI is disabled instantiate the no-op implementation,
  5. otherwise instantiate the real one
  6. eliminate all/most of the server- and sbserver-level if c.nri.IsEnabled() checks

Or maybe instead of an interface, do the same thing with a 'frontend' struct that would then call to a real or a no-op implementation... but anyway, still mostly the same idea ?

If you think that would be a better approach (could be less intrusive/improve readability), I can give it a try and see what I manage to cook up...

Copy link
Member

@kzys kzys Jan 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to remove IsEnabled checks, but this wouldn't cover defer case...

If NRI check is required for defer, how about using nri != nil instead of IsEnabled? It is not coming from your change though.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we make UndoCreateContainer() to properly become a no-op when NRI is disabled, then it should be fine to unconditionally defer calling it on the error return path. There are a few places where currently we need to do an extra sandbox store lookup for the container's sandbox on the error branch if NRI is enabled. But maybe if we did those lookups from within the called NRI API function then we could make those calls unconditionally, too.

Anyway, I'll check if I can check if I could remove the top-level IsEnabled() checks in some safe way.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kzys I updated this PR, moving the enabled/disabled checks from server and sbserver to the common, split out bits. PTAL.

@klihub klihub force-pushed the devel/sbserver-nri-integration branch from 2bede55 to 389bec4 Compare February 4, 2023 13:57
@klihub
Copy link
Member Author

klihub commented Feb 4, 2023

/retest-required

@klihub klihub requested a review from kzys February 8, 2023 07:16
Split out the criService-agnostic bits of nri-api* from
pkg/cri/server to pkg/cri/nri to allow sharing a single
implementation betwen the server and sbserver versions.
Rework the interfaces to not require access to package
internals.

Signed-off-by: Krisztian Litkey <[email protected]>
Hook the NRI service plugin into CRI sbserver request
processing.

Signed-off-by: Krisztian Litkey <[email protected]>
@klihub klihub force-pushed the devel/sbserver-nri-integration branch from 389bec4 to ebbcb57 Compare February 13, 2023 20:11
Copy link
Member

@mikebrow mikebrow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mikebrow
Copy link
Member

/ok-to-test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

5 participants