Skip to content

Generate GRPC contracts for runtime APIs#7979

Merged
mxpv merged 3 commits intocontainerd:mainfrom
mxpv:grpc
Feb 2, 2023
Merged

Generate GRPC contracts for runtime APIs#7979
mxpv merged 3 commits intocontainerd:mainfrom
mxpv:grpc

Conversation

@mxpv
Copy link
Copy Markdown
Member

@mxpv mxpv commented Jan 19, 2023

We'd like to introduce GRPC based shims as experimental features in 1.7.
There were a few attempts before, original issue and motivation discussed here: #2399
Primarily this aims to unblock non-Go/Rust shims implementations.

This PR is step 1 - it adds generated GRPC code to addition to TTRPC.
It shamelessly steals Derek's approach from #7609 to generate TTRPC and GRPC code in one package.

For sandbox API we just generate both TTRPC and GRPC code in one package (TTRPC based contracts now get TTRPC prefix). We can do that because we don't care about backward compatibility as we haven't introduced this API publicly.

For Task service, in order to keep backward compatibility in 1.7, I add another temporary package task_grpc next to task. It reuses structures, but dups service proto. We'll consolidate these in one package in 2.0.

For task service, to keep backward compatibility, this PR introduces task/v3, which includes TTRPC contracts (with TTRPC prefix) along with GRPC code.

@mxpv
Copy link
Copy Markdown
Member Author

mxpv commented Jan 19, 2023

/test pull-containerd-sandboxed-node-e2e

Copy link
Copy Markdown
Member

@kzys kzys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow. It works without Protobuild changes 😎

@fuweid
Copy link
Copy Markdown
Member

fuweid commented Jan 28, 2023

Just curiosity about task/v3 memory cost. I remember that ttrpc is used to reduce memory usage in shim level. So is it going to use grpc in runc shim in the following release? Thanks

@dmcgowan
Copy link
Copy Markdown
Member

@fuweid This shouldn't change anything about existing shims. This doesn't have any client changes yet, but I would think we need a way to either negotiate the protocol or configure it. It could be easy to negotiate just through file naming, socket name or shim naming scheme.

@fuweid
Copy link
Copy Markdown
Member

fuweid commented Jan 28, 2023

This shouldn't change anything about existing shims. This doesn't have any client changes yet,

It is kind of API-level changes.
Want to see next steps because I have concerns about the memory cost and maintain efforts.

but I would think we need a way to either negotiate the protocol or configure it.

Yes. Before migrate into task/v3, I think we should provide the memory cost report on this.
IMO, memory spike is kind of regression. :)

@mxpv mxpv force-pushed the grpc branch 3 times, most recently from 62ecad0 to e3e8dbc Compare February 2, 2023 17:54
@mxpv
Copy link
Copy Markdown
Member Author

mxpv commented Feb 2, 2023

/test pull-containerd-sandboxed-node-e2e

@mxpv mxpv merged commit 3d32da8 into containerd:main Feb 2, 2023
@mxpv mxpv deleted the grpc branch February 2, 2023 19:49
@Burning1020
Copy link
Copy Markdown
Member

@mxpv I'm just curious that what kind of the non-Go/Rust shims is. 😂

@mxpv
Copy link
Copy Markdown
Member Author

mxpv commented Feb 16, 2023

@Burning1020 as an example of such use case: On MacOS you'd want to use ObjC or Swift to interact with OS frameworks (like Virtualization framework). Both Go and Rust would need some sort of wrapper to be maintained.

@ningmingxiao
Copy link
Copy Markdown
Contributor

ningmingxiao commented Apr 16, 2026

how about create a new plugin in containerd,the containerd plugin trans grpc to ttrpc to connect shim (if shim use ttrpc )we don't need let goshim support grpc server, I afraid it use more memory. @mxpv

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

8 participants