Could you add example values

@AkihiroSuda Ok, different case maybe need different extraoptions, take nydus as an example, nydus-snapshotter(remote-snapshotter) should delivery nydus image metadata file and nydus daemom configs to kata-shim runtime, so these two items can be add to extraoptions to delivery.

it is about interface change. If the case is more specifically, like demo or something, it will be easy to understand.

@luodw I think it is important to emphasize (in the interface comments) that the ExtraOptions field is opaque to containerd and is ignored by the containerd mount.Mount API. It aims to provide a way for the snapshotter to pass extra information to the container runtime.

An example of such extra information, in the nydus case, can be the nydus image bootstrap file location and the nydusd configuration, etc. It will be put into a predefined structure in a string format and be decoded in kata runtime, where this information is then passed to nydusd for real usage.

@fuweid @AkihiroSuda @ktock Thanks for your comments! Does the approach make sense to you? We don't have an end-to-end demo yet. But we can work on integration first if you prefer to see it working end-to-end wise (which requires modification to nydus-snapshotter, containerd and kata).

@fuweid Thanks, I have run a end-to-end demo in my dev machine successfully, and maybe it is more clear to show the code,

1. This WIP PR is the modification for nydus-snapshotter to use the ExtraOption and pass config information;
2. This WIP PR is the modification for kata-shim runtime to parse the ExtraOption and use it;

How is this used for mounting a filesystem?
If the purpose is configuring the runtime, isn't task-related structs (e.g. TaskInfo) the proper place to fix?

@ktock This field is not used to mount a filesystem like overlayfs, it's main role is used to delivery config information. As remote-snapshotter(take stargz-snapshotter for example) only return []mount.Mount, so if we want to delivery some config information from remote-snapshotter to runtime, adding ExtraOptions field may be a suitable way.

In containerd, it will ignore ExtraOptions and do mount overlayfs as original logic. In runtime, it will parse the ExtraOptions and use it. In nydus case, nydus-snapshotter will add nydus image metedata file path and config to ExtraOptions and delivery to kata-runtime, kata-runtime used these info to request nydusd (fuse/virtiofs daemon) to mount nydus rafs fllesystem.

Wondering if this should be more of a MetaData field (making it more clear it's entirely up to the consumer what to use it for)

@thaJeztah Do you mean renaming ExtraOption []string to MetaData []string? It does make it more clear.

yes, I was thinking along those lines (but of course could use input from others); mostly was reading along, and if I understand it correctly, effectively this information is arbitrary "extra information", that can be used by other systems for <insert purpose here>.

Naming it MetaData (or something like that), makes it clearer that whatever is in that field is not a strict "contract" (from containerd's perspective).

Of course the description of the field should also outline this, and could (perhaps) add some recommendations (e.g. recommend to use a common prefix (io.katacontainers.someopt=foo)) <--- thinking out loud here; I know we did the same recommendation for labels

@thaJeztah Thanks for you advise, I agree that MetaDatas []string is more clear. I have updated the field and it's description, hope for your reivew again.

This should be Metadata (or MetaData) (singular), as data is already plural.

But ok to wait with changing as the discussion hasn't completed yet, so maintainers may have more input on this 😅