Skip to content

[RFC] nspawn: add --share-ipc and --share-uts#2982

Closed
alban wants to merge 1 commit intosystemd:masterfrom
kinvolk:alban/nspawn-fly
Closed

[RFC] nspawn: add --share-ipc and --share-uts#2982
alban wants to merge 1 commit intosystemd:masterfrom
kinvolk:alban/nspawn-fly

Conversation

@alban
Copy link
Member

@alban alban commented Apr 7, 2016

In Kubernetes, containers/pods are started via a json configuration file and the Kubernetes PodSpec has the following options:

  • HostPID bool: Use the host's pid namespace (CLONE_NEWPID)
  • HostIPC bool: Use the host's ipc namespace (CLONE_NEWIPC)
  • HostNetwork bool: Use the host's network namespace (CLONE_NEWNET)

When using the container run-time rkt instead of Docker in Kubernetes, we currently lose the HostPID/HostIPC feature: systemd-nspawn does not allow to select the pid/ipc/uts namespaces individually. There is --share-system but it selects pid/ipc/uts together.

This patch is a RFC to suggest more options in systemd-nspawn to allow Kubernetes to maintain the HostPID/HostIPC features when using the rkt container run-time.

Additionally, rkt still wants to run systemd in the pod and we use systemd-nspawn --boot for this. What would be the limitations of using --share-ipc and --share-uts with --boot? I guess that systemd with --share-uts should no longer set the hostname. Anything else?


References:

/cc @philips @jonboulle @yifan-gu @iaguis

@poettering
Copy link
Member

Hmm, this appears really strange to me, I fail to see any usecase for the ability to choose these bits independently from each other. Does Kubernetes have a usecase for this, or do they just do it because they can?

This appears super-strange to me...

@alban
Copy link
Member Author

alban commented Apr 8, 2016

Does Kubernetes have a usecase for this, or do they just do it because they can?

I am not sure. I have not seen pods using HostIPC without HostPID (except in Kubernetes unit tests) but on the other hand I have not looked at a lot of pods.

@yifan-gu do you know of more use cases?

@alban
Copy link
Member Author

alban commented Apr 8, 2016

Comment capured from the thread https://github.com/docker/docker/pull/9074/files#r20151640 about different containers communicating in the same IPC namespace:

Our real-life use-cases are:

We have a Kafka producer that listens on a SysV message queue.
We run MRI Ruby which isn't good with multithreading, so we use SysV semaphores for circuit breakers between processes and bulkheading.
(...)

@martinpitt
Copy link
Contributor

Please ignore the ubuntu-amd64 test failure. The reason for this got fixed two days ago.

@poettering
Copy link
Member

poettering commented Apr 12, 2016

I am very strongly of the opinion that such swicthes are pretty crazy, and we should not have them. In fact, I am convinced we probably should stop supporting --share-system as well, and drop it.

That said, I am fine with adding a "debugging" option for this, via env vars. I.e. by setting some env var (let's say SYSTEMD_NSPAWN_SHARE_IPC=1 and SYSTEMD_NSPAWN_SHARE_UTC=1) or so the bits for that are dropped from the clone() invocation. But I don't really want to see this as documented feature, hence it should not show up in NEWS or the man pages, and certainly not be accessible via .nspawn files.

Seems like a pretty poort choice to expose this in Kubernetes and docker, but I guess this "debug" feature would be enough to make nspawn work this this.

Note that the "share ipc" stuff is particularly borked, as Linux primary IPC (which is AF_UNIX) is unaffected by it.

@poettering poettering added the reviewed/needs-rework 🔨 PR has been reviewed and needs another round of reworks label Apr 12, 2016
@poettering
Copy link
Member

Closing this for now. If you are still interested in this, please post a new patch that adds the suggested env vars instead.

@poettering poettering closed this Jul 22, 2016
@alban
Copy link
Member Author

alban commented Aug 2, 2016

/cc @tmrts

@poettering
Copy link
Member

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

nspawn reviewed/needs-rework 🔨 PR has been reviewed and needs another round of reworks

Development

Successfully merging this pull request may close these issues.

3 participants