Skip to content

[rootless] Support detach netns#2535

Closed
fahedouch wants to merge 4 commits intocontainerd:mainfrom
fahedouch:support-detach-netns
Closed

[rootless] Support detach netns#2535
fahedouch wants to merge 4 commits intocontainerd:mainfrom
fahedouch:support-detach-netns

Conversation

@fahedouch
Copy link
Copy Markdown
Member

By supporting detach network namespace:

@fahedouch fahedouch added the area/rootless Rootless mode label Sep 26, 2023
@fahedouch fahedouch changed the title Support detach netns [rootless] Support detach netns Sep 26, 2023
@fahedouch fahedouch marked this pull request as draft September 26, 2023 23:04
@fahedouch fahedouch force-pushed the support-detach-netns branch 2 times, most recently from 57b93c1 to 42adb17 Compare September 28, 2023 17:01
@AkihiroSuda
Copy link
Copy Markdown
Member

Uncompilable

$ make
GO111MODULE=on CGO_ENABLED=0 GOOS=linux go build -ldflags "-s -w -X github.com/containerd/nerdctl/pkg/version.Version=v1.5.0-94-gf1e23343 -X github.com/containerd/nerdctl/pkg/version.Revision=f1e23343475f469ad64fef27c49d7da8bee3389f"   -o /home/suda/gopath/src/github.com/rootless-containers/rootlesskit/_output/nerdctl github.com/containerd/nerdctl/cmd/nerdctl
pkg/cmd/container/create.go:50:2: no required module provides package github.com/rootless-containers/rootlesskit/v2/pkg/child; to add it:
        go get github.com/rootless-containers/rootlesskit/v2/pkg/child
make: *** [Makefile:57: nerdctl] Error 1

@fahedouch
Copy link
Copy Markdown
Member Author

Uncompilable

$ make
GO111MODULE=on CGO_ENABLED=0 GOOS=linux go build -ldflags "-s -w -X github.com/containerd/nerdctl/pkg/version.Version=v1.5.0-94-gf1e23343 -X github.com/containerd/nerdctl/pkg/version.Revision=f1e23343475f469ad64fef27c49d7da8bee3389f"   -o /home/suda/gopath/src/github.com/rootless-containers/rootlesskit/_output/nerdctl github.com/containerd/nerdctl/cmd/nerdctl
pkg/cmd/container/create.go:50:2: no required module provides package github.com/rootless-containers/rootlesskit/v2/pkg/child; to add it:
        go get github.com/rootless-containers/rootlesskit/v2/pkg/child
make: *** [Makefile:57: nerdctl] Error 1

Sorry I forgot to push go mod graph updates

@AkihiroSuda
Copy link
Copy Markdown
Member

CI didn't run, probably due to merge conflict

@fahedouch fahedouch force-pushed the support-detach-netns branch 2 times, most recently from db7a762 to ffffa76 Compare October 24, 2023 10:02
@AkihiroSuda
Copy link
Copy Markdown
Member

#54 [build-full 18/27] RUN fname="rootlesskit-$(cat /target_uname_m).tar.gz" &&   curl -o "${fname}" -fSL "[https://github.com/rootless-containers/rootlesskit/releases/download//${fname}](https://github.com/rootless-containers/rootlesskit/releases/download//$%7Bfname%7D)" &&   grep "${fname}" "/SHA256SUMS.d/rootlesskit-" | sha256sum -c &&   tar xzf "${fname}" -C /out/bin &&   rm -f "${fname}" /out/bin/rootlesskit-docker-proxy &&   echo "- RootlessKit: " >> /out/share/doc/nerdctl-full/README.md
#54 0.237   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
#54 0.237                                  Dload  Upload   Total   Spent    Left  Speed
#54 0.237 
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     9    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
#54 0.343 curl: (22) The requested URL returned error: 404
#54 ERROR: process "/bin/sh -c fname=\"rootlesskit-$(cat /target_uname_m).tar.gz\" &&   curl -o \"${fname}\" -fSL \"[https://github.com/rootless-containers/rootlesskit/releases/download/${ROOTLESSKIT_VERSION}/${fname}\](https://github.com/rootless-containers/rootlesskit/releases/download/$%7BROOTLESSKIT_VERSION%7D/$%7Bfname%7D/)" &&   grep \"${fname}\" \"/SHA256SUMS.d/rootlesskit-${ROOTLESSKIT_VERSION}\" | sha256sum -c &&   tar xzf \"${fname}\" -C /out/bin &&   rm -f \"${fname}\" /out/bin/rootlesskit-docker-proxy &&   echo \"- RootlessKit: ${ROOTLESSKIT_VERSION}\" >> /out/share/doc/nerdctl-full/README.md" did not complete successfully: exit code: 22
------
 > [build-full 18/27] RUN fname="rootlesskit-$(cat /target_uname_m).tar.gz" &&   curl -o "${fname}" -fSL "[https://github.com/rootless-containers/rootlesskit/releases/download//${fname}](https://github.com/rootless-containers/rootlesskit/releases/download//$%7Bfname%7D)" &&   grep "${fname}" "/SHA256SUMS.d/rootlesskit-" | sha256sum -c &&   tar xzf "${fname}" -C /out/bin &&   rm -f "${fname}" /out/bin/rootlesskit-docker-proxy &&   echo "- RootlessKit: " >> /out/share/doc/nerdctl-full/README.md:
  0     9    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
0.343 curl: (22) The requested URL returned error: 404
------
Dockerfile:169
--------------------
 168 |     ARG ROOTLESSKIT_VERSION
 169 | >>> RUN fname="rootlesskit-$(cat /target_uname_m).tar.gz" && \
 170 | >>>   curl -o "${fname}" -fSL "[https://github.com/rootless-containers/rootlesskit/releases/download/${ROOTLESSKIT_VERSION}/${fname}](https://github.com/rootless-containers/rootlesskit/releases/download/$%7BROOTLESSKIT_VERSION%7D/$%7Bfname%7D)" && \
 171 | >>>   grep "${fname}" "/SHA256SUMS.d/rootlesskit-${ROOTLESSKIT_VERSION}" | sha256sum -c && \
 172 | >>>   tar xzf "${fname}" -C /out/bin && \
 173 | >>>   rm -f "${fname}" /out/bin/rootlesskit-docker-proxy && \
 174 | >>>   echo "- RootlessKit: ${ROOTLESSKIT_VERSION}" >> /out/share/doc/nerdctl-full/README.md
 175 |     ARG SLIRP4NETNS_VERSION
--------------------

https://github.com/containerd/nerdctl/actions/runs/6625309327/job/17996040183?pr=2535

Comment thread .github/workflows/test.yml
@AkihiroSuda
Copy link
Copy Markdown
Member

Still doesn't compile (for GOOS=windows)

# github.com/containerd/nerdctl/pkg/ocihook
Error: pkg/ocihook/ocihook.go:97:33: undefined: rootlessutil.RootlessKitStateDir
# github.com/rootless-containers/rootlesskit/v2/pkg/common
Error: ../../../go/pkg/mod/github.com/rootless-containers/rootlesskit/[email protected]/pkg/common/exec.go:46:4: unknown field Pdeathsig in struct literal of type syscall.SysProcAttr
# github.com/rootless-containers/rootlesskit/v2/pkg/sigproxy/signal
Error: ../../../go/pkg/mod/github.com/rootless-containers/rootlesskit/[email protected]/pkg/sigproxy/signal/signal.go:15:20: undefined: SignalMap
make[1]: *** [Makefile:57: nerdctl] Error 1

@fahedouch fahedouch force-pushed the support-detach-netns branch from 5724344 to f1342cd Compare October 27, 2023 23:38
Signed-off-by: fahed dorgaa <[email protected]>
@fahedouch fahedouch force-pushed the support-detach-netns branch from 0c8aad1 to 436a18b Compare October 28, 2023 12:18
Signed-off-by: fahed dorgaa <[email protected]>
Comment thread pkg/ocihook/ocihook.go
// nsents verified here we are in detached netwoprk ns
// nsPath verified is pointing to the nested detached ns
// user ns is the detch user ns
cniRes, err := opts.cni.Setup(ctx, opts.fullID, nsPath, namespaceOpts...)
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

 time="2023-10-29T13:30:10Z" level=fatal msg="failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: , stderr: time=\"2023-10-29T13:30:10Z\" level=fatal msg=\"failed to call cni.Setup: plugin type=\\\"firewall\\\" failed (add): failed to list iptables chains: running [/usr/sbin/iptables -t filter -S --wait]: exit status 3: iptables v1.8.4 (legacy): can't initialize iptables table `filter': Permission denied (you must be root)\\nPerhaps iptables or your kernel needs to be upgraded.\\n\"\nFailed to write to log, write /home/rootless/.local/share/nerdctl/1935db59/containers/nerdctl-test/0bb9ad1b47ae32ef3ffda72250edcf338ccbadf8d9572d9503e67faded379d57/oci-hook.createRuntime.log: file already closed: unknown"

https://github.com/containerd/nerdctl/actions/runs/6683660354/job/18160087021?pr=2535

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you dump /proc/self/status right after netns.Set to see if CAP_NET_ADMIN is gained in the namespaces?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we need to re-exec the ocihook process to gain the caps

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member Author

@fahedouch fahedouch Oct 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you dump /proc/self/status right after netns.Set to see if CAP_NET_ADMIN is gained in the namespaces?

SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000000000000
SigCgt: fffffffd7fc1feff
CapInh: 0000000000000000
CapPrm: 000001ffffffffff
CapEff: 000001ffffffffff
CapBnd: 000001ffffffffff
CapAmb: 0000000000000000
 capsh --decode=000001ffffffffff
0x000001ffffffffff=cap_chown,cap_dac_override,cap_dac_read_search,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_module,cap_sys_rawio,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_boot,cap_sys_nice,cap_sys_resource,cap_sys_time,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,cap_audit_control,cap_setfcap,cap_mac_override,cap_mac_admin,cap_syslog,cap_wake_alarm,cap_block_suspend,cap_audit_read,cap_perfmon,cap_bpf,cap_checkpoint_restore

@AkihiroSuda cap_net_admin is already present right after netns.Set

Copy link
Copy Markdown
Member Author

@fahedouch fahedouch Oct 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

	if rootlessutil.IsRootlessChild() && detachNetNs {
		stateDir, err := rootlessutil.RootlessKitStateDir()
		if err != nil {
			return err
		}
		ns, err := netns.GetFromPath(filepath.Join(stateDir, "netns"))
		if err != nil {
			return err
		}
		if err = netns.Set(ns); err != nil {
			return fmt.Errorf("switch to detached netns: %w", err)
		}
		ok, err := rootlessutil.HasCaps()
		if err != nil {
			return err
		}
		fmt.Println(ok)

hasCaps()
the above code is printing true

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some progress in:

We can just run the hook with nsenter -n/detached/netns -- nerdctl internal oci-hook ... and call it a day.

@AkihiroSuda
Copy link
Copy Markdown
Member

#2723 will be ready soon, so let me close this PR. Thanks again for working on this.

@AkihiroSuda AkihiroSuda closed this Jan 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/rootless Rootless mode

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants