Skip to content

Commit baea2bf

Browse files
arigselaclaude
andcommitted
fix(openshell): inject AppArmor unconfined annotation on sandbox pods
The openshell sandbox `agent` container needs to call mount(2) with MS_SHARED to set up its own network namespace via `ip netns`. The default containerd AppArmor profile (`cri-containerd.apparmor.d`) denies `mount` outright, regardless of CAP_SYS_ADMIN — AppArmor blocks the syscall before the kernel's capability check. Without this, the pod CrashLoops with: mount --make-shared /run/netns failed: Permission denied (See issue #310 for the full diagnostic history.) Pod-spec `securityContext.appArmorProfile.type: Unconfined` SHOULD work but doesn't always propagate through containerd — kubernetes/kubernetes#125069. The legacy annotation form (`container.apparmor.security.beta.kubernetes.io/<name>`) does work. Add a Kyverno ClusterPolicy that mutates sandbox pods at admission to inject this annotation. Selector is narrow: - Namespace: openshell - ownerReferences contains kind=Sandbox (excludes the gateway StatefulSet pod and any unrelated pods in the namespace) - The pod has a container literally named `agent` Live-tested against the cluster before commit. New sandbox pod ran to Ready=true with 0 restarts on first try; AgentHarness reached Ready=True with reason=SandboxReady. Remove this policy when one of these lands upstream: - openshell ships NetworkMode::Platform (skips netns creation entirely) — NVIDIA/OpenShell#899 - sigs.k8s.io/agent-sandbox propagates runtimeClassName so we can pin sandbox pods to a less restrictive runtime (e.g. crun) - kubelet/containerd honor the pod-level appArmorProfile field Closes #310. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
1 parent 91af407 commit baea2bf

1 file changed

Lines changed: 62 additions & 0 deletions

File tree

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
apiVersion: kyverno.io/v1
2+
kind: ClusterPolicy
3+
metadata:
4+
name: openshell-sandbox-apparmor-unconfined
5+
annotations:
6+
policies.kyverno.io/title: OpenShell Sandbox AppArmor Unconfined
7+
policies.kyverno.io/category: Workaround
8+
policies.kyverno.io/severity: medium
9+
policies.kyverno.io/subject: Pod
10+
policies.kyverno.io/description: >-
11+
Adds the AppArmor "unconfined" annotation to OpenShell sandbox pods
12+
(Pods in the openshell namespace owned by a Sandbox CR) so their
13+
`agent` container can call mount(2) with MS_SHARED to set up its
14+
own network namespace via `ip netns`.
15+
16+
Background: the sandbox agent's startup runs `ip netns add ...`,
17+
which internally does `mount --make-shared /run/netns`. The default
18+
containerd AppArmor profile (`cri-containerd.apparmor.d`) denies
19+
`mount` outright — capabilities like CAP_SYS_ADMIN don't help
20+
because AppArmor blocks the syscall before the kernel's capability
21+
check. Without this annotation the pod CrashLoops with
22+
"mount --make-shared /run/netns failed: Permission denied".
23+
24+
The Pod-spec `securityContext.appArmorProfile.type: Unconfined`
25+
field SHOULD work but doesn't always propagate through containerd
26+
— see https://github.com/kubernetes/kubernetes/issues/125069. The
27+
legacy annotation form does.
28+
29+
Remove this policy once one of these upstream changes lands:
30+
- openshell ships `NetworkMode::Platform` (skips netns creation
31+
entirely) — tracked at https://github.com/NVIDIA/OpenShell/issues/899
32+
- sigs.k8s.io/agent-sandbox propagates `runtimeClassName` from
33+
the Sandbox CR, allowing us to pin sandbox pods to `crun`
34+
- kubelet/containerd honor pod-level `appArmorProfile.Unconfined`
35+
spec:
36+
# mutateExistingOnPolicyUpdate: false — only mutate new admissions.
37+
rules:
38+
- name: annotate-agent-apparmor-unconfined
39+
match:
40+
any:
41+
- resources:
42+
kinds:
43+
- Pod
44+
namespaces:
45+
- openshell
46+
preconditions:
47+
all:
48+
# Only sandbox pods (owned by a Sandbox CR). The gateway pod
49+
# itself is owned by a StatefulSet, so it's excluded.
50+
- key: "{{ request.object.metadata.ownerReferences[?kind=='Sandbox'] | length(@) }}"
51+
operator: GreaterThanOrEquals
52+
value: 1
53+
# And only when the pod has an `agent` container — defends against
54+
# future Sandbox CR variants that might use a different container name.
55+
- key: "{{ request.object.spec.containers[?name=='agent'] | length(@) }}"
56+
operator: GreaterThanOrEquals
57+
value: 1
58+
mutate:
59+
patchStrategicMerge:
60+
metadata:
61+
annotations:
62+
container.apparmor.security.beta.kubernetes.io/agent: unconfined

0 commit comments

Comments
 (0)