Skip to content

agent: mask directories in maskedPaths with a shared read-only tmpfs - #14189

Merged
BbolroC merged 3 commits into
kata-containers:mainfrom
dims:agent-mask-dirs
Nov 2, 2026
Merged

BbolroC merged 3 commits into
kata-containers:mainfrom
dims:agent-mask-dirs

Conversation

@dims

@dims dims commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

kata-agent bind-mounts /dev/null over each maskedPaths entry and treats ENOTDIR as success, so
directories such as /sys/firmware are never masked. With containerd's defaults every kata
container can read the guest's ACPI tables, SMBIOS table and memory map, and a rootfs directory
in the list stays writable.

Mount an empty read-only tmpfs (nr_blocks=1,nr_inodes=1) over the first masked directory and
bind-mount it over the others, so one tmpfs covers them all. runc and crun do the same:

The tmpfs gets the spec's mount label when SELinux is enabled in the guest, like the other
mounts.

Fixes: #14188

Tested on x86_64 with kata-static 4.3.1 (runtime-rs, cloud-hypervisor and qemu, guest kernel
6.18.56) and the agent from this branch in the guest image:

  • containerd defaults: /proc/acpi and /sys/firmware empty, one tmpfs superblock, writes EROFS
  • a 19-entry list with 12 existing directories, a duplicate, files, a missing path and a path
    under a file: 12 masks on one superblock, files read empty, the rest skipped
  • the stock agent with the same list: no directory masks, /etc/ssl and /etc/apk writable
  • rustjail unit tests, make check, and make test with the same results as main

@dims

dims commented Nov 1, 2026

Copy link
Copy Markdown
Contributor Author

cc @zvonkok @fidencio

Comment thread src/agent/rustjail/src/mount.rs Resolved

@fidencio fidencio left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks @dims!

the comment change is optional, and worth doing only if there's any other change requested.

mask_path bind-mounts /dev/null over each entry of linux.maskedPaths and
treats ENOTDIR as success, so a directory such as /sys/firmware or
/proc/acpi stays visible in the guest. containerd's default list has five
directories, and it adds one thermal_throttle directory per CPU.

Mount an empty read-only tmpfs (nr_blocks=1,nr_inodes=1) over the first
masked directory and bind-mount it over the others. A bind shares the
superblock, so it is read-only too and the container gets one tmpfs
instance however many directories are masked. When SELinux is enabled
in the guest, the tmpfs gets the spec's mount label, like the other
mounts. runc and crun mask directories the same way.

Fixes: kata-containers#14188
Signed-off-by: Davanum Srinivas <[email protected]>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Critical masking defects can abort container creation for valid masked-path orders and omit the supplied tmpfs mount label on SELinux-enabled guests.

3 open issues
What changed in this PR

This PR addresses directory entries in maskedPaths being left exposed by kata-agent, as reported in #14188. It adds directory masking alongside the existing file masking.

Changes:

  • Masks directories with an empty, read-only tmpfs shared across entries.
  • Adds mount-label handling for SELinux-enabled guests.
  • Extends unit tests for directory reuse, file masks, and skipped paths.
File Description
src/​agent/​rustjail/​src/​mount.rs Implements shared tmpfs directory masking, passes the spec’s mount label, and extends masking tests.

Review effort: Balanced

Comment thread src/agent/rustjail/src/mount.rs Resolved
Comment thread src/agent/rustjail/src/mount.rs Resolved
Comment thread src/agent/rustjail/src/mount.rs Resolved
selinux::is_enabled looks for selinuxfs in /proc/mounts. finish_rootfs
runs after pivot_rootfs has detached the guest root, so /proc/mounts in
the container no longer lists the guest's selinuxfs and the check
returns false. On an SELinux guest the masking tmpfs then gets no
context option.

do_init_child already reads the SELinux state before it unshares the
mount namespace and uses it for the process label. Pass that value to
finish_rootfs and mask_paths.

Signed-off-by: Davanum Srinivas <[email protected]>

@BbolroC BbolroC left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, it looks good to me. I've left just one comment, which is non-blocker. Thanks!

Comment thread src/agent/rustjail/src/mount.rs Resolved
@BbolroC

BbolroC commented Nov 2, 2026

Copy link
Copy Markdown
Member

@zvonkok back to your question:

One last thing to check since Copilot complained on keeping a handle open to /dev/null 
once you TMPFS /dev does the privileged_wihtout_host_devices = true | false  still work properly.

Here is my two cents:

  • privileged_without_host_devices does its work early on during init_rootfs (deciding which host devices are listed in the spec and created via mknod), before pivot_root
  • This PR only touches masking, which runs later in finish_rootfs. No commit changes how /dev is mounted or populated. The /dev handling in finish_rootfs (the read-only remount) is untouched.

So, privileged_wihtout_host_devices = true | false should work with this PR. Thanks!

mask_path bind-mounted the shared tmpfs by the path it was mounted on.
A later entry in maskedPaths can mask a parent of that path, and the
next bind then fails with ENOENT. With /a/b, /a and /c in that order,
/a/b holds the tmpfs, masking /a hides it, and the bind onto /c aborts
the container start.

Open the root of the tmpfs with O_PATH after the first mount and bind
from /proc/self/fd instead. The descriptor stays attached to the tmpfs
whatever is mounted over its path later. runc and crun keep a
descriptor as well.

The unit tests mock mount(). The mock now records its calls, so the
tests check what each masked path gets mounted from: /dev/null for a
file, a read-only tmpfs with the mount label for the first directory,
and the descriptor of that tmpfs for every later directory, also when
a child is listed before its parent. test_mask_path builds its tree in
a temporary directory instead of using /tmp and /dev.

Signed-off-by: Davanum Srinivas <[email protected]>

@zvonkok zvonkok left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @dims

@BbolroC
BbolroC merged commit 2307616 into kata-containers:main Nov 2, 2026
540 of 556 checks passed
BbolroC added a commit to BbolroC/kata-containers that referenced this pull request Nov 3, 2026
The IBM SEL check in `k8s-confidential.bats` runs
"cd /sys/firmware/uv; cat prot_virt_guest | grep 1" over ssh inside the
workload container. containerd lists /sys/firmware in maskedPaths, and since
the agent started masking directories with an empty read-only tmpfs (kata-containers#14189),
the container no longer sees /sys/firmware/uv. grep then exits non-zero,
the ssh loop reports "Failed to connect to pod." six times and the test dies
with "Confidential compute is expected but not enabled.", although the guest
is a protected env.

A privileged container has no masked paths, so set privileged on the
Deployment's container for qemu-se* only. The other TEE flavours check
dmesg or cpuid and do not read /sys/firmware, so they keep the unmodified
pod YAML.

This is a test-only workaround. It is not a recommended configuration,
and real workloads should not run privileged just to see /sys/firmware.

Assisted-By: IBM Bob
Signed-off-by: Hyounggyu Choi <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

agent: directories in maskedPaths are not masked

5 participants