systemd version the issue has been seen with

238

Used distribution
Fedora Workstation 28, with setenforce 0.

Expected behaviour you didn't see
unprivileged users cannot cause system services to fail with "Permission denied" errors.

Unexpected behaviour you saw
unprivileged user with access to FUSE can cause system services to fail with "Permission denied" errors when they are (re)started.

Steps to reproduce the problem

1. mkdir -p ~/mnt/mnt
2. bindfs --no-allow-other ~/mnt/mnt ~/mnt/mnt
3. bindfs --no-allow-other ~/mnt ~/mnt
4. systemctl start test.service

# /etc/systemd/system/test.service
[Service]

DynamicUser=yes
User=test-service
StateDirectory=test-service

ProtectSystem=strict

Type=oneshot

#ExecStart=/bin/ls -l /proc/self/ns
#ExecStart=/bin/cat /proc/self/mountinfo
#ExecStart=/bin/ls -ld /var/lib/test-service

ExecStart=/bin/touch /var/lib/test-service

Additional information

This is adapted from @siebenmann's problem(s).

#9835 (comment)

the trace we see is:

[....]
26787 openat(AT_FDCWD, "/run/systemd/unit-root/h/281/ckstst", O_RDONLY|O_CLOEXEC|O_PATH|O_DIRECTORY) = 4
26787 name_to_handle_at(4, "thinclient_drives", {handle_bytes=128}, 0x7ffe1eca7454, 0) = -1 EACCES (Permission denied)
26787 openat(4, "thinclient_drives", O_RDONLY|O_CLOEXEC|O_PATH) = -1 EACCES (Permission denied)
26787 newfstatat(4, "thinclient_drives", 0x7ffe1eca7560, AT_SYMLINK_NOFOLLOW) = -1 EACCES (Permission denied)
26787 close(4)                          = 0
26787 close(3)                          = 0
26787 chdir("/")                        = 0
26787 setresgid(62583, 62583, 62583)    = 0
[...]

On Ubuntu 18.04's version of systemd (based on version 237), the issue happens any time we have a FUSE mount (active or inactive) in a NFS mounted home directory that is not world-accessible (and thus not accessible to UID 0 on the NFS client). This happens with eg sshfs.

@siebenmann Sounds like it could be something along the lines of ProtectHome=read-only -> bind_remount_recursive_with_mountinfo() -> path_is_mount_point(). Ouch. Thanks for the specific details of your case.

He mentioned on Twitter that changing some directory mode to 0755 allowed root to start accessing his problem mount, at least enough for root to be able to unmount it. At first I didn't understand why systemd would be blocked by mode 0000, when I think it runs as root with CAP_DAC_READ_SEARCH. I think the implication was that CAP_DAC_READ_SEARCH has no effect on NFS, which sounds very plausible.

But we can start looking at this with FUSE as above, if NFS is harder to set up for testing.

Simply creating one --no-allow-other mount didn't seem to cause a problem. Perhaps one of the fallbacks systemd uses has the ability to avoid touching inside the specific mount point which it is querying. (A similar ability is required for umount, when you need to unmount an unresponsive NFS filesystem).

It seems that you can't nest no-root-allowed mounts. The mount program is usually going to be running as (set-uid) root. But, I was at least able to trick systemd into trying to access inside a no-root-allowed mount.