Skip to content

Comments

tmpfiles: make --purge more restrictive, and various other tweaks#33383

Merged
bluca merged 10 commits intosystemd:mainfrom
poettering:tmpfiles-limit-purge
Jun 18, 2024
Merged

tmpfiles: make --purge more restrictive, and various other tweaks#33383
bluca merged 10 commits intosystemd:mainfrom
poettering:tmpfiles-limit-purge

Conversation

@poettering
Copy link
Member

@poettering poettering commented Jun 18, 2024

My fix for #33349, alternative for #33353 that tries to properly address things.

This attacks the problem on three fronts:

  1. We refuse to run --purge without specification of at least one tmpfiles.d/ drop-in filename
  2. We document in the man page much clearer what this does with <emphasis> and Warning!.
  3. We document in the --help text better what this actually does, and how it differs from --remove.

I am pretty sure with these changes it's quite hard to still fuck up your system with this, i.e. any chance to learn about --purge should already make clear to you that this is not what you want, and even if you ignore that, you have to go out your way to specify the wrong tmpfiles.d/ snippet on the command line.

Plus a multitude of other fixes, some quite important, such as one that fixes a case where --dry-run wasn't dry at all.

Fixes: #33349
Closes: #33353

@github-actions github-actions bot added documentation please-review PR is ready for (re-)review by a maintainer labels Jun 18, 2024
@YHNdnzj YHNdnzj added good-to-merge/waiting-for-ci 👍 PR is good to merge, but CI hasn't passed at time of review. Please merge if you see CI has passed needs-stable-backport good-to-merge/with-minor-suggestions and removed please-review PR is ready for (re-)review by a maintainer good-to-merge/waiting-for-ci 👍 PR is good to merge, but CI hasn't passed at time of review. Please merge if you see CI has passed labels Jun 18, 2024
@bluca
Copy link
Member

bluca commented Jun 18, 2024

Thanks, this looks right the right fix to me

@bluca
Copy link
Member

bluca commented Jun 18, 2024

I'll prep a stable update after this is merged

@YHNdnzj
Copy link
Member

YHNdnzj commented Jun 18, 2024

I'll prep a stable update after this is merged

Please also wait for #33258 if possible. I think the accounting really should be fixed

@purpleidea
Copy link

Smalll typo: s/combinatin/combination/

@bluca bluca force-pushed the tmpfiles-limit-purge branch from 6eabc61 to c142a8f Compare June 18, 2024 13:47
@bluca
Copy link
Member

bluca commented Jun 18, 2024

Pushed the trivial fixes to move the check and fix the typo. Once the fast CI is done I'll merge.

@bluca bluca added good-to-merge/waiting-for-ci 👍 PR is good to merge, but CI hasn't passed at time of review. Please merge if you see CI has passed and removed good-to-merge/with-minor-suggestions labels Jun 18, 2024
@bluca
Copy link
Member

bluca commented Jun 18, 2024

I'll prep a stable update after this is merged

Please also wait for #33258 if possible. I think the accounting really should be fixed

That needs Lennart's review I think - we do stable releases regularly, it can go in the next one if it's not done in time

@keszybz
Copy link
Member

keszybz commented Jun 18, 2024

The helper is in https://salsa.debian.org/debian/debhelper/-/blob/main/dh_installtmpfiles?ref_type=heads…
IIUC, dh_installtmpfiles is the standard way to handle a tmpfile.d snippet in a package. And unless the maintainer opts-out, systemd-tmpfiles purge will be hooked up for apt remove --purge. Sorry, but this doesn't seem like a good idea at all.

In some way, I think this is a Debian problem, because other systems don't have the concept of "purge". Rpm certainly doesn't. This means that users would have to invoke systemd-tmpfiles --purge directly. This is still possible, and I'm not happy about this possibility, but it's less likely.

@bluca
Copy link
Member

bluca commented Jun 18, 2024

The helper is in https://salsa.debian.org/debian/debhelper/-/blob/main/dh_installtmpfiles?ref_type=heads… IIUC, dh_installtmpfiles is the standard way to handle a tmpfile.d snippet in a package. And unless the maintainer opts-out, systemd-tmpfiles purge will be hooked up for apt remove --purge. Sorry, but this doesn't seem like a good idea at all.

Yes, that is all correct, as mentioned it's opt-out - in the new compat mode of course as it's an incompatible change, and it is documented. So one has to first switch from debhelper-compat 13 to 14 to get the new behaviour. Purging everything that is left behind is the reason this exists in the first place.

In some way, I think this is a Debian problem, because other systems don't have the concept of "purge". Rpm certainly doesn't. This means that users would have to invoke systemd-tmpfiles --purge directly. This is still possible, and I'm not happy about this possibility, but it's less likely.

Yeah but just invoking it does nothing now, which is the point - the belt and braces. You have to explicitly call systemd-tmpfiles --purge /usr/lib/tmpfiles.d/home.conf, which with the updated documentation doesn't seem any different to me than doing rm -rf /home. I mean, it's even worse, because running rm -rf is a common operation if someone uses the terminal, while let's be real, absolutely nobody would habitually call sd-tmpfiles manually in the real world. I mean, as far as I'm concerned we could even move it out of bin and place it with the other private binaries that are run by services in /usr/lib/systemd.

@keszybz
Copy link
Member

keszybz commented Jun 18, 2024

Yes, that is all correct, as mentioned it's opt-out - in the new compat mode of course as it's an incompatible change, and it is documented. So one has to first switch from debhelper-compat 13 to 14 to get the new behaviour. Purging everything that is left behind is the reason this exists in the first place.

That still seems to be a very bad idea. I'm pretty sure not all packagers read all debhelper-compat changelogs attentively, and even if they did, somebody might add a new tmpfiles line at some later point, either upstream or downstream.

You have to explicitly call systemd-tmpfiles --purge /usr/lib/tmpfiles.d/home.conf

systemd-tmpfiles --purge home.conf is enough. But I'll concede that invoking it manually like that seems unlikely to be done accidentally. But invocations with --purge are wired-up automatically by dh_installtmpfiles, i.e. they will be called from apt purge. That seems very risky.

I really don't have a problem with tmpfiles removing a bunch of files or temporary directories. But the recursive operation on stateful directories I cannot get behind.

@bluca
Copy link
Member

bluca commented Jun 18, 2024

Yes, that is all correct, as mentioned it's opt-out - in the new compat mode of course as it's an incompatible change, and it is documented. So one has to first switch from debhelper-compat 13 to 14 to get the new behaviour. Purging everything that is left behind is the reason this exists in the first place.

That still seems to be a very bad idea. I'm pretty sure not all packagers read all debhelper-compat changelogs attentively, and even if they did, somebody might add a new tmpfiles line at some later point, either upstream or downstream.

I mean, a package also might add a unit that does dd if=/dev/zero of=/dev/sda on boot - being a packager means auditing what you package, and means reading the documentation of the tools that you use, this is not really anything new, nor it is a convincing reason for not integrating it. If the possibility of some unknown hypothetical person opting in to some functionality without reading what it does would be an issue, then nothing could be done, ever.

You have to explicitly call systemd-tmpfiles --purge /usr/lib/tmpfiles.d/home.conf

systemd-tmpfiles --purge home.conf is enough. But I'll concede that invoking it manually like that seems unlikely to be done accidentally. But invocations with --purge are wired-up automatically by dh_installtmpfiles, i.e. they will be called from apt purge. That seems very risky.

I really don't have a problem with tmpfiles removing a bunch of files or temporary directories. But the recursive operation on stateful directories I cannot get behind.

But I'm not asking you to add this to RPM - as you already said, RPM doesn't really have the concept of 'purging', so it doesn't really apply anyway. So what's the problem? You are not convinced that it's good to have, so you won't use it. I am, so I will. Everybody wins?

@keszybz
Copy link
Member

keszybz commented Jun 18, 2024

We'll just have to agree to disagree. I'll just say that I think --purge in the upstream form is a somewhat-iffy-but-essentially-unlikely-to-be-problematic, but in the apt purge form is a hole waiting for a foot.

In Fedora, I pushed the following two commits:

@bluca
Copy link
Member

bluca commented Jun 18, 2024

We could also drop it from the manpage if you like, I'm ok with that - as mentioned, I think this is for scripted use anyway

@keszybz
Copy link
Member

keszybz commented Jun 18, 2024

The man page now has an explanation and warnings, so I think it's better to keep it.

@poettering
Copy link
Member Author

Oh man, this really sucks. I build mkosi images regularly that only contain a /usr/ and that augment the root fs via tmpfiles.d/. If you drop this then you are actively breaking my setup. Uncool. And why even? I don't get it.

Please put that back, there's a reason this exists and I use it regularly. It's also what makes systemd-nspawn --volatile=full --bind-user=… work and things.

@poettering
Copy link
Member Author

You know, I am happy to just kill the --purge thing entirely again, if this means you are breaking systemd that come up with /usr/ only. I find the purge usecase much weaker than the usecase of systems with /usr/ only.

Kinda sucks that the /usr/ merge is kinda pointless in Fedora now, if you cannot actually deploy a system anymore with just /usr/...

@poettering
Copy link
Member Author

Hmm, so the rpm doesn't ship sysusers.d/ stuff either anymore? Man this is so disappointing. I wasn't aware Fedora is going backwards so badly.

@bluca
Copy link
Member

bluca commented Jun 18, 2024

Join the Debian dark side. We have cookies :-P

@poettering
Copy link
Member Author

This was rushed through. There were points raised in the discussion in the original issue and the other PR that are not addressed. I'm a bit disappointed that you didn't even wait for my review, since this was done partially in response to my pull request.

I actually agree. I don't think there was any need to hurry this in like this, not even giving me the chance to address @YHNdnzj review points myself.

@poettering
Copy link
Member Author

btw, one way out for this would be to add yet another line type modifier to things. For example, we could say that if you combine some line type with o (for "ownership") it will be subject to --purge, but nothing else. I think that would be reasonably safe.

Example:

# create a dir, and make it subject to --purge
do /run/foobar 0755 root root
# create a dir, but make --purge have no effect
d /run/waldo 0755 root root
# just declare ownership
o /run/piff

or so.

@bluca
Copy link
Member

bluca commented Jun 18, 2024

Oh man, this really sucks. I build mkosi images regularly that only contain a /usr/ and that augment the root fs via tmpfiles.d/. If you drop this then you are actively breaking my setup. Uncool. And why even? I don't get it.

Please put that back, there's a reason this exists and I use it regularly. It's also what makes systemd-nspawn --volatile=full --bind-user=… work and things.

We could just add it to the local mkosi config in the repo? If that's how you use it, it wouldn't make a difference whether it came from the package or from mkosi.extra, no?

@bb010g
Copy link

bb010g commented Jun 24, 2024

btw, one way out for this would be to add yet another line type modifier to things. For example, we could say that if you combine some line type with o (for "ownership") it will be subject to --purge, but nothing else. I think that would be reasonably safe.

Reading through the current tmpfiles.d(5) manual page, the following jumps out at me as a case where o should not be used:

# Modify sysfs but don't fail if we are in a container with a read-only /proc
w- /proc/sys/vm/swappiness - - - - 10

systemd-tmpfiles --purge should generally not try to delete /proc/sys/ files.

As a developer, I would appreciate the o line type modifier as a way to communicate, via upstream tmpfiles.d(5) files, to distribution maintainers what files and directories the project expects to (not) own. The FILES section of Unix manuals is already available for this purpose, but tmpfiles.d(5) files are more likely than manuals to stay in sync as code updates. It would also make me more confident about using tmpfiles.d(5) files to create files a project doesn't own, now that I know systemd-tmpfiles --purge exists and is likely to be used by maintainers in uninstallation hooks.

As a distribution maintainer, I could see this being useful for split packages. For example:

# /usr/lib/tmpfiles.d/hello.conf
o /var/spool/hello

# /usr/lib/tmpfiles.d/hello-foo.conf
d /var/spool/hello 0755 root root

# /usr/lib/tmpfiles.d/hello-bar.conf
d /var/spool/hello 0755 root root

The package hello does not spool. Its dependencies hello-foo and hello-bar both spool to the same directory, but removing hello-foo while hello-bar is still installed must not purge the entire spool directory. Removing hello means hello-foo and hello-bar must also be removed, so the entire spool directory can be purged when hello is uninstalled.

jamacku pushed a commit to jamacku/systemd-rhel10 that referenced this pull request Jun 26, 2024
Follow-up for systemd/systemd#33383.

rhel-only: bugfix

Related: RHEL-40924
github-actions bot pushed a commit to redhat-plumbers/systemd-rhel10 that referenced this pull request Jun 26, 2024
Follow-up for systemd/systemd#33383.

rhel-only: bugfix

Related: RHEL-40924
jamacku pushed a commit to jamacku/systemd-rhel10 that referenced this pull request Dec 16, 2024
Follow-up for systemd/systemd#33383.

rhel-only: bugfix

Related: RHEL-40924
jamacku pushed a commit to jamacku/systemd-rhel10 that referenced this pull request Dec 16, 2024
Follow-up for systemd/systemd#33383.

rhel-only: bugfix

Related: RHEL-40924
jamacku pushed a commit to jamacku/systemd-rhel10 that referenced this pull request Dec 16, 2024
Follow-up for systemd/systemd#33383.

rhel-only: bugfix

Related: RHEL-40924
jamacku pushed a commit to jamacku/systemd-rhel10 that referenced this pull request Dec 16, 2024
Follow-up for systemd/systemd#33383.

rhel-only: bugfix

Related: RHEL-40924
jamacku pushed a commit to jamacku/systemd-rhel10 that referenced this pull request Dec 16, 2024
Follow-up for systemd/systemd#33383.

rhel-only: bugfix

Related: RHEL-40924
jamacku pushed a commit to jamacku/systemd-rhel10 that referenced this pull request Dec 16, 2024
Follow-up for systemd/systemd#33383.

rhel-only: bugfix

Related: RHEL-40924
jamacku pushed a commit to jamacku/systemd-rhel10 that referenced this pull request Dec 17, 2024
Follow-up for systemd/systemd#33383.

rhel-only: bugfix

Related: RHEL-40924
jamacku pushed a commit to redhat-plumbers/systemd-rhel10 that referenced this pull request Dec 17, 2024
Follow-up for systemd/systemd#33383.

rhel-only: bugfix

Related: RHEL-40924
jamacku pushed a commit to redhat-plumbers/systemd-rhel10 that referenced this pull request Dec 20, 2024
Follow-up for systemd/systemd#33383.

rhel-only: bugfix

Related: RHEL-40924
jamacku pushed a commit to redhat-plumbers/systemd-rhel10 that referenced this pull request Jan 8, 2025
Follow-up for systemd/systemd#33383.

rhel-only: bugfix

Related: RHEL-40924
jamacku pushed a commit to redhat-plumbers/systemd-rhel10 that referenced this pull request Feb 3, 2025
Follow-up for systemd/systemd#33383.

rhel-only: bugfix

Related: RHEL-40924
jamacku pushed a commit to redhat-plumbers/systemd-rhel10 that referenced this pull request Feb 10, 2025
Follow-up for systemd/systemd#33383.

rhel-only: bugfix

Related: RHEL-40924
jamacku pushed a commit to redhat-plumbers/systemd-rhel10 that referenced this pull request Feb 14, 2025
Follow-up for systemd/systemd#33383.

rhel-only: bugfix

Related: RHEL-40924
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

refuse systemd-tmpfiles --purge invocation without config file specified on cmdline

6 participants