Allow gc-ing with a rootless daemon by thufschmitt · Pull Request #5380 · NixOS/nix

thufschmitt · 2021-10-13T05:10:09Z

Depends on #5226

This PR adds the possibility to defer the collection of the gc roots to an external daemon (communicating with it through a UNIX socket).

The motivation for that is that (excluding #5226), finding the GC roots is the only part of Nix that strictly requires root. So deferring that part to an external tool means that it’s possible to run the Nix daemon as a non-root user (with the obvious security gains that come with it).

Along with it is included such an external tracing deamon (under src/nix-find-roots), written in such a way that it doesn’t depend on anything but the c++ stl.

https://github.com/regnat/rootless-nix-daemon-test provides a sample setup in which the nix-daemon is running as a normal nix-daemon user.

Using this feature is currently guarded behind the external-gc-daemon experimental flag

Fix #5208

nixos-discourse · 2021-10-14T13:12:58Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/tweag-nix-dev-updates-19/15488/1

fricklerhandwerk · 2022-12-19T17:14:30Z

Discussed in Nix team meeting on 2022-12-19:

@edolstra: this is only useful outside of NixOS, because if you own the store, you can mess with the entire system
- @Ericson2314: this could be fixed in principle
  - @edolstra: yes, if we had a restricted daemon that only takes content-addressed
@Ericson2314 post meeting created an issue to discuss this.

nixos-discourse · 2022-12-19T17:15:58Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2022-12-16-nix-team-meeting-minutes-17/24120/1

nixos-discourse · 2023-01-13T13:14:05Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2023-01-13-nix-team-meeting-minutes-23/24644/1

dpulls · 2023-01-30T11:21:51Z

🎉 All dependencies have been resolved !

tests/gc-external-daemon.sh

src/libstore/globals.hh

nixos-discourse · 2023-06-29T13:37:22Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/tweag-nix-dev-update-50/29793/1

tests/functional/gc-external-daemon/common.sh

Not really needed since it's configurable from the config (and people can always use `$NIX_CONFIG` if they really need to configure it from the CLI)

@mopleen

Based on an offline review by @mopleen (thanks!)

It is no longer needed. See f4a8426 which did the same thing in the rest of Nix.

The test plan is taken from https://github.com/thufschmitt/rootless-nix-daemon-test. That intentionally used non-NixOS to get around the ambient Nix daemon, but with newer NixOS we can in fact disable the ambient Nix daemon an run our own! A few things which are needed to make this nicer in the future - NixOS/nixpkgs#3702 A now-fixed issue, but won't be available until 23.05 - NixOS/nixpkgs#263248 NixOS/nixpkgs#263250 Newly opened issues inspired by the process of writing this test.

@eclairevoyant

I misunderstand what was going on and NixOS/nixpkgs#263248 is a non-issue. That means we can improve the code right away. Thank you @eclairevoyant for tipping me off that I was mistaken!

Otherwise the tests get stuck when using the daemon (I didn't understand why though, so this is a dirty patch more than a proper fix, but at least it works now)

Move its definition before a loop that might change `$!`

Ericson2314 · 2024-02-23T16:48:50Z

~~Are we just waiting for a final review after re-agreeing on the "don't use other libraries approach"? We did say we wanted to do this after releasing 2.20.~~

Oh there was the matter of docs from @roberth too.

Not supported by the installer because there are many moving parts, but a rough installation guide can help be used for people to get a custom installation for their needs

That would defeat the whole purpose of the thing as it would provide a nice escalation path from a Nix vulnerability to root access

We got rid of them in the rest of the code, so let's do the same here.

Has been made a bit more lenient upstream, so match that

Co-Authored-By: Robert Hensing <[email protected]>

roberth

Have reviewed the docs.

roberth · 2024-03-06T20:13:36Z

doc/manual/src/installation/installing-rootless-daemon.md

@@ -0,0 +1,138 @@
+# Using Nix in multi-user mode with a non-root daemon
+
+> Experimental blurb


This looks like a placeholder. Was this WIP?

roberth · 2024-03-06T20:24:25Z

doc/manual/src/installation/installing-rootless-daemon.md

+    sudo systemctl start nix-daemon.socket
+    sudo systemctl start nix-find-roots.socket
+    ```
+10. Profit


Suggested change

10. Profit

The daemons are now set up. You may try it:

```sh

head -c 20 /dev/urandom >test.data

nix store add-file test.data

nix store gc --dry-run

```

To my surprise nix store gc --dry-run seems to be real.
It is horribly slow on my store though, so I've interrupted it.

roberth · 2024-03-06T21:30:52Z

doc/manual/src/installation/installing-rootless-daemon.md

+    WantedBy=sockets.target
+    EOF
+    ```
+7. Install the systemd services for the tracing daemon:


Is it a "gc roots" or "tracing" daemon?
We should use one name everywhere.

roberth · 2024-03-06T21:34:50Z

tests/nixos/rootless-daemon.nix

+    # No root daemon
+    nix.enable = false;


non-blocking:

Exposing a module for this kind of setup would be nice, but needs a few improvements.

I think you could blacklist the daemon module, at least on recent enough NixOS the daemon-specific stuff is separate from the Nix installation and config file.

I think we need a nix.daemon.enable in NixOS to be sure.

twoolie · 2024-09-11T02:12:26Z

Is there anything still blocking progress on this PR?

GHSA-h4vv-h3jq-v493 has shown that getting to a rootless daemon is still important.

tomberek · 2024-09-11T04:01:58Z

Is there anything still blocking progress on this PR?

GHSA-h4vv-h3jq-v493 has shown that getting to a rootless daemon is still important.

Needs a rebase, some attention, and someone to shepherd it forward. I'm not aware of any fundamental blockers.

nixos-discourse · 2024-10-02T20:34:42Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-10-02-nix-team-meeting-minutes-183/53815/1

artemist · 2026-01-15T18:46:00Z

tests/nixos/rootless-daemon.nix

+
+    systemd.user.sockets.nix-daemon = {
+    };
+    systemd.user.services.nix-daemon = {


You probably don't want user services, those will only start on a user's first login session. This should be a system service running as the user nix-daemon

thufschmitt added the feature Feature request or proposal label Oct 13, 2021

thufschmitt force-pushed the rootless-daemon branch 2 times, most recently from ffe4be5 to 26355d3 Compare October 13, 2021 11:16

thufschmitt marked this pull request as ready for review November 15, 2021 17:43

thufschmitt force-pushed the rootless-daemon branch from 5421ae6 to 9cd313c Compare February 23, 2022 09:44

thufschmitt force-pushed the rootless-daemon branch 2 times, most recently from 2fceae5 to 6345401 Compare April 13, 2022 09:31

thufschmitt force-pushed the rootless-daemon branch from 6c2dc6f to 18def5b Compare November 2, 2022 09:48

This was referenced Jan 13, 2023

Rootless nix-daemon #789

Open

Non-root multi-user installation #324

Open

thufschmitt force-pushed the rootless-daemon branch from 18def5b to 8e785f3 Compare June 14, 2023 10:01

thufschmitt requested a review from edolstra as a code owner June 14, 2023 10:01

github-actions bot added store Issues and pull requests concerning the Nix store with-tests Issues related to testing. PRs with tests have some priority labels Jun 14, 2023

thufschmitt force-pushed the rootless-daemon branch from 8e785f3 to 3b4bc32 Compare June 14, 2023 11:57

Ericson2314 mentioned this pull request Jun 26, 2023

Steam Deck support #7173

Open

Ericson2314 reviewed Jun 26, 2023

View reviewed changes

tests/gc-external-daemon.sh Outdated Show resolved Hide resolved

Ericson2314 reviewed Jun 26, 2023

View reviewed changes

src/libstore/globals.hh Outdated Show resolved Hide resolved

github-actions bot added the tests label Jun 26, 2023

thufschmitt force-pushed the rootless-daemon branch from 3531cc1 to 12f16d9 Compare June 26, 2023 10:05

github-actions bot removed the with-tests Issues related to testing. PRs with tests have some priority label Oct 17, 2023

thufschmitt mentioned this pull request Oct 17, 2023

Make it possible to run the Nix daemon as a non-root user #5208

Open

github-actions bot added the with-tests Issues related to testing. PRs with tests have some priority label Oct 24, 2023

Ericson2314 reviewed Oct 24, 2023

View reviewed changes

tests/functional/gc-external-daemon/common.sh Outdated Show resolved Hide resolved

Théophane Hufschmitt and others added 12 commits February 23, 2024 10:00

Remove the NIX_GC_SOCKET_PATH environment variable

bc445dc

Not really needed since it's configurable from the config (and people can always use `$NIX_CONFIG` if they really need to configure it from the CLI)

nix-find-roots: Don't assume that argv[0] exists

a97c309

nix-find-roots: Cleanup

8dc4ff6

Based on an offline review by @mopleen (thanks!)

Mark non-header functions static, API docs in header

07e6ee9

tests: Fix config file name

d170ab4

nix-find-roots: Don't explicitly link c++fs on Darwin

c53498b

It is no longer needed. See f4a8426 which did the same thing in the rest of Nix.

Make rootless daemon NixOS test setup more declarative

fb72519

I misunderstand what was going on and NixOS/nixpkgs#263248 is a non-issue. That means we can improve the code right away. Thank you @eclairevoyant for tipping me off that I was mistaken!

Restore some things lost in the rebase

9e79d2d

pname + version

7f64327

tests: Pipe the nix-find-roots output to /dev/null

e859565

Otherwise the tests get stuck when using the daemon (I didn't understand why though, so this is a dirty patch more than a proper fix, but at least it works now)

Make sure that pdiGcDaemon is the right pid

1dbba94

Move its definition before a loop that might change `$!`

thufschmitt force-pushed the rootless-daemon branch from 545e88b to 1dbba94 Compare February 23, 2024 09:00

Théophane Hufschmitt and others added 5 commits February 27, 2024 06:37

Document the installation process with a non-root daemon

e68ee65

Not supported by the installer because there are many moving parts, but a rough installation guide can help be used for people to get a custom installation for their needs

Don't run the tracing daemon from the Nix store

86ae77b

That would defeat the whole purpose of the thing as it would provide a nice escalation path from a Nix vulnerability to root access

find-roots: Remove the using std directives

79a2997

We got rid of them in the rest of the code, so let's do the same here.

find-roots: Fix regex to match store paths

7d3328e

Has been made a bit more lenient upstream, so match that

Reword the experimental feature description.

bb9b33a

Co-Authored-By: Robert Hensing <[email protected]>

thufschmitt requested review from Ericson2314 and roberth March 6, 2024 08:03

roberth reviewed Mar 6, 2024

View reviewed changes

edolstra requested a review from fricklerhandwerk as a code owner November 12, 2024 19:47

dhess mentioned this pull request Nov 15, 2024

docker: Allow building for non-root user #9854

Merged

3 tasks

brainrake mentioned this pull request Feb 13, 2025

om health should not recommend insecure trusted-users juspay/omnix#409

Closed

artemist reviewed Jan 15, 2026

View reviewed changes

artemist mentioned this pull request Jan 20, 2026

Allow gc-ing with a rootless daemon #15026

Closed

		@@ -0,0 +1,138 @@
		# Using Nix in multi-user mode with a non-root daemon

		> Experimental blurb

-. Profit
+The daemons are now set up. You may try it:
+```sh
+head -c 20 /dev/urandom >test.data
+nix store add-file test.data
+nix store gc --dry-run
+```

Uh oh!

Conversation

thufschmitt commented Oct 13, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nixos-discourse commented Oct 14, 2021

Uh oh!

fricklerhandwerk commented Dec 19, 2022

Uh oh!

nixos-discourse commented Dec 19, 2022

Uh oh!

nixos-discourse commented Jan 13, 2023

Uh oh!

dpulls bot commented Jan 30, 2023

Uh oh!

Uh oh!

Uh oh!

nixos-discourse commented Jun 29, 2023

Uh oh!

Uh oh!

Ericson2314 commented Feb 23, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

roberth left a comment

Choose a reason for hiding this comment

Uh oh!

roberth Mar 6, 2024

Choose a reason for hiding this comment

Uh oh!

roberth Mar 6, 2024

Choose a reason for hiding this comment

Uh oh!

roberth Mar 6, 2024

Choose a reason for hiding this comment

Uh oh!

roberth Mar 6, 2024

Choose a reason for hiding this comment

Uh oh!

twoolie commented Sep 11, 2024

Uh oh!

tomberek commented Sep 11, 2024

Uh oh!

nixos-discourse commented Oct 2, 2024

Uh oh!

artemist Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

thufschmitt commented Oct 13, 2021 •

edited

Loading

Ericson2314 commented Feb 23, 2024 •

edited

Loading