Skip to content

Cannot remount /etc while rebuilding with etc.overlay enabled #303262

@oluceps

Description

@oluceps

Describe the bug

Cannot remount /etc while rebuilding with etc.overlay enabled.
This issue does not always appear.

May related:
#270727
#291398

Steps To Reproduce

  systemd.sysusers.enable = true;
  system.etc.overlay.enable = true;
  system.etc.overlay.mutable = true;
doas nixos-rebuild switch (or with colmena apply-local --sudo)
<..snip
ktop.systemd1 /org/freedesktop/systemd1 org.freedesktop.systemd1.Manager ListUnitsByNames as 1 -- dbus-broker.service' exited with value 1 at /nix/store/0wsk0nwb6bq5f8bfxwny6bww68c44pji-nixos-system-kaambl-24.05.20240408.4cba8b5/bin/switch-to-configuration line 145.
kaambl | Activation failed: Child process exited with error code: 1
       | Failed: Child process exited with error code: 1
[ERROR] Failed to complete requested operation - Last 1 lines of logs:
[ERROR]  failure) Child process exited with error code: 1
[ERROR] Failed to deploy to kaambl - Last 20 lines of logs:
[ERROR]   stderr) Successfully installed Lanzaboote.
[ERROR]   stderr) stopping the following units: agenix-install-secrets.service, dae.service, systemd-tmpfiles-resetup.service, systemd-udevd-control.socket, systemd-udevd-kernel.socket, systemd-udevd.service
[ERROR]   stderr) activating the configuration...
[ERROR]   stdout) remounting /etc...
[ERROR]   stderr) mount: /tmp/tmp.T2fXevh7N0: overlay already mounted on /etc.
[ERROR]   stderr)        dmesg(1) may have more information after failed mount system call.
[ERROR]   stderr) Moving mount
[ERROR]   stderr) Mounting beneath top mount
[ERROR]   stderr) Invalid argument | move-mount.c: 553: main: move_mount
[ERROR]   stdout) Attaching mount /tmp/tmp.T2fXevh7N0 -> /etc
[ERROR]   stdout) Moving single attached mount
[ERROR]   stdout) Activation script snippet 'etc' failed (1)
[ERROR]   stderr) Reload daemon failed: Connection reset by peer
[ERROR]   stderr) reloading user units for elen...
[ERROR]   stderr) su: Cannot determine your user name.
[ERROR]   stderr) restarting sysinit-reactivation.target
[ERROR]   stderr) Failed to restart sysinit-reactivation.target: Connection timed out
[ERROR]   stderr) See system logs and 'systemctl status sysinit-reactivation.target' for details.
[ERROR]   stderr) '/nix/store/ydkp4xlbpmvf1j5xp09rw70vy3vb5n5a-system-path/bin/busctl --json=short call org.freedesktop.systemd1 /org/freedesktop/systemd1 org.freedesktop.systemd1.Manager ListUnitsByNames as 1 -- dbus-broker.service' exited with value 1 at /nix/store/0wsk0nwb6bq5f8bfxwny6bww68c44pji-nixos-system-kaambl-24.05.20240408.4cba8b5/bin/switch-to-configuration line 145.
[ERROR]  failure) Child process exited with error code: 1
[ERROR] -----
[ERROR] Operation failed with error: Child process exited with error code: 1
Hint: Backtrace available - Use `RUST_BACKTRACE=1` environment variable to display a backtrace

kernel log:

Apr 11 02:01:29 kaambl kernel: erofs: (device loop1): mounted with root inode @ nid 36.
Apr 11 02:01:29 kaambl kernel: overlayfs: upperdir is in-use as upperdir/workdir of another mount, mount with '-o index=off' to override exclusive upperdir protection.

Expected behavior

/etc successfully mounted and switch complete.

Screenshots

If applicable, add screenshots to help explain your problem.

Additional context

Add any other context about the problem here.

Notify maintainers

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

> nix-info -m
 - system: `"x86_64-linux"`
 - host os: `Linux 6.8.4-cachyos, NixOS, 24.05 (Uakari), 24.05.20240408.4cba8b5`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.18.2`
 - channels(root): `"nixos"`
 - nixpkgs: `/nix/store/yzkrxddg9fjcjcahb197lgrsz4i9cbhh-450afzqlzzgw6wnyc3dwysf3i5yxyqkr-source`

Add a 👍 reaction to issues you find important.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions