Skip to content

zfs-service: shutdown still wedges after #1104 when DRBD holds the zvols #1270

Open

Description

@lexfrei

The shutdown export timeout from #1104 does not fix the DRBD-on-zvol case reported in #1085. The shutdown still wedges, now in the container's PID namespace instead of the service process.

I run Talos v1.13.6 arm64 with the zfs extension 2.4.3-v1.13.6, which includes #1104 (558e19a is in the v1.13.6 tag). LINSTOR/DRBD sits on the zvols. A reboot after talosctl upgrade and a talosctl reset both stayed in rebooting until I hard-reset the instance.

The timeout fires, but the export does not stop. zpool gets SIGKILL while it is blocked in the export ioctl, so it cannot exit and stays a zombie in the service's PID namespace. PID 1 has already returned from main(), and the kernel then waits for the rest of that namespace. Read from /proc on the wedged node:

zfs-service  pid 5172   State: Z
  thread 19392  State: S
    do_wait
    kernel_wait4
    zap_pid_ns_processes
    do_exit

zpool        pid 19391  State: Z, same PID namespace
  thread 19393  State: D
    taskq_wait [spl]
    spa_export_common [zfs]
    zfs_ioc_pool_export [zfs]
    zfsdev_ioctl

The console shows the same timing: Sending SIGTERM to task ext-zfs-service at 92.39s and Sending SIGKILL at 102.42s, with no Finished in between. The service never got to the clean exit the timeout is meant to give it.

The export blocks because DRBD still holds the zvols. 6 of 7 /sys/block/zd*/holders pointed at drbd* devices, and by then the CRI is down, so linstor-satellite cannot bring them down. As a control, I froze linstor-satellite and ran drbdsetup down all before the same talosctl reset. ext-zfs-service then finished 0.3s after SIGTERM, and the reset completed.

The code on main is the same, so I expect 1.14 to behave the same way. ZFS_EXPORT_TIMEOUT=0 looks like the working option for this layout. Could the export be skipped when a zvol has holders, or could 0 be the documented setting for DRBD on zvols? I can attach a support bundle and the serial console log.

Activity

  1. lexfrei commented on Sep 30, 2026

    @lexfrei
    Author

    Confirmed ZFS_EXPORT_TIMEOUT=0 avoids it. With an ExtensionServiceConfig for zfs-service setting it, a node with 6 zvols held by DRBD rebooted in about 90s, and the pool imported fine on boot. Note that the running service does not pick up the new environment until its next start, so the first reboot after adding the config still goes through the old export.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions