Skip to content

[Backport release-1.7] fix(kubernetes): keep NFS publish from hanging on a dead server - #5017

Merged
myasnikovdaniil merged 1 commit into
release-1.7from
backport-4791-to-release-1.7
Oct 21, 2026
Merged

myasnikovdaniil merged 1 commit into
release-1.7from
backport-4791-to-release-1.7

Conversation

@github-actions

Copy link
Copy Markdown

Description

Backport of #4791 to release-1.7.

The kubevirt-csi node plugin mounts the NFS export root into a temp
dir to create /data and migrate old files. That mount was hard, so
with the server gone the mkdir, readdir and rename calls on it never
returned. kubelet gives up after 2m and retries, and every retry
blocked the same way, piling up goroutines and nfs-init-* dirs.

Mount the temp root soft, so its RPCs fail once the connection to the
server is declared dead instead of blocking forever; it only does
metadata work that a retried publish redoes. retry=0 keeps mount.nfs
from retrying a failed temp mount for 2m. timeo and retrans keep their
defaults, because the first mount of a server sets them for the
connection the pods' mounts of it share. The volume mount the pod gets
stays hard, since a soft mount can report a failed write the server
applied.

Over NFSv4 that soft failure takes minutes, and a stat of a target
that is already hard-mounted can block for good, so a publish for a
target path that already has one running now returns Aborted instead
of starting another.

A soft stat can fail with EIO on a file /data already holds, and a
rename issued once the server is back would replace it, so the
migration now fails the publish on any stat error other than a
missing destination. It checks the destination with lstat, so a
user's symlink in /data that loops or dangles counts as present
instead of failing every publish or being replaced by the rename.

The skip it makes for a destination already in /data leaves the root
entry where the pod cannot see it, so it is now logged as a warning.
/data is created with a plain mkdir, since with the server gone
MkdirAll fails its stat on the mount point and then reports the local
temp dir as already existing instead of the EIO. The temp mount
errors now name the volume, like the migration errors do.

Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <[email protected]>
(cherry picked from commit f76dedf)
@coderabbitai

coderabbitai Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration
  • Configuration used: Repository: cozystack/cozystack/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: d412a0b8-6ccc-46d5-bf2f-6b9202503f52

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@myasnikovdaniil
myasnikovdaniil merged commit 2c620a7 into release-1.7 Oct 21, 2026
2 checks passed
@myasnikovdaniil
myasnikovdaniil deleted the backport-4791-to-release-1.7 branch October 21, 2026 09:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants