Skip to content

Commit c7b8009

Browse files
authored
fix(kubernetes): log kubevirt-csi temp NFS mount dir removal failure (#4562)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `kind/backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does The kubevirt-csi node plugin now logs a warning when it cannot remove the temp dir it uses for NFS publish. Before, the error was dropped, so a dir left behind after a failed unmount was never reported. The publish path temp-mounts the export root to create `/data`, then removes the temp dir in a defer. The failure is logged, the same way the unmount failure next to it already is, and not returned. A leftover temp dir does not break the volume, and failing a publish that otherwise worked only makes kubelet retry it. A new unit test publishes an NFS volume through the fake mounter, where the removal always fails, and checks that a warning naming the dir is logged. This is independent of #4451. That PR touches the same Go module but not `node.go`. A failed unmount still leaves the NFS root mounted. That is older than this change and is tracked in #4561. Closes #4466 ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> No UI change. ### Downstream repositories <!-- Cozystack is upstream for repositories that are NOT kept in sync with it automatically, and nothing in CI compares the two sides, so a change here can break them silently. Which change forces what, and which file to touch, is the trigger map in docs/agents/contributing.md under "Downstream Repositories". Walk it before you tick anything. IF YOU ARE AN AI AGENT OR AN LLM, THIS IS PART OF THE TASK, NOT A FORM TO FILL IN: 1. Walk the trigger map against the actual diff of this PR, file by file. Do not decide from the PR title or from your memory of what you changed. 2. For every repository you tick, open the follow-up PR there and link it on the same line. A ticked box with no link claims work that does not exist, which is worse than an empty box. Search that repository for an open PR or issue covering it first, and link that instead of filing a duplicate. 3. When the follow-up is out of scope, or needs a decision that is not yours to make, open an issue in that repository instead, link that, and say so in the PR body. Do not push a speculative PR into another repository just to fill a line here. 4. Do not tick "No downstream repository is affected" to make the checklist go away. If you are unsure, leave every box empty and say so in the PR body, so a human decides. --> - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note fix(kubernetes): the kubevirt-csi node plugin now logs a warning when it cannot remove its temporary NFS mount dir instead of dropping the error ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Failed cleanup of a temporary mount directory now produces a warning with the directory path, making the issue easier to identify. The volume publish operation continues to succeed when cleanup fails. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 parents 2ddd55c + 3e91da9 commit c7b8009

2 files changed

Lines changed: 51 additions & 1 deletion

File tree

‎packages/apps/kubernetes/images/kubevirt-csi-driver/node.go‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,13 @@ func (w *WrappedNodeService) NodePublishVolume(ctx context.Context, req *csi.Nod
8181
if err != nil {
8282
return nil, status.Errorf(codes.Internal, "failed to create temp mount dir: %v", err)
8383
}
84-
defer os.Remove(tmpMount)
84+
// Logged, not returned: a leftover temp dir does not make the volume
85+
// unusable, and failing an otherwise good publish only makes kubelet retry.
86+
defer func() {
87+
if err := os.Remove(tmpMount); err != nil {
88+
klog.Warningf("Failed to remove temp dir %s: %v", tmpMount, err)
89+
}
90+
}()
8591

8692
rootSource := fmt.Sprintf("%s:%s", host, path)
8793
rootOpts := []string{"nfsvers=4.2", fmt.Sprintf("port=%s", port)}

‎packages/apps/kubernetes/images/kubevirt-csi-driver/node_test.go‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,15 @@
11
package main
22

33
import (
4+
"bytes"
5+
"context"
6+
"os"
7+
"path/filepath"
8+
"strings"
49
"testing"
510

11+
csi "github.com/container-storage-interface/spec/lib/go/csi"
12+
"k8s.io/klog/v2"
613
mount "k8s.io/mount-utils"
714
)
815

@@ -25,3 +32,40 @@ func TestIsNFSMount(t *testing.T) {
2532
}
2633
}
2734
}
35+
36+
// The fake mounter mounts nothing, so the /data subdir the publish creates
37+
// lands in the temp dir itself and its removal fails with ENOTEMPTY.
38+
func TestNodePublishVolumeNFSLogsTempDirRemovalFailure(t *testing.T) {
39+
tmpRoot := t.TempDir()
40+
t.Setenv("TMPDIR", tmpRoot)
41+
42+
var logs bytes.Buffer
43+
klog.LogToStderr(false)
44+
klog.SetOutput(&logs)
45+
t.Cleanup(func() {
46+
klog.SetOutput(os.Stderr)
47+
klog.LogToStderr(true)
48+
})
49+
50+
w := &WrappedNodeService{mounter: mount.NewFakeMounter(nil)}
51+
req := &csi.NodePublishVolumeRequest{
52+
VolumeId: "pvc-1",
53+
TargetPath: filepath.Join(t.TempDir(), "target"),
54+
PublishContext: map[string]string{nfsExportKey: "nfs://192.0.2.10:2049/export"},
55+
}
56+
if _, err := w.NodePublishVolume(context.Background(), req); err != nil {
57+
t.Fatalf("NodePublishVolume: %v", err)
58+
}
59+
klog.Flush()
60+
61+
leftovers, err := filepath.Glob(filepath.Join(tmpRoot, "nfs-init-pvc-1-*"))
62+
if err != nil || len(leftovers) != 1 {
63+
t.Fatalf("expected one leftover temp dir, got %v (err %v)", leftovers, err)
64+
}
65+
for _, line := range strings.Split(logs.String(), "\n") {
66+
if strings.HasPrefix(line, "W") && strings.Contains(line, leftovers[0]) {
67+
return
68+
}
69+
}
70+
t.Errorf("removal failure of %s was not logged as a warning; log output:\n%s", leftovers[0], logs.String())
71+
}

0 commit comments

Comments
 (0)