Skip to content

[cinder-csi-plugin] CreateContainerError when arbitrary container mountPath contains double quotes #1061

@johannesfrey

Description

@johannesfrey

/kind bug

What happened:

During an update of the cinder-csi-nodeplugin DaemonSet one Pod of the cinder-csi-nodeplugin could not be started again (CreateContainerError) because on that Node there was a Pod running as well that contained double quotes in its mountPath.

A kubectl describe revealed following:

  Warning  Failed     11s                kubelet, kube-node03  Error: failed to generate container "3a0bd914c862e613de04175150aaee6e645983bdbbc9f0f7ecce6af5e4060d20" spec: parsing '/kubelet/pods/a236ebe7-40b7-48b5-8458-ff1321d0ee67/volumes/kubernetes.io~configmap/content/"bar"' failed: unable to unquote root field: invalid syntax

The above error comes directly from containerd in my case. See: https://github.com/containerd/containerd/blob/v1.3.4/mount/mountinfo_linux.go#L90

From a Kubernetes validation point of view using double quotes seems to be allowed, as the resource that uses it can be created without any problems. The cinder-csi-nodeplugin pod on that Node can only be started again when the Pod that references the double quote mount is being deleted.

What you expected to happen:

That the cinder-csi-nodeplugin handles this error more gracefully.

How to reproduce it:

  • Have the cinder-csi-nodeplugin DaemonSet running
  • Deploy follwing test resources:
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: content
    data:
      test: |-
        foo
    ---
    apiVersion: v1
    kind: Pod
    metadata:
      name: busybox
      labels:
        app: busybox
    spec:
      containers:
      - image: busybox
        command:
          - /bin/sh
          - -c
          - "tail -f /dev/null"
        imagePullPolicy: IfNotPresent
        name: busybox
        volumeMounts:
        - name: content
          mountPath: /tmp
        - name: content
          mountPath: '/tmp/"bar"'
          subPath: '"bar"'
      restartPolicy: Always
      volumes:
      - name: content
        configMap:
          name: content
  • Delete the cinder-csi-nodeplugin Pod on the Node where the above Pod has been started
  • After deletion the cinder-csi-nodeplugin tries to start again but remains in CreateContainerError state with the unquote error event mentioned above

Anything else we need to know?:

The cinder-csi-nodeplugin Pods is not affected while it is running when a resource is deployed with a double quote in its mountPath. Only when the cinder-csi-nodeplugin Pod is being deleted/updated the error appears.
The issue was observed in version v1.17.0 but could also be reproduced on version v.1.16.0.

Environment:

  • openstack-cloud-controller-manager(or other related binary) version: v1.17.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugCategorizes issue or PR as related to a bug.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions