Skip to content

[Bug]: PVC with FileSystemResizePending condition causes pod creation deadlock #9980

Closed

Description

@jmealo

[Bug]: PVC with FileSystemResizePending condition causes pod creation deadlock

Summary

Fixed by #9981

When a PVC has the FileSystemResizePending condition (volume resized at storage layer, filesystem resize pending), CloudNativePG incorrectly classifies it as "resizing" even when no pod is attached. This prevents pod creation, causing a deadlock where the filesystem resize can never complete.

What happened?

After a PVC resize operation completes at the storage layer, Kubernetes sets the FileSystemResizePending condition on the PVC. This condition indicates that the volume has been resized but the filesystem resize is pending - it requires a pod to mount the volume to complete the resize.

Current behavior: CloudNativePG's classifyPVC() function returns resizing for such PVCs, which prevents the operator from creating a new pod. Without a pod to mount the volume, the filesystem resize cannot complete.

Expected behavior: When a PVC has FileSystemResizePending and no pod is attached, it should be classified as dangling to trigger pod recreation. This allows the pod to mount the volume and complete the filesystem resize.

Deadlock sequence

  1. User or dynamic storage triggers a PVC resize
  2. Storage layer completes the volume resize
  3. Kubernetes adds FileSystemResizePending condition to PVC
  4. Pod is deleted or crashes
  5. CloudNativePG sees PVC is "resizing" and does NOT create a new pod
  6. Filesystem resize never completes because no pod mounts the volume
  7. Cluster is stuck - manual intervention required

Reproduction

This can be reproduced on any Kubernetes cluster with a CSI driver that supports volume expansion:

  1. Create a CloudNativePG cluster with a small PVC (e.g., 1Gi)
  2. Resize the PVC to a larger size (e.g., 2Gi)
  3. Delete the pod while the resize is in progress (after storage resize completes but before filesystem resize)
  4. Observe that the pod is never recreated
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: test-resize
spec:
  instances: 1
  storage:
    size: 1Gi
    storageClass: standard  # Must support volume expansion

Environment

  • CloudNativePG Version: All versions (trunk/main)
  • Kubernetes Version: Any with CSI volume expansion support
  • Environment: Any (observed on AKS, likely affects all)
  • Installation Method: Any

Relevant log output

The cluster will show the PVC stuck in resizing state with no pod:

$ kubectl get pvc -l cnpg.io/cluster=test-resize
NAME            STATUS   VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
test-resize-1   Bound    pv-xxx   2Gi        RWO            standard       10m

$ kubectl get pvc test-resize-1 -o jsonpath='{.status.conditions[*].type}'
Resizing FileSystemResizePending

$ kubectl get pods -l cnpg.io/cluster=test-resize
No resources found.

Root cause

In pkg/reconciler/persistentvolumeclaim/status.go, the classifyPVC() function checks isResizing() before checking if a pod exists:

// Current (buggy) logic:
if isResizing(pvc) {
    return resizing  // Returns resizing even if no pod exists
}

if hasPod(pvc, podList) {
    return healthy
}

The fix is to check pod existence first, and only classify as resizing if either:

  1. A pod exists (filesystem resize can complete), OR
  2. No FileSystemResizePending condition (storage resize still in progress)

Proposed fix

Cluster resource

apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
  name: test-resize
spec:
  instances: 1
  storage:
    size: 1Gi
    storageClass: standard  # Must support volume expansion

Relevant log output

$ kubectl get pvc -l cnpg.io/cluster=test-resize
NAME            STATUS   VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
test-resize-1   Bound    pv-xxx   2Gi        RWO            standard       10m

$ kubectl get pvc test-resize-1 -o jsonpath='{.status.conditions[*].type}'
Resizing FileSystemResizePending

$ kubectl get pods -l cnpg.io/cluster=test-resize
No resources found.

Code of Conduct

  • I agree to follow this project's Code of Conduct

Activity

  1. added a commit that references this issue on Feb 21, 2026
    0c0fa23
  2. added 2 commits that reference this issue on Feb 21, 2026
    64ea8ac
    ac6549c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

triagePending triage

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions