Skip to content

[Bug]: Fencing is unreachable when WAL disk is full #10299

Description

@mnencia

Is there an existing issue already for this bug?

  • I have searched for an existing issue, and could not find anything. I believe this is a new bug.

I have read the troubleshooting guide

  • I have read the troubleshooting guide and I think this is a new bug.

I am running a supported version of CloudNativePG

  • I have read the troubleshooting guide and I think this is a new bug.

Version

trunk (main)

What version of Kubernetes are you using?

1.32

What is your Kubernetes environment?

Other

How did you install the operator?

Other

What happened?

When the WAL disk is full, the instance manager performs a disk space check at cmd.go:182-189 and calls os.Exit(4) before the controller-runtime manager starts. Because the manager never starts, the reconciler that processes fencing annotations never runs. This means fencing cannot be applied to instances with a full WAL disk, making it impossible to recover the cluster through fencing.

Expected behavior: The instance manager should always start the controller-runtime manager so that fencing annotations are processed regardless of WAL disk space. The WAL disk space check should happen later, before starting PostgreSQL, not before starting the manager.

Root cause: In internal/cmd/manager/instance/run/cmd.go, the WAL space check runs as a pre-start gate before mgr.Start(). The fencing logic lives inside the lifecycle's runPostgresAndWait, which is only reachable after the manager starts.

Code of Conduct

  • I agree to follow this project's Code of Conduct

Activity

  1. self-assigned this
    on Mar 13, 2026
  2. added theissue type on Mar 13, 2026
  3. added 2 commits that reference this issue on Mar 16, 2026
    19aa54c
  4. added a commit that references this issue on Mar 20, 2026
    1cdd9e3
  5. removed their assignment
    on Mar 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bug 🐛Something isn't working

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions