[Feature]: Surface provisioning failures (quota, admission, scheduling) to Cluster.status.conditions #10733

Is there an existing issue already for this feature request/idea?

• I have searched for an existing issue, and could not find anything. I believe this is a new feature request to be evaluated.

What problem is this feature going to solve? Why should it be added?

Problem

When a Cluster gets stuck in phase "Setting up primary", the actual cause is invisible in the Cluster object. CNPG creates a Job to bootstrap the primary instance, but if Kubernetes cannot create the Job's Pod (ResourceQuota exceeded, admission webhook rejection, node scheduling constraints), the failure is recorded only as Warning / FailedCreate events on the Job object by the job-controller. CNPG does not observe these failures and the Cluster stays in "Setting up primary" indefinitely with a generic reason.

sequenceDiagram
    participant CNPG as CNPG Operator
    participant Cluster as Cluster Object
    participant Job as Job *-initdb
    participant JC as job-controller
    participant K8s as Admission / Scheduler

    CNPG->>Cluster: phase = "Setting up primary"
    CNPG->>Job: create Job ✅
    Note over CNPG: no further observation<br/>of Job's pod creation

    loop every ~10s backoff
        JC->>K8s: create Pod
        K8s-->>JC: 403 exceeded quota / Unschedulable
        JC->>Job: Warning FailedCreate
    end

    Note over Cluster: phase stays "Setting up primary"<br/>phaseReason: "Creating primary instance X"<br/>no indication of the real problem

Observed in the wild:

$ kubectl вescribe cluster my-cluster
...
Status:
  Phase:         Setting up primary
  Phase Reason:  Creating primary instance my-cluster-1
Events:
  Normal  CreatingInstance  cloudnative-pg  Primary instance (initdb)
  # ← nothing about the actual error

$ kubectl get events
Warning  FailedCreate  job/my-cluster-1-initdb  job-controller
  Error creating: pods "..." is forbidden: exceeded quota: quota-requests-cpu,
  requested: requests.cpu=4, used: requests.cpu=30m, limited: requests.cpu=1
  # ← error is here, on a different object

Impact

• Users and platform operators have no way to determine why provisioning is blocked without kubectl get events or kubectl describe job — neither accessible through higher-level platform abstractions (e.g., Helm releases, Crossplane Compositions, GitOps pipelines) that mirror cluster.status
• Support load increases as users report "cluster stuck with no error"
• No programmatic way to detect and alert on stuck provisioning vs normal provisioning

Describe the solution you'd like

Proposed Solution

Add a Provisioning condition to cluster.status.conditions that reflects the actual state of the provisioning lifecycle, including failure reasons surfaced from owned resources.

Target state:

status:
  phase: Setting up primary
  conditions:
    - type: Provisioning
      status: "False"
      reason: QuotaExceeded       # enum: InProgress | QuotaExceeded | AdmissionDenied
                                   #       Unschedulable | PVCProvisioningFailed | Succeeded
      message: "exceeded quota: quota-requests-cpu, requested: requests.cpu=4, limited: requests.cpu=1"
      lastTransitionTime: "2024-01-01T00:00:00Z"

The reason field as an enum allows consumers to act programmatically without string parsing. The message contains the human-readable detail from Kubernetes.

Implementation Approaches

I intentionally leave the implementation choice to maintainers. Two approaches are viable:

A — Proactive (preferred, fully level-triggered):
Before or shortly after Job creation, read ResourceQuota.status in the cluster namespace and compare against the cluster's resource requests. If quota would be exceeded, set Provisioning=False/QuotaExceeded immediately — without relying on Events.

B — Reactive (covers all admission cases):
When the owned Job has status.active=0, status.succeeded=0 for N consecutive reconcile cycles, read Events on the Job with reason=FailedCreate and copy the most recent message to the Cluster condition. This generalises to admission webhooks, LimitRanges, and other pod creation blockers beyond quota.

Note: approach A is more robust (level-triggered, no dependence on Event TTL of ~1h) but approach B covers a broader set of admission failures. A combination may be optimal.

Failure Classes Covered

The existing IsPodUnschedulable() utility in pkg/utils/pod_conditions.go already reads pod.status.conditions[PodScheduled] for the scheduling case — extending its usage to update a Cluster condition would be a small addition.

Describe alternatives you've considered

Alternatives Considered

• Update phaseReason only: Already exists as a string field, but it is overwritten on every phase transition and is not structured (no reason enum, no lastTransitionTime). Not machine-readable.
• Emit Warning Event on Cluster only: Events are ephemeral (default TTL 1h). Useful as a complement but insufficient as the primary status mechanism.
• Document the current behavior: We are happy to contribute a documentation PR describing where to look for errors during provisioning. However, documentation does not eliminate the operational gap for automated systems consuming cluster.status.

Additional context

Version

• CloudNative-PG: 1.28.1
• Kubernetes: 1.31
• Reproduction: create namespace with requests.cpu=1 ResourceQuota, create Cluster with resources.requests.cpu: 4

Backport?

N/A

Are you willing to actively contribute to this feature?

No

Code of Conduct

• I agree to follow this project's Code of Conduct