Currently, CloudNativePG has two mutually exclusive bootstrap methods for a new cluster: initdb (for a new empty cluster) and recovery (for a cluster restored from a backup or another live cluster). Both methods assume the cluster is being created "from scratch," even if it's meant to replace an existing one (e.g., for PITR following a data loss scenario).

This "from scratch" methodology is not an accident; it's a direct result of the project's history and prioritised growth. We have heavily favoured the "cattle, not pets" model, where any cluster can be terminated and recreated from scratch. This was, and still is, a powerful and robust approach for initial provisioning and basic scaling.

However, as CloudNativePG matures and our users deploy it in more complex, stateful scenarios (like strict GitOps workflows and demanding business continuity), the limitations of a purely cattle-based model are becoming clearer. A cluster that has existed for some time shouldn't be treated the same as a brand-new one, especially during a Point-in-Time Recovery (PITR) operation where its existing data is highly valuable.

This discussion aims to explore those specific limitations and propose a more flexible model that blends the best of both worlds.

Current challenges

The hard distinction between initdb and recovery at the cluster's bootstrap phase is the root of a few issues:

1. Lack of idempotency: in a GitOps environment, a Cluster resource should be idempotent. If we need to perform a PITR on an existing cluster, the current process (deleting/recreating the Cluster resource) results in a new cluster from the operator's perspective. This is not idempotent and can break external references, monitoring, and replica cluster definitions that point to the "old" cluster.

2. Inefficient PITR: because a PITR is treated as a "new cluster" creation, we cannot perform an incremental copy or a rewind-like operation on an existing cluster's PVCs. We are forced to re-stream or re-copy the entire dataset, even if the existing PVCs are 99% up-to-date.

3. Code and logic duplication: the recovery process (using pg_basebackup from a live source) is functionally identical to the process for scaling up by adding a new replica. We have duplicated this "clone from primary" logic in two different parts of the operator's workflow (bootstrap vs. scaling).

4. Inflexible scaling strategy: scaling from 0 nodes (bootstrap) and scaling from 1 node (adding a replica) should be part of the same conceptual process. However, we currently have no control over how a new replica is created. It's always a pg_basebackup from the primary. We should be able to decide the source for any new instance (primary or replica).

Proposal of a unified "Instance Provisioning" model

This discussion aims to brainstorm a unified provisioning model, where creating any instance (whether it's the first, a new primary, or a replica) is driven by the same unified logic. This implies we should move away from the rigid, mutually-exclusive bootstrap section.

In this new model, the operator's primary job would be to ensure a Pod and its PVC exist and have the correct data. The source of that data should be a flexible, configurable strategy defined directly in the Cluster spec.

Here are a few starting points to seed the discussion (this list is not exhaustive):

• Handling existing dstate:

  • What to do with the content of existing PVCs (e.g., enabling incremental recovery, pg_rewind, or re-use).

• Defining the data source:

  • The relationship with external clusters (e.g., standardising replica cluster creation as just another data source).
  • The notion of a recovery source (which could be a backup archive, an external cluster's backup, or a live cluster).

• Provisioning methods & capabilities:

  • Storage class capabilities (like volume snapshots and PVC cloning) as a primary provisioning method.
  • Plugin capabilities (for custom restore or provisioning logic).
  • The notion of a backup destination (local WAL archive vs. a full base backup catalog) and how it's used.

...and more.

Let the discussion begin.