Skip to content

fix(walrestore): allow history files when cluster timeline is unset - #10818

Merged
mnencia merged 3 commits into
cloudnative-pg:mainfrom
dennispidun:fix/bootstrap-recovery-timeline-history-guard
Jun 19, 2026
Merged

mnencia merged 3 commits into
cloudnative-pg:mainfrom
dennispidun:fix/bootstrap-recovery-timeline-history-guard

Conversation

@dennispidun

@dennispidun dennispidun commented Jun 1, 2026 •

Copy link
Copy Markdown
Contributor

During bootstrap recovery from an external archive, the recovering pod is not yet CurrentPrimary or TargetPrimary and cluster.Status.TimelineID is 0, because no primary has promoted and reported its timeline. The existing guard (fileTimeline > clusterTimeline) therefore blocks every .history file, causing PostgreSQL to stop recovery at the basebackup's timeline and silently drop transactions on newer timelines.

Add an early-return exemption when clusterTimeline == 0: the split-brain scenario the guard protects against requires an active primary, which cannot exist when TimelineID is unset. Allowing history files in this state lets PostgreSQL walk forward across all timeline switches during bootstrap recovery.

This PR also adds an e2e test that restores across a post-switchover timeline and verifies data written on the newer timeline is recovered.

Closes #10442
Closes #10817

Assisted-by: Claude Opus 4.8

@dennispidun
dennispidun requested a review from a team June 1, 2026 10:46
@dosubot dosubot Bot added the size:XS This PR changes 0-9 lines, ignoring generated files. label Jun 1, 2026
@cnpg-bot cnpg-bot added backport-requested ◀️ This pull request should be backported to all supported releases release-1.25 release-1.28 labels Jun 1, 2026
@github-actions

github-actions Bot commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

❗ By default, the pull request is configured to backport to all release branches.

  • To stop backporting this pr, remove the label: backport-requested ◀️ or add the label 'do not backport'
  • To stop backporting this pr to a certain release branch, remove the specific branch label: release-x.y

@dennispidun

Copy link
Copy Markdown
Contributor Author

To verify, I deployed this on an AKS cluster with my usual setup and replaced the -full-recoverys' bootstrap-controller initContainer with kyverno so that my change is only available in the critical path for this issue. Now, my backup data is recoverable, even with timeline changes in-between.

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: cnpg-full-recovery-patched-manager
spec:
  background: false
  rules:
    - name: patch-bootstrap-controller-image
      match:
        any:
          - resources:
              kinds: [Pod]
              selector:
                matchLabels:
                  cnpg.io/jobRole: full-recovery
      mutate:
        patchStrategicMerge:
          spec:
            imagePullSecrets:
              - name: pull-secret
            initContainers:
              - (name): bootstrap-controller
                image: cloudnative-pg:1.28.1-full-recovery-fix

(One needs to build the image with the patch and host it in some registry somewhere)

As im not an active developer of cnpg, im not quite sure how/what to validate additionally to bring it to main. So I would appreciate any further help here.

@dennispidun
dennispidun force-pushed the fix/bootstrap-recovery-timeline-history-guard branch from d48a2a8 to 0fb7a47 Compare June 3, 2026 10:31
@magnus-longva-bouvet

Copy link
Copy Markdown

@dennispidun , great work on and finding and documenting this bug. Can confirm it affects me as well.

Do you know which steps are required to produce an error, like in issue 10442, and which steps are required to produce a silent truncation, like you document in this issue? The reproduction steps look so similar.

@dennispidun

Copy link
Copy Markdown
Contributor Author

@dennispidun , great work on and finding and documenting this bug. Can confirm it affects me as well.

Do you know which steps are required to produce an error, like in issue 10442, and which steps are required to produce a silent truncation, like you document in this issue? The reproduction steps look so similar.

It all depends on the processes your cluster went through. You could follow these steps (unfortunately I don't have a fully minimal cluster spec setup for you, but it should be easily applicable if you have worked with cnpg before):

  1. setup a database cluster (>1 nodes) with backup/wal archiving
  2. after initdb has run create a base backup (that's important, without any, your cluster is also not recoverable)
  3. putting in some data, a table and a row is sufficient
  4. force WAL Switch with: SELECT pg_switch_wal();
  5. promote one of your secondaries, eg with the cnpg plugin, it has a promote subcommand
  6. Put more rows to the table and force a Wal Switch again

You should see that under the wals path there are some *.history files and 000001 and 000002 folders, these are the timelines, the numbers are hex values for the timeline id. You can also observe the current timeline in the clusters status section.

Now, when creating a new cluster with recovery bootstrap options targeting this archive location, you can select a targetTime, which can land in timeline 1 or a targetTime in timeline 2. if you select one in timeline 2 (the one after your second row writes) it will fail, because Postgres never sees the history files and cannot follow the timeline switch. Basically some tooling from cnpg (where I did apply the fix) is refusing to download and provide the correct files, to potentially prevent split brain scenarios in active clusters.

Here, the recovery will fail, because the targetTime can not be reached. Another interesting case appears to happen when you not specify a targetTime which in theory should do a full recovery of that archived data. Here, again Postgres cannot follow the timeline switch because it has no access to the history files.
Here it's silently truncating archived data from timeline 2 which is even worse, because it thinks it has everything, it reports green.

If you now perform a full base backup on the source after the timeline switch, you would temporarily fix the second case, as it will use that backup to restore your cluster. But everything in between timeline 1 and 2 is basically not restorable.

Do you know a way to bring more attention to this issue, as I think it is really bad if you are not able to perform PITR or have silent data loss when performing full recoveries.
Unfortunately I do not have the full picture of CNPG and every component in it, that's why I for myself took the safe route to only patch where it's necessary, in this case inside the full recovery containers.
It's dirty, creates maintenance effort and is not really what I would like to deal with. But with a few hundred clusters I saw it as the only true fix for it.

Maybe you also know how to further validate and help bringing in this fix at the right place. Maybe there is more to do, maybe it's not fully correct. I was worrying a bit that it might be not what we really want because there was a test named almost like the description of this bug, which was weird for me, you can check it out in the changes.

@magnus-longva-bouvet

Copy link
Copy Markdown

Thanks for the detalied answer @dennispidun .

I can reliably reproduce the behaviour you describe in #10817, where recovery fails silently. However, I can not reproduce the behaviour which @pzeinlinger describes in issue #10442, where recovery fails loudly (I believe I did observe loud failures during a disaster recovery event last week, but I can't reproduce it). Both you and @pzeinlinger state that "latest" recovery target will reproduce the issue, but I only get silent failures so far in my experiments.

I'll make a post about it on the CNPG-channel(s) on the CNCF Slack. You can join here if you're not a member already. https://communityinviter.com/apps/cloud-native/cncf

@dosubot dosubot Bot added size:S This PR changes 10-29 lines, ignoring generated files. and removed size:XS This PR changes 0-9 lines, ignoring generated files. labels Jun 18, 2026
@leonardoce
leonardoce force-pushed the fix/bootstrap-recovery-timeline-history-guard branch from 3dfbbf2 to 3d722a6 Compare June 18, 2026 13:02
@dosubot dosubot Bot added the lgtm This PR has been approved by a maintainer label Jun 18, 2026
@leonardoce
leonardoce force-pushed the fix/bootstrap-recovery-timeline-history-guard branch from 3d722a6 to 42c18dc Compare June 18, 2026 13:05
@leonardoce

Copy link
Copy Markdown
Contributor

/test

@github-actions

Copy link
Copy Markdown
Contributor

@dennispidun

Copy link
Copy Markdown
Contributor Author

@magnus-longva-bouvet thanks for bringing it up in the slack channel. I will definitely join that space.

If anything needed from my side, let me know. Otherwise, thank you very much!

@cnpg-bot cnpg-bot added the ok to merge 👌 This PR can be merged label Jun 18, 2026
@gbartolini
gbartolini force-pushed the fix/bootstrap-recovery-timeline-history-guard branch from 42c18dc to 19b7671 Compare June 18, 2026 22:44
@gbartolini

Copy link
Copy Markdown
Contributor

@dennispidun we need your DCO in order to merge the patch. Can you please resubmit your patch or respond here that you agree on me adding to the commit message this line when I merge?

Signed-off-by: Dennis Pidun <[email protected]>

@gbartolini gbartolini added the do not merge 🙅 This PR cannot be merged (yet) label Jun 18, 2026
@gbartolini

Copy link
Copy Markdown
Contributor

Adding "do-not-merge" label due to missing DCO

@gbartolini gbartolini changed the title fix(walrestore): allow history files during bootstrap recovery when cluster timeline is unset fix(walrestore): allow history files when cluster timeline is unset Jun 18, 2026
@gbartolini

Copy link
Copy Markdown
Contributor

/test

@github-actions

Copy link
Copy Markdown
Contributor

@gbartolini

Copy link
Copy Markdown
Contributor

Tests are all green, we just need the DCO now.

@leonardoce
leonardoce force-pushed the fix/bootstrap-recovery-timeline-history-guard branch from 0386491 to 808287f Compare June 19, 2026 07:12
@dennispidun
dennispidun force-pushed the fix/bootstrap-recovery-timeline-history-guard branch 3 times, most recently from 808287f to 876a62f Compare June 19, 2026 07:25
dennispidun and others added 3 commits June 19, 2026 13:59
…luster timeline is unset

During bootstrap recovery from an external archive, the recovering pod is not
yet CurrentPrimary or TargetPrimary (status not yet set, or local cache stale),
and cluster.Status.TimelineID is 0 because no primary has promoted and reported
its timeline. The existing guard (fileTimeline > clusterTimeline) therefore
blocks every .history file, causing PostgreSQL to stop recovery at the
basebackup's timeline and silently drop transactions on newer timelines.

Add an early-return exemption when clusterTimeline == 0: the split-brain
scenario the guard protects against requires an active primary, which cannot
exist when TimelineID is unset. Allowing history files in this state lets
PostgreSQL walk forward across all timeline switches during bootstrap recovery.

Fixes cloudnative-pg#10442, fixes cloudnative-pg#10817

Signed-off-by: Dennis Pidun <[email protected]>
Signed-off-by: Leonardo Cecchi <[email protected]>
…g it

The bootstrap-recovery e2e test deletes the first restored cluster and later
recreates a cluster with the same name. DeleteResourcesFromFile is
fire-and-forget, so the recreated cluster could race the teardown and adopt
stale PVCs instead of performing a fresh restore from the object store,
masking or breaking the post-switchover assertions.

Wait for both the Cluster object and its PVCs to be fully removed before
continuing. Also clarify the clusterTimeline == 0 guard comment to drop the
speculative stale-cache wording and state the safety argument directly.

Assisted-by: Claude Opus 4.8

Signed-off-by: Gabriele Bartolini <[email protected]>
@mnencia
mnencia force-pushed the fix/bootstrap-recovery-timeline-history-guard branch from 876a62f to f6b07ae Compare June 19, 2026 11:59
@mnencia mnencia removed the do not merge 🙅 This PR cannot be merged (yet) label Jun 19, 2026
@mnencia
mnencia merged commit d149135 into cloudnative-pg:main Jun 19, 2026
33 checks passed
cnpg-bot pushed a commit that referenced this pull request Jun 19, 2026
…10818)

During bootstrap recovery from an external archive, the recovering pod
is not yet `CurrentPrimary` or `TargetPrimary` and
`cluster.Status.TimelineID` is 0, because no primary has promoted and
reported its timeline. The existing guard (`fileTimeline >
clusterTimeline`) therefore blocks every `.history` file, causing
PostgreSQL to stop recovery at the basebackup's timeline and silently
drop transactions on newer timelines.

Add an early-return exemption when `clusterTimeline == 0`: the
split-brain scenario the guard protects against requires an active
primary, which cannot exist when `TimelineID` is unset. Allowing history
files in this state lets PostgreSQL walk forward across all timeline
switches during bootstrap recovery.

This PR also adds an e2e test that restores across a post-switchover
timeline and verifies data written on the newer timeline is recovered.

Closes #10442
Closes #10817

Assisted-by: Claude Opus 4.8

Signed-off-by: Dennis Pidun <[email protected]>
Signed-off-by: Leonardo Cecchi <[email protected]>
Signed-off-by: Gabriele Bartolini <[email protected]>
Co-authored-by: Leonardo Cecchi <[email protected]>
Co-authored-by: Gabriele Bartolini <[email protected]>
(cherry picked from commit d149135)
cnpg-bot pushed a commit that referenced this pull request Jun 19, 2026
…10818)

During bootstrap recovery from an external archive, the recovering pod
is not yet `CurrentPrimary` or `TargetPrimary` and
`cluster.Status.TimelineID` is 0, because no primary has promoted and
reported its timeline. The existing guard (`fileTimeline >
clusterTimeline`) therefore blocks every `.history` file, causing
PostgreSQL to stop recovery at the basebackup's timeline and silently
drop transactions on newer timelines.

Add an early-return exemption when `clusterTimeline == 0`: the
split-brain scenario the guard protects against requires an active
primary, which cannot exist when `TimelineID` is unset. Allowing history
files in this state lets PostgreSQL walk forward across all timeline
switches during bootstrap recovery.

This PR also adds an e2e test that restores across a post-switchover
timeline and verifies data written on the newer timeline is recovered.

Closes #10442
Closes #10817

Assisted-by: Claude Opus 4.8

Signed-off-by: Dennis Pidun <[email protected]>
Signed-off-by: Leonardo Cecchi <[email protected]>
Signed-off-by: Gabriele Bartolini <[email protected]>
Co-authored-by: Leonardo Cecchi <[email protected]>
Co-authored-by: Gabriele Bartolini <[email protected]>
(cherry picked from commit d149135)
mnencia pushed a commit that referenced this pull request Jun 19, 2026
…10818)

During bootstrap recovery from an external archive, the recovering pod
is not yet `CurrentPrimary` or `TargetPrimary` and
`cluster.Status.TimelineID` is 0, because no primary has promoted and
reported its timeline. The existing guard (`fileTimeline >
clusterTimeline`) therefore blocks every `.history` file, causing
PostgreSQL to stop recovery at the basebackup's timeline and silently
drop transactions on newer timelines.

Add an early-return exemption when `clusterTimeline == 0`: the
split-brain scenario the guard protects against requires an active
primary, which cannot exist when `TimelineID` is unset. Allowing history
files in this state lets PostgreSQL walk forward across all timeline
switches during bootstrap recovery.

This PR also adds an e2e test that restores across a post-switchover
timeline and verifies data written on the newer timeline is recovered.

Closes #10442
Closes #10817

Assisted-by: Claude Opus 4.8

Signed-off-by: Dennis Pidun <[email protected]>
Signed-off-by: Leonardo Cecchi <[email protected]>
Signed-off-by: Gabriele Bartolini <[email protected]>
Co-authored-by: Leonardo Cecchi <[email protected]>
Co-authored-by: Gabriele Bartolini <[email protected]>
(cherry picked from commit d149135)
Signed-off-by: Marco Nenciarini <[email protected]>
mnencia added a commit that referenced this pull request Jun 19, 2026
Adds the 1.30.0-rc1 release note for #10818.

Signed-off-by: Marco Nenciarini <[email protected]>
renovate Bot added a commit to sdwilsh/ansible-playbooks that referenced this pull request Jul 7, 2026
##### [\`v1.29.2\`](https://github.com/cloudnative-pg/cloudnative-pg/releases/tag/v1.29.2)

**Release date:** Jun 29, 2026

##### Important changes

- Updated the deprecation notice for native (in-tree) Barman Cloud support to reflect that it will now be removed in CloudNativePG 1.31.0, rather than 1.30.0. Users are still encouraged to migrate to the Barman Cloud Plugin. ([#11083](cloudnative-pg/cloudnative-pg#11083)) <!-- 1.29 1.28 -->

- The `cluster` reference is now immutable on the `Database`, `Pooler`, `Publication`, `Subscription`, and `ScheduledBackup` resources. Pointing one of these objects at a different cluster has no well-defined semantics and previously left the controllers in an inconsistent state; the update is now rejected at the API server via a CEL validation rule. ([#10743](cloudnative-pg/cloudnative-pg#10743)) <!-- 1.29 1.28 1.25 -->

##### Enhancements

- Enabled `pg_upgrade` in-place major upgrades to PostgreSQL 19 or later for clusters that use Image Volume extensions, building on the extension-path support added to `pg_upgrade` in PostgreSQL 19. During the upgrade `Job`, the source- and target-version extension images are mounted side by side, so the old server keeps its libraries and a failed upgrade reverts cleanly. ([#10366](cloudnative-pg/cloudnative-pg#10366)) <!-- 1.29 -->

- Added a label selector to the `Cluster` scale subresource (`status.selector`), making a `Cluster` a valid `targetRef` for the Vertical Pod Autoscaler (VPA) and Horizontal Pod Autoscaler (HPA), which can now map a `Cluster` to its instance pods. Contributed by [@sebv004](https://github.com/sebv004). ([#8996](cloudnative-pg/cloudnative-pg#8996)) <!-- 1.29 1.28 1.25 -->

- The operator now emits a `Warning` `PrimaryStatusCheckFailed` event on the `Cluster` when the primary pod is `Ready` from the kubelet perspective but the operator's `/pg/status` check fails and failover is deferred, giving users visibility into the deferral via `kubectl describe cluster`. ([#10509](cloudnative-pg/cloudnative-pg#10509)) <!-- 1.29 1.28 1.25 -->

- The operator now reloads a CNPG-i plugin automatically when its pods are rolled: it watches the `EndpointSlices` backing plugin `Services` and re-enqueues every cluster using the plugin once the new pods become `Ready`, so an upgraded plugin is picked up without waiting for the next resync. ([#10836](cloudnative-pg/cloudnative-pg#10836)) <!-- 1.29 1.28 1.25 -->

##### Security and Supply Chain

- **`CVE-2026-55769` / `GHSA-x8c2-3p4r-v9r6`: `search_path` pinning on operator-issued connections**: a database owner could plant overloaded built-in operators in the `public` schema and alter the `search_path` so that operator introspection probes, running as the cluster superuser, resolved those overloads before `pg_catalog`, a `CWE-426` privilege-escalation chain (same class as `CVE-2018-1058`) that could lead to in-pod RCE via `COPY ... FROM PROGRAM`. The operator now pins `search_path = pg_catalog, public, pg_temp` on every pooled connection so it ships in the startup message and takes precedence over tenant-controlled defaults. ([#10774](cloudnative-pg/cloudnative-pg#10774), [`GHSA-x8c2-3p4r-v9r6`](GHSA-x8c2-3p4r-v9r6)) <!-- 1.29 1.28 1.25 -->

- **`CVE-2026-55765` / `GHSA-w3gf-xc94-wvmj`: operator-side SCRAM-SHA-256 password encoding**: the operator now SCRAM-SHA-256 encodes cleartext role passwords before issuing `CREATE`/`ALTER ROLE ... PASSWORD`, so the literal PostgreSQL parses (and that extensions such as `pg_stat_statements` or `pgaudit` may capture) is the SCRAM verifier rather than the cleartext secret. Pre-hashed (MD5 or SCRAM) values are forwarded unchanged, and the per-Secret annotation `cnpg.io/passwordPassthrough: "enabled"` opts out. ([#10724](cloudnative-pg/cloudnative-pg#10724), [`GHSA-w3gf-xc94-wvmj`](GHSA-w3gf-xc94-wvmj)) <!-- 1.29 1.28 1.25 -->

##### Changes

- Added support for Kubernetes 1.36. ([#10900](cloudnative-pg/cloudnative-pg#10900)) <!-- 1.29 1.28 -->

- Updated the default PostgreSQL version to 18.4. ([#10719](cloudnative-pg/cloudnative-pg#10719)) <!-- 1.29 1.28 1.25 -->

- Updated the Kubernetes versions used to test the operator on public cloud providers. ([#10720](cloudnative-pg/cloudnative-pg#10720), <!-- 1.29 1.28 1.25 --> [#10563](cloudnative-pg/cloudnative-pg#10563), <!-- 1.29 1.28 1.25 --> [#11033](cloudnative-pg/cloudnative-pg#11033)) <!-- 1.29 1.28 1.25 -->

##### Fixes

- Fixed `spec.postgresql.parameters` accepting keys that are not valid PostgreSQL parameter names, which could inject arbitrary directives into `postgresql.conf`; key names are now validated by the webhook. ([#11029](cloudnative-pg/cloudnative-pg#11029)) <!-- 1.29 1.28 1.25 -->

- Fixed declarative `Database`, `Publication`, and `Subscription` objects reporting a stale primary-side status forever after their cluster was demoted to a replica; the controller now re-checks the replica condition and watches the `Cluster` so a demotion is detected promptly. ([#10871](cloudnative-pg/cloudnative-pg#10871)) <!-- 1.29 1.28 1.25 -->

- Fixed non-sequential pod names (for example `-1`, `-3`) caused by the instance serial counter being advanced before the corresponding `Job` and PVCs were created; the bump is now persisted only after those resources exist. ([#10491](cloudnative-pg/cloudnative-pg#10491)) <!-- 1.29 1.28 1.25 -->

- Fixed a switchover deadlock when a WAL-archiver plugin was enabled on an existing cluster: with `primaryUpdateMethod: switchover` the primary could not be rolled out because a clean demotion needs the archiver sidecar that is still missing. The operator now recreates the primary Pod in place so the sidecar is injected and archiving resumes. The check also covers plugins that inject the archiver as a native sidecar (an init container with `restartPolicy: Always`), such as the Barman Cloud plugin. ([#11032](cloudnative-pg/cloudnative-pg#11032), <!-- 1.29 1.28 1.25 --> [#11059](cloudnative-pg/cloudnative-pg#11059)) <!-- 1.29 1.28 1.25 -->

- Fixed a cluster staying in `Setting up primary` indefinitely when the instance-creation Job exhausted its backoff limit; the operator now detects the terminal Job failure and marks the cluster unrecoverable, naming the failed Job and pointing to its logs. ([#11035](cloudnative-pg/cloudnative-pg#11035)) <!-- 1.29 1.28 1.25 -->

- Fixed deletion of a `Database`, `Publication`, or `Subscription` getting stuck in `Terminating` on a replica cluster, where the replica gate ran before the finalizer reconciler and the finalizer was never released. On a replica the PostgreSQL object is left to the primary cluster. ([#10853](cloudnative-pg/cloudnative-pg#10853)) <!-- 1.29 1.28 1.25 -->

- Fixed a conflicting duplicate `Database` or `Subscription` with a `delete` reclaim policy dropping the PostgreSQL object owned by the surviving CR; the drop is now gated on a recorded reconciliation. ([#10870](cloudnative-pg/cloudnative-pg#10870)) <!-- 1.29 1.28 1.25 -->

- Fixed the `postgres` superuser being left locked out after superuser access was disabled and then re-enabled, because the cached secret version was not invalidated and the password was never re-applied. Diagnosed by [@mhartmann-jaconi](https://github.com/mhartmann-jaconi). ([#10834](cloudnative-pg/cloudnative-pg#10834)) <!-- 1.29 1.28 1.25 -->

- Fixed backups getting stuck in the `started` phase when the instance manager running them was restarted (for example by the in-place upgrade following an operator upgrade) before the backup reached `running`; the reconciliation is now rescheduled so the lost session is detected. ([#10859](cloudnative-pg/cloudnative-pg#10859)) <!-- 1.29 1.28 -->

- Fixed `exec`/`attach` streaming to negotiate WebSocket with a SPDY fallback, restoring compatibility both with Kubernetes versions that have removed SPDY and with platforms such as OpenShift that reject WebSocket exec upgrades. Contributed by [@bartscheers](https://github.com/bartscheers). ([#10876](cloudnative-pg/cloudnative-pg#10876), [#10933](cloudnative-pg/cloudnative-pg#10933)) <!-- 1.29 -->

- Fixed resource leaks when concurrent `Backup` objects raced: backups now run in strict creation-time order, so an already-executing backup is never preempted by a newer one and its replication slot and PostgreSQL session are no longer orphaned on the primary. Contributed by [@GabriFedi97](https://github.com/GabriFedi97). ([#10747](cloudnative-pg/cloudnative-pg#10747)) <!-- 1.29 1.28 1.25 -->

- Fixed role reconciliation clearing the password on a PostgreSQL role when the referenced Secret could not be fetched; the role is now left untouched until the Secret becomes available, and per-action errors are aggregated for better visibility. ([#10053](cloudnative-pg/cloudnative-pg#10053)) <!-- 1.29 1.28 1.25 -->

- Fixed a bootstrap failure where a metrics-exporter setup error (commonly a duplicate-key race with the controller) rolled back `streaming_replica` creation and wedged replica joins. The metrics-exporter step now runs in a separate transaction. Contributed by [@BlaiseAntony](https://github.com/BlaiseAntony). ([#10749](cloudnative-pg/cloudnative-pg#10749)) <!-- 1.29 1.28 1.25 -->

- Fixed a `ScheduledBackup` controller loop that occurred when a `Backup` was created but its status patch never landed; the controller now adopts an existing `Backup` for the next iteration instead of looping on `AlreadyExists`. ([#10612](cloudnative-pg/cloudnative-pg#10612)) <!-- 1.29 1.28 1.25 -->

- Fixed a nil-pointer panic when reconciling a `Pooler` whose `Cluster` has been deleted. ([#10667](cloudnative-pg/cloudnative-pg#10667)) <!-- 1.29 1.28 -->

- Fixed bootstrap log handling so that all named log pipes (`postgres`, `postgres.csv`, and `postgres.json`) get consumers during `WithActiveInstance`, preventing regular files from being created in place of the named pipes. ([#10043](cloudnative-pg/cloudnative-pg#10043)) <!-- 1.29 1.28 1.25 -->

- Fixed generation of invalid IPv6 URLs by wrapping the address in square brackets. Contributed by [@Infinoid](https://github.com/Infinoid). ([#10682](cloudnative-pg/cloudnative-pg#10682)) <!-- 1.29 1.28 1.25 -->

- Fixed an external cluster plugin still being treated as active when its configuration set `enabled: false`. ([#10932](cloudnative-pg/cloudnative-pg#10932)) <!-- 1.29 1.28 1.25 -->

- Fixed a race during bootstrap recovery from an object store where the restore job could read a stale `Cluster` (primary not yet recorded and timeline still unset) and have its `.history` files rejected by the split-brain guard. When this happened, recovery stopped at the base backup's timeline and silently dropped transactions committed on later timelines. History files are now allowed while the cluster timeline is unset. Contributed by [@dennispidun](https://github.com/dennispidun). ([#10818](cloudnative-pg/cloudnative-pg#10818)) <!-- 1.29 1.28 1.25 -->

- Fixed a cache race during cluster creation when the server and client CA resolve to the same Secret (the default): a stale informer cache triggered a redundant `Create` that failed with `AlreadyExists` and could leave the cluster stuck in `Unable to create required cluster objects`. The operator now reuses the already-fetched CA Secret when the names match. ([#10989](cloudnative-pg/cloudnative-pg#10989)) <!-- 1.29 1.28 1.25 -->

- Fixed the `pg_basebackup` bootstrap path overwriting or failing on a pre-existing `PGDATA` (for example after a replica Pod restart) by enforcing the same pre-flight directory check already applied by the other bootstrap methods; this also protects statically provisioned PVCs from being silently overwritten. ([#11006](cloudnative-pg/cloudnative-pg#11006)) <!-- 1.29 1.28 1.25 -->

- Fixed the `Cluster` phase flapping between `Healthy` and a plugin-failure phase when a post-reconcile plugin hook returned an error; the `Healthy` phase is now registered as the last step of a successful reconciliation, so a loop that ends in a plugin error never reports `Healthy`. Contributed by [@GabriFedi97](https://github.com/GabriFedi97). ([#10421](cloudnative-pg/cloudnative-pg#10421)) <!-- 1.29 1.28 1.25 -->

- Fixed stale certificate data and partial reads after an external server's Secret was rotated (for example a CA bundle shrinking from two certificates to one): the file is now written atomically, so libpq always reads either the old or the new value, never a mix. Contributed by [@Anand-240](https://github.com/Anand-240). ([#10975](cloudnative-pg/cloudnative-pg#10975)) <!-- 1.29 1.28 1.25 -->

- Fixed plugin connectivity to use the plugin `Service` FQDN instead of its short name, avoiding failures when a cluster-level proxy is automatically injected into pods. Contributed by [@kdautrey](https://github.com/kdautrey). ([#10921](cloudnative-pg/cloudnative-pg#10921)) <!-- 1.29 1.28 1.25 -->

- Fixed excessive operator log noise from the per-request `Cluster` create/update validation webhook messages, now logged at `debug` instead of `info`. ([#10984](cloudnative-pg/cloudnative-pg#10984)) <!-- 1.29 1.28 1.25 -->

- Fixed a first-primary bootstrap deadlock where a status-patch conflict after the data PVC was created but before the initialization Job was started left the orphan Pending PVC counted as an instance, blocking the bootstrap gate; the PVC-state reconciler now recreates the bootstrap Job reusing the assigned serial. ([#11039](cloudnative-pg/cloudnative-pg#11039)) <!-- 1.29 1.28 1.25 -->

- Fixed external cluster names and secret selector references being joined into filesystem paths without validation, letting a `..` component or path separator escape the external secrets directory when the instance manager dumps connection material; these values are now rejected at the validating webhook and re-checked at the write site. Reported by [@r0binak](https://github.com/r0binak). ([#11045](cloudnative-pg/cloudnative-pg#11045)) <!-- 1.29 1.28 1.25 -->

- Fixed a backup getting stuck in `pending` forever: the concurrent-backup gate ran on every reconcile and could overwrite an already-completed phase written asynchronously by the instance manager. The gate now runs only while the backup phase is still unset or `pending`. ([#11056](cloudnative-pg/cloudnative-pg#11056)) <!-- 1.29 -->

- Fixed a declarative `VolumeSnapshot` backup being permanently marked as failed when a stale cache made the operator re-create a snapshot it had already provisioned, failing with `AlreadyExists`. The operator now tolerates the collision when the existing snapshot carries this backup's label and adopts it; a collision with a foreign snapshot still surfaces as an error. ([#11071](cloudnative-pg/cloudnative-pg#11071)) <!-- 1.29 1.28 1.25 -->

- Fixed a volume snapshot backup being discarded on a transient instance-manager connection error (for example a dial timeout from a brief pod-network disruption) during the finalize step, even when its snapshots were already provisioned; such network errors are now retried instead of treated as terminal. ([#11069](cloudnative-pg/cloudnative-pg#11069)) <!-- 1.29 1.28 1.25 -->

- Fixed a replica switchover losing its `status.demotionToken` when a reconcile was requeued between storing the token and cleaning up the transition metadata (for example a cleanup patch failing against a flaky webhook); the empty no-change token is no longer patched back over the stored value. ([#11075](cloudnative-pg/cloudnative-pg#11075)) <!-- 1.29 1.28 1.25 -->

- `cnpg` plugin:

  - Fixed `kubectl cnpg psql` on Windows, where execution relied on a Unix-only system call and failed with "not supported by windows"; Windows now launches `kubectl exec` as a child process. Contributed by [@Utkarsh-sharma47](https://github.com/Utkarsh-sharma47). ([#10972](cloudnative-pg/cloudnative-pg#10972)) <!-- 1.29 1.28 1.25 -->

  - Fixed an unbounded memory leak in `kubectl cnpg logs -f` on busy clusters, where a per-log-group timer was never released; timers are now reused across iterations. Contributed by [@Anand-240](https://github.com/Anand-240). ([#10976](cloudnative-pg/cloudnative-pg#10976)) <!-- 1.29 1.28 1.25 -->
sdwilsh pushed a commit to sdwilsh/ansible-playbooks that referenced this pull request Jul 7, 2026
##### [\`v1.30.0\`](https://github.com/cloudnative-pg/cloudnative-pg/releases/tag/v1.30.0)

**Release date:** Jun 29, 2026

##### Important changes

- Updated the deprecation notice for native (in-tree) Barman Cloud support to reflect that it will now be removed in CloudNativePG 1.31.0, rather than 1.30.0. Users are still encouraged to migrate to the Barman Cloud Plugin. ([#11083](cloudnative-pg/cloudnative-pg#11083)) <!-- 1.29 1.28 -->

- The `cluster` reference is now immutable on the `Database`, `Pooler`, `Publication`, `Subscription`, and `ScheduledBackup` resources. Pointing one of these objects at a different cluster has no well-defined semantics and previously left the controllers in an inconsistent state; the update is now rejected at the API server via a CEL validation rule. ([#10743](cloudnative-pg/cloudnative-pg#10743)) <!-- 1.29 1.28 1.25 -->

##### Features

- **Primary `Lease` for safe primary election**: introduced a Kubernetes `Lease` object (named after the cluster) that acts as a mutex serializing primary promotion: the instance manager must hold the lease before acting as primary and releases it on clean shutdown so replicas can promote without waiting for the full TTL. Timings are configurable via the new `.spec.primaryLease` stanza. The lease is a promotion gate, not a fence. Primary isolation remains responsible for fencing. ([#10627](cloudnative-pg/cloudnative-pg#10627))

- **`DatabaseRole` CRD for declarative role management**: introduced a `DatabaseRole` custom resource that manages a PostgreSQL role as a standalone Kubernetes object, instead of declaring it inline in the `Cluster`'s `.spec.managed.roles` stanza. Each role gets its own lifecycle, status, and RBAC, which suits GitOps workflows and lets role definitions live next to the applications that own them. The spec reuses the same `RoleConfiguration` structure as the inline method, so migrating a role is a matter of moving the stanza into its own manifest. A `databaseRoleReclaimPolicy` field (`retain`, the default, or `delete`) controls what happens to the role when the resource is deleted, mirroring persistent volumes. ([#6155](cloudnative-pg/cloudnative-pg#6155))

- **TLS client certificates for declarative roles**: a `DatabaseRole` can now include a `clientCertificate` block to have the operator automatically generate and renew a TLS client certificate, signed by the cluster's client CA and stored in a `<databaserole-name>-client-cert` Secret. This enables password-free PostgreSQL `cert` authentication; the Secret is cleaned up when the feature is disabled or the `DatabaseRole` is deleted. ([#10896](cloudnative-pg/cloudnative-pg#10896))

- **PgBouncer image management via image catalogs**: the `Pooler` resource can now reference an entry in an `ImageCatalog` or `ClusterImageCatalog` through the new `spec.pgbouncer.imageCatalogRef` field, centralizing PgBouncer image management. When a catalog entry is updated, all referencing `Poolers` are automatically reconciled and roll out the new image without any change to their spec. The resolved image is reported in `status.image`, and a new `status.phase` (`active`, `paused`, `inactive`, or `failed`), also surfaced as a `Phase` column in `kubectl get pooler`, summarizes the lifecycle. ([#10568](cloudnative-pg/cloudnative-pg#10568))

##### Enhancements

- Enabled `pg_upgrade` in-place major upgrades to PostgreSQL 19 or later for clusters that use Image Volume extensions, building on the extension-path support added to `pg_upgrade` in PostgreSQL 19. During the upgrade `Job`, the source- and target-version extension images are mounted side by side, so the old server keeps its libraries and a failed upgrade reverts cleanly. ([#10366](cloudnative-pg/cloudnative-pg#10366)) <!-- 1.29 -->

- Added TLS support for the `Pooler` metrics endpoint via `.spec.monitoring.tls.enabled`. When enabled, the metrics server is served over HTTPS, reusing the certificate and key from `.spec.pgbouncer.clientTLSSecret` and reloading it on every handshake to support rotation without a restart; the generated `PodMonitor` scrapes over `https` accordingly. ([#10466](cloudnative-pg/cloudnative-pg#10466))

- Added a label selector to the `Cluster` scale subresource (`status.selector`), making a `Cluster` a valid `targetRef` for the Vertical Pod Autoscaler (VPA) and Horizontal Pod Autoscaler (HPA), which can now map a `Cluster` to its instance pods. Contributed by [@sebv004](https://github.com/sebv004). ([#8996](cloudnative-pg/cloudnative-pg#8996)) <!-- 1.29 1.28 1.25 -->

- The operator now emits a `Warning` `PrimaryStatusCheckFailed` event on the `Cluster` when the primary pod is `Ready` from the kubelet perspective but the operator's `/pg/status` check fails and failover is deferred, giving users visibility into the deferral via `kubectl describe cluster`. ([#10509](cloudnative-pg/cloudnative-pg#10509)) <!-- 1.29 1.28 1.25 -->

- Added the `ENABLE_WEBHOOK_NAMESPACE_SUFFIX` flag, which suffixes the operator's webhook configuration names with `-<OPERATOR_NAMESPACE>` so that multiple operator instances can coexist on the same cluster. The operator only looks up these configurations; users must create and maintain them. Contributed by [@maxlengdell](https://github.com/maxlengdell). ([#10420](cloudnative-pg/cloudnative-pg#10420)) <!-- 1.29 1.28 1.25 -->

- The operator now reloads a CNPG-i plugin automatically when its pods are rolled: it watches the `EndpointSlices` backing plugin `Services` and re-enqueues every cluster using the plugin once the new pods become `Ready`, so an upgraded plugin is picked up without waiting for the next resync. ([#10836](cloudnative-pg/cloudnative-pg#10836)) <!-- 1.29 1.28 1.25 -->

- Instance serial numbers are now assigned by reusing the lowest free slot among existing instance names, instead of always incrementing a global counter. Pod and PVC names stay stable across instance recreation (for example, an instance recreated after a node drain comes back with the same name), and serials freed by deleted instances are reclaimed. A new `Initialized` cluster condition reports whether the cluster has completed its first bootstrap, and `status.latestGeneratedNode` is deprecated: it is no longer written, but is preserved on the CRD for backward compatibility. ([#10548](cloudnative-pg/cloudnative-pg#10548))

- Defaulting and validation now run during reconciliation as a fallback when admission webhooks are unavailable, or configured to ignore failures, so the operator no longer reconciles invalid or incomplete specs. Missing defaults are applied directly, and validation failures are surfaced in the resource status instead of failing silently later. ([#10874](cloudnative-pg/cloudnative-pg#10874))

##### Security

- **`CVE-2026-55769` / `GHSA-x8c2-3p4r-v9r6`: `search_path` pinning on operator-issued connections**: a database owner could plant overloaded built-in operators in the `public` schema and alter the `search_path` so that operator introspection probes, running as the cluster superuser, resolved those overloads before `pg_catalog`, a `CWE-426` privilege-escalation chain (same class as `CVE-2018-1058`) that could lead to in-pod RCE via `COPY ... FROM PROGRAM`. The operator now pins `search_path = pg_catalog, public, pg_temp` on every pooled connection so it ships in the startup message and takes precedence over tenant-controlled defaults. ([#10774](cloudnative-pg/cloudnative-pg#10774), [`GHSA-x8c2-3p4r-v9r6`](GHSA-x8c2-3p4r-v9r6)) <!-- 1.29 1.28 1.25 -->

- **`GHSA-7qwx-x8ff-3px9`: authenticated operator-to-instance-manager calls**: the instance manager's remote webserver relied on network isolation rather than authentication for its operator-only control endpoints, so any party able to reach the pod's status port could invoke them, disrupting backup orchestration and WAL archival and reading operational metadata. (The upgrade endpoint is SHA-256-pinned, so this did not permit arbitrary code execution.) The operator now generates an in-memory ECDSA P-256 client certificate at startup and reconciles its SHA-256 fingerprint into the cluster status; the instance manager rejects requests to sensitive endpoints that do not present a matching certificate. This hardening is not backported; earlier releases should continue to restrict the status port with a `NetworkPolicy`. ([#10579](cloudnative-pg/cloudnative-pg#10579), [`GHSA-7qwx-x8ff-3px9`](GHSA-7qwx-x8ff-3px9))

- **`CVE-2026-55765` / `GHSA-w3gf-xc94-wvmj`: operator-side SCRAM-SHA-256 password encoding**: the operator now SCRAM-SHA-256 encodes cleartext role passwords before issuing `CREATE`/`ALTER ROLE ... PASSWORD`, so the literal PostgreSQL parses (and that extensions such as `pg_stat_statements` or `pgaudit` may capture) is the SCRAM verifier rather than the cleartext secret. Pre-hashed (MD5 or SCRAM) values are forwarded unchanged, and the per-Secret annotation `cnpg.io/passwordPassthrough: "enabled"` opts out. ([#10724](cloudnative-pg/cloudnative-pg#10724), [`GHSA-w3gf-xc94-wvmj`](GHSA-w3gf-xc94-wvmj)) <!-- 1.29 1.28 1.25 -->

##### Changes

- Added support for Kubernetes 1.36. ([#10900](cloudnative-pg/cloudnative-pg#10900)) <!-- 1.29 1.28 -->

- Updated the default PostgreSQL version to 18.4. ([#10719](cloudnative-pg/cloudnative-pg#10719)) <!-- 1.29 1.28 1.25 -->

- Updated the Kubernetes versions used to test the operator on public cloud providers. ([#10720](cloudnative-pg/cloudnative-pg#10720), <!-- 1.29 1.28 1.25 --> [#10563](cloudnative-pg/cloudnative-pg#10563), <!-- 1.29 1.28 1.25 --> [#11033](cloudnative-pg/cloudnative-pg#11033)) <!-- 1.29 1.28 1.25 -->

##### Fixes

- Fixed declarative `Database`, `Publication`, and `Subscription` objects reporting a stale primary-side status forever after their cluster was demoted to a replica; the controller now re-checks the replica condition and watches the `Cluster` so a demotion is detected promptly. ([#10871](cloudnative-pg/cloudnative-pg#10871)) <!-- 1.29 1.28 1.25 -->

- Fixed non-sequential pod names (for example `-1`, `-3`) caused by the instance serial counter being advanced before the corresponding `Job` and PVCs were created; the bump is now persisted only after those resources exist. ([#10491](cloudnative-pg/cloudnative-pg#10491)) <!-- 1.29 1.28 1.25 -->

- Fixed deletion of a `Database`, `Publication`, or `Subscription` getting stuck in `Terminating` on a replica cluster, where the replica gate ran before the finalizer reconciler and the finalizer was never released. On a replica the PostgreSQL object is left to the primary cluster. ([#10853](cloudnative-pg/cloudnative-pg#10853)) <!-- 1.29 1.28 1.25 -->

- Fixed a conflicting duplicate `Database` or `Subscription` with a `delete` reclaim policy dropping the PostgreSQL object owned by the surviving CR; the drop is now gated on a recorded reconciliation. ([#10870](cloudnative-pg/cloudnative-pg#10870)) <!-- 1.29 1.28 1.25 -->

- Fixed the `postgres` superuser being left locked out after superuser access was disabled and then re-enabled, because the cached secret version was not invalidated and the password was never re-applied. Diagnosed by [@mhartmann-jaconi](https://github.com/mhartmann-jaconi). ([#10834](cloudnative-pg/cloudnative-pg#10834)) <!-- 1.29 1.28 1.25 -->

- Fixed backups getting stuck in the `started` phase when the instance manager running them was restarted (for example by the in-place upgrade following an operator upgrade) before the backup reached `running`; the reconciliation is now rescheduled so the lost session is detected. ([#10859](cloudnative-pg/cloudnative-pg#10859)) <!-- 1.29 1.28 -->

- Fixed `exec`/`attach` streaming to negotiate WebSocket with a SPDY fallback, restoring compatibility both with Kubernetes versions that have removed SPDY and with platforms such as OpenShift that reject WebSocket exec upgrades. Contributed by [@bartscheers](https://github.com/bartscheers). ([#10876](cloudnative-pg/cloudnative-pg#10876), [#10933](cloudnative-pg/cloudnative-pg#10933)) <!-- 1.29 -->

- Fixed resource leaks when concurrent `Backup` objects raced: backups now run in strict creation-time order, so an already-executing backup is never preempted by a newer one and its replication slot and PostgreSQL session are no longer orphaned on the primary. Contributed by [@GabriFedi97](https://github.com/GabriFedi97). ([#10747](cloudnative-pg/cloudnative-pg#10747)) <!-- 1.29 1.28 1.25 -->

- Fixed role reconciliation clearing the password on a PostgreSQL role when the referenced Secret could not be fetched; the role is now left untouched until the Secret becomes available, and per-action errors are aggregated for better visibility. ([#10053](cloudnative-pg/cloudnative-pg#10053)) <!-- 1.29 1.28 1.25 -->

- Fixed a bootstrap failure where a metrics-exporter setup error (commonly a duplicate-key race with the controller) rolled back `streaming_replica` creation and wedged replica joins. The metrics-exporter step now runs in a separate transaction. Contributed by [@BlaiseAntony](https://github.com/BlaiseAntony). ([#10749](cloudnative-pg/cloudnative-pg#10749)) <!-- 1.29 1.28 1.25 -->

- Fixed a `ScheduledBackup` controller loop that occurred when a `Backup` was created but its status patch never landed; the controller now adopts an existing `Backup` for the next iteration instead of looping on `AlreadyExists`. ([#10612](cloudnative-pg/cloudnative-pg#10612)) <!-- 1.29 1.28 1.25 -->

- Fixed a nil-pointer panic when reconciling a `Pooler` whose `Cluster` has been deleted. ([#10667](cloudnative-pg/cloudnative-pg#10667)) <!-- 1.29 1.28 -->

- Fixed bootstrap log handling so that all named log pipes (`postgres`, `postgres.csv`, and `postgres.json`) get consumers during `WithActiveInstance`, preventing regular files from being created in place of the named pipes. ([#10043](cloudnative-pg/cloudnative-pg#10043)) <!-- 1.29 1.28 1.25 -->

- Fixed generation of invalid IPv6 URLs by wrapping the address in square brackets. Contributed by [@Infinoid](https://github.com/Infinoid). ([#10682](cloudnative-pg/cloudnative-pg#10682)) <!-- 1.29 1.28 1.25 -->

- Fixed an external cluster plugin still being treated as active when its configuration set `enabled: false`. ([#10932](cloudnative-pg/cloudnative-pg#10932)) <!-- 1.29 1.28 1.25 -->

- Fixed a race during bootstrap recovery from an object store where the restore job could read a stale `Cluster` (primary not yet recorded and timeline still unset) and have its `.history` files rejected by the split-brain guard. When this happened, recovery stopped at the base backup's timeline and silently dropped transactions committed on later timelines. History files are now allowed while the cluster timeline is unset. Contributed by [@dennispidun](https://github.com/dennispidun). ([#10818](cloudnative-pg/cloudnative-pg#10818)) <!-- 1.29 1.28 1.25 -->

- Fixed a race where deleting an instance's PVCs could leave the instance permanently stuck: if the data PVC was removed while a WAL PVC was still terminating, the operator recreated the instance bound to the terminating volume, leaving the Pod unschedulable and blocking all further reconciliation. The operator now waits for terminating PVCs to be fully removed before recreating or reattaching an instance, and surfaces the wait through a log line and the cluster phase. ([#11017](cloudnative-pg/cloudnative-pg#11017))

- Fixed a cache race during cluster creation when the server and client CA resolve to the same Secret (the default): a stale informer cache triggered a redundant `Create` that failed with `AlreadyExists` and could leave the cluster stuck in `Unable to create required cluster objects`. The operator now reuses the already-fetched CA Secret when the names match. ([#10989](cloudnative-pg/cloudnative-pg#10989)) <!-- 1.29 1.28 1.25 -->

- Fixed the `pg_basebackup` bootstrap path overwriting or failing on a pre-existing `PGDATA` (for example after a replica Pod restart) by enforcing the same pre-flight directory check already applied by the other bootstrap methods; this also protects statically provisioned PVCs from being silently overwritten. ([#11006](cloudnative-pg/cloudnative-pg#11006)) <!-- 1.29 1.28 1.25 -->

- Fixed the `Cluster` phase flapping between `Healthy` and a plugin-failure phase when a post-reconcile plugin hook returned an error; the `Healthy` phase is now registered as the last step of a successful reconciliation, so a loop that ends in a plugin error never reports `Healthy`. Contributed by [@GabriFedi97](https://github.com/GabriFedi97). ([#10421](cloudnative-pg/cloudnative-pg#10421)) <!-- 1.29 1.28 1.25 -->

- Fixed stale certificate data and partial reads after an external server's Secret was rotated (for example a CA bundle shrinking from two certificates to one): the file is now written atomically, so libpq always reads either the old or the new value, never a mix. Contributed by [@Anand-240](https://github.com/Anand-240). ([#10975](cloudnative-pg/cloudnative-pg#10975)) <!-- 1.29 1.28 1.25 -->

- Fixed plugin connectivity to use the plugin `Service` FQDN instead of its short name, avoiding failures when a cluster-level proxy is automatically injected into pods. Contributed by [@kdautrey](https://github.com/kdautrey). ([#10921](cloudnative-pg/cloudnative-pg#10921)) <!-- 1.29 1.28 1.25 -->

- Fixed excessive operator log noise from the per-request `Cluster` create/update validation webhook messages, now logged at `debug` instead of `info`. ([#10984](cloudnative-pg/cloudnative-pg#10984)) <!-- 1.29 1.28 1.25 -->

- Fixed `spec.postgresql.parameters` accepting keys that are not valid PostgreSQL parameter names, which could inject arbitrary directives into `postgresql.conf`; key names are now validated by the webhook. ([#11029](cloudnative-pg/cloudnative-pg#11029)) <!-- 1.29 1.28 1.25 -->

- Fixed a switchover deadlock when a WAL-archiver plugin was enabled on an existing cluster: with `primaryUpdateMethod: switchover` the primary could not be rolled out because a clean demotion needs the archiver sidecar that is still missing. The operator now recreates the primary Pod in place so the sidecar is injected and archiving resumes. The check also covers plugins that inject the archiver as a native sidecar (an init container with `restartPolicy: Always`), such as the Barman Cloud plugin. ([#11032](cloudnative-pg/cloudnative-pg#11032), <!-- 1.29 1.28 1.25 --> [#11059](cloudnative-pg/cloudnative-pg#11059)) <!-- 1.29 1.28 1.25 -->

- Fixed a cluster staying in `Setting up primary` indefinitely when the instance-creation Job exhausted its backoff limit; the operator now detects the terminal Job failure and marks the cluster unrecoverable, naming the failed Job and pointing to its logs. ([#11035](cloudnative-pg/cloudnative-pg#11035)) <!-- 1.29 1.28 1.25 -->

- Fixed a first-primary bootstrap deadlock where a status-patch conflict after the data PVC was created but before the initialization Job was started left the orphan Pending PVC counted as an instance, blocking the bootstrap gate; the PVC-state reconciler now recreates the bootstrap Job reusing the assigned serial. ([#11039](cloudnative-pg/cloudnative-pg#11039)) <!-- 1.29 1.28 1.25 -->

- Fixed external cluster names and secret selector references being joined into filesystem paths without validation, letting a `..` component or path separator escape the external secrets directory when the instance manager dumps connection material; these values are now rejected at the validating webhook and re-checked at the write site. Reported by [@r0binak](https://github.com/r0binak). ([#11045](cloudnative-pg/cloudnative-pg#11045)) <!-- 1.29 1.28 1.25 -->

- Fixed a backup getting stuck in `pending` forever: the concurrent-backup gate ran on every reconcile and could overwrite an already-completed phase written asynchronously by the instance manager. The gate now runs only while the backup phase is still unset or `pending`. ([#11056](cloudnative-pg/cloudnative-pg#11056)) <!-- 1.29 -->

- Fixed a declarative `VolumeSnapshot` backup being permanently marked as failed when a stale cache made the operator re-create a snapshot it had already provisioned, failing with `AlreadyExists`. The operator now tolerates the collision when the existing snapshot carries this backup's label and adopts it; a collision with a foreign snapshot still surfaces as an error. ([#11071](cloudnative-pg/cloudnative-pg#11071)) <!-- 1.29 1.28 1.25 -->

- Fixed a volume snapshot backup being discarded on a transient instance-manager connection error (for example a dial timeout from a brief pod-network disruption) during the finalize step, even when its snapshots were already provisioned; such network errors are now retried instead of treated as terminal. ([#11069](cloudnative-pg/cloudnative-pg#11069)) <!-- 1.29 1.28 1.25 -->

- Fixed a replica switchover losing its `status.demotionToken` when a reconcile was requeued between storing the token and cleaning up the transition metadata (for example a cleanup patch failing against a flaky webhook); the empty no-change token is no longer patched back over the stored value. ([#11075](cloudnative-pg/cloudnative-pg#11075)) <!-- 1.29 1.28 1.25 -->

- `cnpg` plugin:

  - Fixed `kubectl cnpg psql` on Windows, where execution relied on a Unix-only system call and failed with "not supported by windows"; Windows now launches `kubectl exec` as a child process. Contributed by [@Utkarsh-sharma47](https://github.com/Utkarsh-sharma47). ([#10972](cloudnative-pg/cloudnative-pg#10972)) <!-- 1.29 1.28 1.25 -->

  - Fixed an unbounded memory leak in `kubectl cnpg logs -f` on busy clusters, where a per-log-group timer was never released; timers are now reused across iterations. Contributed by [@Anand-240](https://github.com/Anand-240). ([#10976](cloudnative-pg/cloudnative-pg#10976)) <!-- 1.29 1.28 1.25 -->

##### Supported versions

- Kubernetes 1.36, 1.35, and 1.34
- PostgreSQL 18, 17, 16, 15, and 14
  - PostgreSQL 18.4 is the default image
  - PostgreSQL 14 support ends on November 12, 2026

---
##### [\`v1.29.2\`](https://github.com/cloudnative-pg/cloudnative-pg/releases/tag/v1.29.2)

**Release date:** Jun 29, 2026

##### Important changes

- Updated the deprecation notice for native (in-tree) Barman Cloud support to reflect that it will now be removed in CloudNativePG 1.31.0, rather than 1.30.0. Users are still encouraged to migrate to the Barman Cloud Plugin. ([#11083](cloudnative-pg/cloudnative-pg#11083)) <!-- 1.29 1.28 -->

- The `cluster` reference is now immutable on the `Database`, `Pooler`, `Publication`, `Subscription`, and `ScheduledBackup` resources. Pointing one of these objects at a different cluster has no well-defined semantics and previously left the controllers in an inconsistent state; the update is now rejected at the API server via a CEL validation rule. ([#10743](cloudnative-pg/cloudnative-pg#10743)) <!-- 1.29 1.28 1.25 -->

##### Enhancements

- Enabled `pg_upgrade` in-place major upgrades to PostgreSQL 19 or later for clusters that use Image Volume extensions, building on the extension-path support added to `pg_upgrade` in PostgreSQL 19. During the upgrade `Job`, the source- and target-version extension images are mounted side by side, so the old server keeps its libraries and a failed upgrade reverts cleanly. ([#10366](cloudnative-pg/cloudnative-pg#10366)) <!-- 1.29 -->

- Added a label selector to the `Cluster` scale subresource (`status.selector`), making a `Cluster` a valid `targetRef` for the Vertical Pod Autoscaler (VPA) and Horizontal Pod Autoscaler (HPA), which can now map a `Cluster` to its instance pods. Contributed by [@sebv004](https://github.com/sebv004). ([#8996](cloudnative-pg/cloudnative-pg#8996)) <!-- 1.29 1.28 1.25 -->

- The operator now emits a `Warning` `PrimaryStatusCheckFailed` event on the `Cluster` when the primary pod is `Ready` from the kubelet perspective but the operator's `/pg/status` check fails and failover is deferred, giving users visibility into the deferral via `kubectl describe cluster`. ([#10509](cloudnative-pg/cloudnative-pg#10509)) <!-- 1.29 1.28 1.25 -->

- The operator now reloads a CNPG-i plugin automatically when its pods are rolled: it watches the `EndpointSlices` backing plugin `Services` and re-enqueues every cluster using the plugin once the new pods become `Ready`, so an upgraded plugin is picked up without waiting for the next resync. ([#10836](cloudnative-pg/cloudnative-pg#10836)) <!-- 1.29 1.28 1.25 -->

##### Security and Supply Chain

- **`CVE-2026-55769` / `GHSA-x8c2-3p4r-v9r6`: `search_path` pinning on operator-issued connections**: a database owner could plant overloaded built-in operators in the `public` schema and alter the `search_path` so that operator introspection probes, running as the cluster superuser, resolved those overloads before `pg_catalog`, a `CWE-426` privilege-escalation chain (same class as `CVE-2018-1058`) that could lead to in-pod RCE via `COPY ... FROM PROGRAM`. The operator now pins `search_path = pg_catalog, public, pg_temp` on every pooled connection so it ships in the startup message and takes precedence over tenant-controlled defaults. ([#10774](cloudnative-pg/cloudnative-pg#10774), [`GHSA-x8c2-3p4r-v9r6`](GHSA-x8c2-3p4r-v9r6)) <!-- 1.29 1.28 1.25 -->

- **`CVE-2026-55765` / `GHSA-w3gf-xc94-wvmj`: operator-side SCRAM-SHA-256 password encoding**: the operator now SCRAM-SHA-256 encodes cleartext role passwords before issuing `CREATE`/`ALTER ROLE ... PASSWORD`, so the literal PostgreSQL parses (and that extensions such as `pg_stat_statements` or `pgaudit` may capture) is the SCRAM verifier rather than the cleartext secret. Pre-hashed (MD5 or SCRAM) values are forwarded unchanged, and the per-Secret annotation `cnpg.io/passwordPassthrough: "enabled"` opts out. ([#10724](cloudnative-pg/cloudnative-pg#10724), [`GHSA-w3gf-xc94-wvmj`](GHSA-w3gf-xc94-wvmj)) <!-- 1.29 1.28 1.25 -->

##### Changes

- Added support for Kubernetes 1.36. ([#10900](cloudnative-pg/cloudnative-pg#10900)) <!-- 1.29 1.28 -->

- Updated the default PostgreSQL version to 18.4. ([#10719](cloudnative-pg/cloudnative-pg#10719)) <!-- 1.29 1.28 1.25 -->

- Updated the Kubernetes versions used to test the operator on public cloud providers. ([#10720](cloudnative-pg/cloudnative-pg#10720), <!-- 1.29 1.28 1.25 --> [#10563](cloudnative-pg/cloudnative-pg#10563), <!-- 1.29 1.28 1.25 --> [#11033](cloudnative-pg/cloudnative-pg#11033)) <!-- 1.29 1.28 1.25 -->

##### Fixes

- Fixed `spec.postgresql.parameters` accepting keys that are not valid PostgreSQL parameter names, which could inject arbitrary directives into `postgresql.conf`; key names are now validated by the webhook. ([#11029](cloudnative-pg/cloudnative-pg#11029)) <!-- 1.29 1.28 1.25 -->

- Fixed declarative `Database`, `Publication`, and `Subscription` objects reporting a stale primary-side status forever after their cluster was demoted to a replica; the controller now re-checks the replica condition and watches the `Cluster` so a demotion is detected promptly. ([#10871](cloudnative-pg/cloudnative-pg#10871)) <!-- 1.29 1.28 1.25 -->

- Fixed non-sequential pod names (for example `-1`, `-3`) caused by the instance serial counter being advanced before the corresponding `Job` and PVCs were created; the bump is now persisted only after those resources exist. ([#10491](cloudnative-pg/cloudnative-pg#10491)) <!-- 1.29 1.28 1.25 -->

- Fixed a switchover deadlock when a WAL-archiver plugin was enabled on an existing cluster: with `primaryUpdateMethod: switchover` the primary could not be rolled out because a clean demotion needs the archiver sidecar that is still missing. The operator now recreates the primary Pod in place so the sidecar is injected and archiving resumes. The check also covers plugins that inject the archiver as a native sidecar (an init container with `restartPolicy: Always`), such as the Barman Cloud plugin. ([#11032](cloudnative-pg/cloudnative-pg#11032), <!-- 1.29 1.28 1.25 --> [#11059](cloudnative-pg/cloudnative-pg#11059)) <!-- 1.29 1.28 1.25 -->

- Fixed a cluster staying in `Setting up primary` indefinitely when the instance-creation Job exhausted its backoff limit; the operator now detects the terminal Job failure and marks the cluster unrecoverable, naming the failed Job and pointing to its logs. ([#11035](cloudnative-pg/cloudnative-pg#11035)) <!-- 1.29 1.28 1.25 -->

- Fixed deletion of a `Database`, `Publication`, or `Subscription` getting stuck in `Terminating` on a replica cluster, where the replica gate ran before the finalizer reconciler and the finalizer was never released. On a replica the PostgreSQL object is left to the primary cluster. ([#10853](cloudnative-pg/cloudnative-pg#10853)) <!-- 1.29 1.28 1.25 -->

- Fixed a conflicting duplicate `Database` or `Subscription` with a `delete` reclaim policy dropping the PostgreSQL object owned by the surviving CR; the drop is now gated on a recorded reconciliation. ([#10870](cloudnative-pg/cloudnative-pg#10870)) <!-- 1.29 1.28 1.25 -->

- Fixed the `postgres` superuser being left locked out after superuser access was disabled and then re-enabled, because the cached secret version was not invalidated and the password was never re-applied. Diagnosed by [@mhartmann-jaconi](https://github.com/mhartmann-jaconi). ([#10834](cloudnative-pg/cloudnative-pg#10834)) <!-- 1.29 1.28 1.25 -->

- Fixed backups getting stuck in the `started` phase when the instance manager running them was restarted (for example by the in-place upgrade following an operator upgrade) before the backup reached `running`; the reconciliation is now rescheduled so the lost session is detected. ([#10859](cloudnative-pg/cloudnative-pg#10859)) <!-- 1.29 1.28 -->

- Fixed `exec`/`attach` streaming to negotiate WebSocket with a SPDY fallback, restoring compatibility both with Kubernetes versions that have removed SPDY and with platforms such as OpenShift that reject WebSocket exec upgrades. Contributed by [@bartscheers](https://github.com/bartscheers). ([#10876](cloudnative-pg/cloudnative-pg#10876), [#10933](cloudnative-pg/cloudnative-pg#10933)) <!-- 1.29 -->

- Fixed resource leaks when concurrent `Backup` objects raced: backups now run in strict creation-time order, so an already-executing backup is never preempted by a newer one and its replication slot and PostgreSQL session are no longer orphaned on the primary. Contributed by [@GabriFedi97](https://github.com/GabriFedi97). ([#10747](cloudnative-pg/cloudnative-pg#10747)) <!-- 1.29 1.28 1.25 -->

- Fixed role reconciliation clearing the password on a PostgreSQL role when the referenced Secret could not be fetched; the role is now left untouched until the Secret becomes available, and per-action errors are aggregated for better visibility. ([#10053](cloudnative-pg/cloudnative-pg#10053)) <!-- 1.29 1.28 1.25 -->

- Fixed a bootstrap failure where a metrics-exporter setup error (commonly a duplicate-key race with the controller) rolled back `streaming_replica` creation and wedged replica joins. The metrics-exporter step now runs in a separate transaction. Contributed by [@BlaiseAntony](https://github.com/BlaiseAntony). ([#10749](cloudnative-pg/cloudnative-pg#10749)) <!-- 1.29 1.28 1.25 -->

- Fixed a `ScheduledBackup` controller loop that occurred when a `Backup` was created but its status patch never landed; the controller now adopts an existing `Backup` for the next iteration instead of looping on `AlreadyExists`. ([#10612](cloudnative-pg/cloudnative-pg#10612)) <!-- 1.29 1.28 1.25 -->

- Fixed a nil-pointer panic when reconciling a `Pooler` whose `Cluster` has been deleted. ([#10667](cloudnative-pg/cloudnative-pg#10667)) <!-- 1.29 1.28 -->

- Fixed bootstrap log handling so that all named log pipes (`postgres`, `postgres.csv`, and `postgres.json`) get consumers during `WithActiveInstance`, preventing regular files from being created in place of the named pipes. ([#10043](cloudnative-pg/cloudnative-pg#10043)) <!-- 1.29 1.28 1.25 -->

- Fixed generation of invalid IPv6 URLs by wrapping the address in square brackets. Contributed by [@Infinoid](https://github.com/Infinoid). ([#10682](cloudnative-pg/cloudnative-pg#10682)) <!-- 1.29 1.28 1.25 -->

- Fixed an external cluster plugin still being treated as active when its configuration set `enabled: false`. ([#10932](cloudnative-pg/cloudnative-pg#10932)) <!-- 1.29 1.28 1.25 -->

- Fixed a race during bootstrap recovery from an object store where the restore job could read a stale `Cluster` (primary not yet recorded and timeline still unset) and have its `.history` files rejected by the split-brain guard. When this happened, recovery stopped at the base backup's timeline and silently dropped transactions committed on later timelines. History files are now allowed while the cluster timeline is unset. Contributed by [@dennispidun](https://github.com/dennispidun). ([#10818](cloudnative-pg/cloudnative-pg#10818)) <!-- 1.29 1.28 1.25 -->

- Fixed a cache race during cluster creation when the server and client CA resolve to the same Secret (the default): a stale informer cache triggered a redundant `Create` that failed with `AlreadyExists` and could leave the cluster stuck in `Unable to create required cluster objects`. The operator now reuses the already-fetched CA Secret when the names match. ([#10989](cloudnative-pg/cloudnative-pg#10989)) <!-- 1.29 1.28 1.25 -->

- Fixed the `pg_basebackup` bootstrap path overwriting or failing on a pre-existing `PGDATA` (for example after a replica Pod restart) by enforcing the same pre-flight directory check already applied by the other bootstrap methods; this also protects statically provisioned PVCs from being silently overwritten. ([#11006](cloudnative-pg/cloudnative-pg#11006)) <!-- 1.29 1.28 1.25 -->

- Fixed the `Cluster` phase flapping between `Healthy` and a plugin-failure phase when a post-reconcile plugin hook returned an error; the `Healthy` phase is now registered as the last step of a successful reconciliation, so a loop that ends in a plugin error never reports `Healthy`. Contributed by [@GabriFedi97](https://github.com/GabriFedi97). ([#10421](cloudnative-pg/cloudnative-pg#10421)) <!-- 1.29 1.28 1.25 -->

- Fixed stale certificate data and partial reads after an external server's Secret was rotated (for example a CA bundle shrinking from two certificates to one): the file is now written atomically, so libpq always reads either the old or the new value, never a mix. Contributed by [@Anand-240](https://github.com/Anand-240). ([#10975](cloudnative-pg/cloudnative-pg#10975)) <!-- 1.29 1.28 1.25 -->

- Fixed plugin connectivity to use the plugin `Service` FQDN instead of its short name, avoiding failures when a cluster-level proxy is automatically injected into pods. Contributed by [@kdautrey](https://github.com/kdautrey). ([#10921](cloudnative-pg/cloudnative-pg#10921)) <!-- 1.29 1.28 1.25 -->

- Fixed excessive operator log noise from the per-request `Cluster` create/update validation webhook messages, now logged at `debug` instead of `info`. ([#10984](cloudnative-pg/cloudnative-pg#10984)) <!-- 1.29 1.28 1.25 -->

- Fixed a first-primary bootstrap deadlock where a status-patch conflict after the data PVC was created but before the initialization Job was started left the orphan Pending PVC counted as an instance, blocking the bootstrap gate; the PVC-state reconciler now recreates the bootstrap Job reusing the assigned serial. ([#11039](cloudnative-pg/cloudnative-pg#11039)) <!-- 1.29 1.28 1.25 -->

- Fixed external cluster names and secret selector references being joined into filesystem paths without validation, letting a `..` component or path separator escape the external secrets directory when the instance manager dumps connection material; these values are now rejected at the validating webhook and re-checked at the write site. Reported by [@r0binak](https://github.com/r0binak). ([#11045](cloudnative-pg/cloudnative-pg#11045)) <!-- 1.29 1.28 1.25 -->

- Fixed a backup getting stuck in `pending` forever: the concurrent-backup gate ran on every reconcile and could overwrite an already-completed phase written asynchronously by the instance manager. The gate now runs only while the backup phase is still unset or `pending`. ([#11056](cloudnative-pg/cloudnative-pg#11056)) <!-- 1.29 -->

- Fixed a declarative `VolumeSnapshot` backup being permanently marked as failed when a stale cache made the operator re-create a snapshot it had already provisioned, failing with `AlreadyExists`. The operator now tolerates the collision when the existing snapshot carries this backup's label and adopts it; a collision with a foreign snapshot still surfaces as an error. ([#11071](cloudnative-pg/cloudnative-pg#11071)) <!-- 1.29 1.28 1.25 -->

- Fixed a volume snapshot backup being discarded on a transient instance-manager connection error (for example a dial timeout from a brief pod-network disruption) during the finalize step, even when its snapshots were already provisioned; such network errors are now retried instead of treated as terminal. ([#11069](cloudnative-pg/cloudnative-pg#11069)) <!-- 1.29 1.28 1.25 -->

- Fixed a replica switchover losing its `status.demotionToken` when a reconcile was requeued between storing the token and cleaning up the transition metadata (for example a cleanup patch failing against a flaky webhook); the empty no-change token is no longer patched back over the stored value. ([#11075](cloudnative-pg/cloudnative-pg#11075)) <!-- 1.29 1.28 1.25 -->

- `cnpg` plugin:

  - Fixed `kubectl cnpg psql` on Windows, where execution relied on a Unix-only system call and failed with "not supported by windows"; Windows now launches `kubectl exec` as a child process. Contributed by [@Utkarsh-sharma47](https://github.com/Utkarsh-sharma47). ([#10972](cloudnative-pg/cloudnative-pg#10972)) <!-- 1.29 1.28 1.25 -->

  - Fixed an unbounded memory leak in `kubectl cnpg logs -f` on busy clusters, where a per-log-group timer was never released; timers are now reused across iterations. Contributed by [@Anand-240](https://github.com/Anand-240). ([#10976](cloudnative-pg/cloudnative-pg#10976)) <!-- 1.29 1.28 1.25 -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport-requested ◀️ This pull request should be backported to all supported releases lgtm This PR has been approved by a maintainer ok to merge 👌 This PR can be merged release-1.28 size:S This PR changes 10-29 lines, ignoring generated files.

Projects

None yet

6 participants