Skip to content

Commit e05acca

Browse files
androndoclaude
andcommitted
fix(postgres-operator): bump CloudNativePG to 1.28.1 to fix backup session skew
The chart pinned the operator image to 1.27.3 (values override) while shipping the CNPG CRDs from chart appVersion 1.27.1 — upstream publishes no 1.27.2/1.27.3 chart (the line jumps 0.26.1=1.27.1 -> 0.27.0=1.28.0), so the image bump in 845192b could not carry matching CRDs. The 1.27.3 operator records backup sessions in status.instanceID.sessionID on the Backup CR, but the 1.27.1 CRD does not define that field, so the API server prunes it; the operator then cannot correlate the running backup with the instance session, concludes the instance manager restarted, and fails EVERY backup cluster-wide with 'instance manager was restarted during backup' (native barmanObjectStore and the barman-cloud plugin alike). This blocked #3300 end to end. Bump the vendored cloudnative-pg chart 0.26.1 -> 0.27.1 (appVersion 1.28.1) and drop the image.tag override so the operator, its CRDs and the injected instance manager all move together. 1.28.1 defines status.instanceID.sessionID, so backups validate correctly. Verified on a dev cluster: after the bump the 'instance manager was restarted' failure is gone and plugin backups proceed through the barman upload stage. Note: Helm does not upgrade CRD schemas on , so existing clusters need the regenerated CNPG CRDs applied out of band (kubectl apply) for the fix to take effect — a fresh install picks them up automatically. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Signed-off-by: Andrey Kolkov <[email protected]>
1 parent 56d9721 commit e05acca

8 files changed

Lines changed: 1003 additions & 128 deletions

File tree

‎packages/system/postgres-operator/Makefile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ update:
1010
rm -rf charts
1111
helm repo add cnpg https://cloudnative-pg.github.io/charts
1212
helm repo update cnpg
13-
helm pull cnpg/cloudnative-pg --untar --untardir charts --version 0.26.1
13+
helm pull cnpg/cloudnative-pg --untar --untardir charts --version 0.27.1
1414
rm -rf charts/cloudnative-pg/charts
1515
helm pull cnpg/plugin-barman-cloud --untar --untardir charts --version 0.7.0
1616
rm -rf charts/plugin-barman-cloud/charts
Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
dependencies:
22
- name: cluster
33
repository: https://cloudnative-pg.github.io/grafana-dashboards
4-
version: 0.0.2
5-
digest: sha256:fcf16ad357c17be3dd79c138723e78e9e101fecc5d07d9371299c32b9f85dbd9
6-
generated: "2024-04-25T12:32:36.61779032-04:00"
4+
version: 0.0.5
5+
digest: sha256:92acaa7742cad61339d69da604eda609e3d5e02f05efa224f5a58f2b845cd2b4
6+
generated: "2026-01-19T21:05:18.955160552+02:00"

‎packages/system/postgres-operator/charts/cloudnative-pg/Chart.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
apiVersion: v2
2-
appVersion: 1.27.1
2+
appVersion: 1.28.1
33
dependencies:
44
- alias: monitoring
55
condition: monitoring.grafanaDashboard.create
@@ -22,4 +22,4 @@ name: cloudnative-pg
2222
sources:
2323
- https://github.com/cloudnative-pg/charts
2424
type: application
25-
version: 0.26.1
25+
version: 0.27.1

‎packages/system/postgres-operator/charts/cloudnative-pg/README.md‎

Lines changed: 2 additions & 3 deletions
Large diffs are not rendered by default.

‎packages/system/postgres-operator/charts/cloudnative-pg/templates/crds/crds.yaml‎

Lines changed: 910 additions & 43 deletions
Large diffs are not rendered by default.

‎packages/system/postgres-operator/charts/cloudnative-pg/templates/deployment.yaml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -101,7 +101,7 @@ spec:
101101
livenessProbe:
102102
httpGet:
103103
path: /readyz
104-
port: {{ .Values.webhook.port }}
104+
port: webhook-server
105105
scheme: HTTPS
106106
{{- if .Values.webhook.livenessProbe.initialDelaySeconds }}
107107
initialDelaySeconds: {{ .Values.webhook.livenessProbe.initialDelaySeconds }}
@@ -117,7 +117,7 @@ spec:
117117
readinessProbe:
118118
httpGet:
119119
path: /readyz
120-
port: {{ .Values.webhook.port }}
120+
port: webhook-server
121121
scheme: HTTPS
122122
{{- if .Values.webhook.readinessProbe.initialDelaySeconds }}
123123
initialDelaySeconds: {{ .Values.webhook.readinessProbe.initialDelaySeconds }}
@@ -132,7 +132,7 @@ spec:
132132
{{- end }}
133133
httpGet:
134134
path: /readyz
135-
port: {{ .Values.webhook.port }}
135+
port: webhook-server
136136
scheme: HTTPS
137137
{{- if .Values.webhook.startupProbe.periodSeconds }}
138138
periodSeconds: {{ .Values.webhook.startupProbe.periodSeconds }}

‎packages/system/postgres-operator/charts/cloudnative-pg/values.yaml‎

Lines changed: 75 additions & 71 deletions
Original file line numberDiff line numberDiff line change
@@ -210,30 +210,30 @@ monitoringQueriesConfigMap:
210210
queries: |
211211
backends:
212212
query: |
213-
SELECT sa.datname
214-
, sa.usename
215-
, sa.application_name
216-
, states.state
217-
, COALESCE(sa.count, 0) AS total
218-
, COALESCE(sa.max_tx_secs, 0) AS max_tx_duration_seconds
219-
FROM ( VALUES ('active')
220-
, ('idle')
221-
, ('idle in transaction')
222-
, ('idle in transaction (aborted)')
223-
, ('fastpath function call')
224-
, ('disabled')
225-
) AS states(state)
226-
LEFT JOIN (
227-
SELECT datname
228-
, state
229-
, usename
230-
, COALESCE(application_name, '') AS application_name
231-
, COUNT(*)
232-
, COALESCE(EXTRACT (EPOCH FROM (max(now() - xact_start))), 0) AS max_tx_secs
233-
FROM pg_catalog.pg_stat_activity
234-
GROUP BY datname, state, usename, application_name
235-
) sa ON states.state = sa.state
236-
WHERE sa.usename IS NOT NULL
213+
SELECT sa.datname
214+
, sa.usename
215+
, sa.application_name
216+
, states.state
217+
, COALESCE(sa.count, 0) AS total
218+
, COALESCE(sa.max_tx_secs, 0) AS max_tx_duration_seconds
219+
FROM ( VALUES ('active')
220+
, ('idle')
221+
, ('idle in transaction')
222+
, ('idle in transaction (aborted)')
223+
, ('fastpath function call')
224+
, ('disabled')
225+
) AS states(state)
226+
LEFT JOIN (
227+
SELECT datname
228+
, state
229+
, usename
230+
, COALESCE(application_name, '') AS application_name
231+
, COUNT(*)
232+
, COALESCE(EXTRACT (EPOCH FROM (max(now() - xact_start))), 0) AS max_tx_secs
233+
FROM pg_catalog.pg_stat_activity
234+
GROUP BY datname, state, usename, application_name
235+
) sa ON states.state = sa.state
236+
WHERE sa.usename IS NOT NULL
237237
metrics:
238238
- datname:
239239
usage: "LABEL"
@@ -256,22 +256,22 @@ monitoringQueriesConfigMap:
256256
257257
backends_waiting:
258258
query: |
259-
SELECT count(*) AS total
260-
FROM pg_catalog.pg_locks blocked_locks
261-
JOIN pg_catalog.pg_locks blocking_locks
262-
ON blocking_locks.locktype = blocked_locks.locktype
263-
AND blocking_locks.database IS NOT DISTINCT FROM blocked_locks.database
264-
AND blocking_locks.relation IS NOT DISTINCT FROM blocked_locks.relation
265-
AND blocking_locks.page IS NOT DISTINCT FROM blocked_locks.page
266-
AND blocking_locks.tuple IS NOT DISTINCT FROM blocked_locks.tuple
267-
AND blocking_locks.virtualxid IS NOT DISTINCT FROM blocked_locks.virtualxid
268-
AND blocking_locks.transactionid IS NOT DISTINCT FROM blocked_locks.transactionid
269-
AND blocking_locks.classid IS NOT DISTINCT FROM blocked_locks.classid
270-
AND blocking_locks.objid IS NOT DISTINCT FROM blocked_locks.objid
271-
AND blocking_locks.objsubid IS NOT DISTINCT FROM blocked_locks.objsubid
272-
AND blocking_locks.pid != blocked_locks.pid
273-
JOIN pg_catalog.pg_stat_activity blocking_activity ON blocking_activity.pid = blocking_locks.pid
274-
WHERE NOT blocked_locks.granted
259+
SELECT count(*) AS total
260+
FROM pg_catalog.pg_locks blocked_locks
261+
JOIN pg_catalog.pg_locks blocking_locks
262+
ON blocking_locks.locktype = blocked_locks.locktype
263+
AND blocking_locks.database IS NOT DISTINCT FROM blocked_locks.database
264+
AND blocking_locks.relation IS NOT DISTINCT FROM blocked_locks.relation
265+
AND blocking_locks.page IS NOT DISTINCT FROM blocked_locks.page
266+
AND blocking_locks.tuple IS NOT DISTINCT FROM blocked_locks.tuple
267+
AND blocking_locks.virtualxid IS NOT DISTINCT FROM blocked_locks.virtualxid
268+
AND blocking_locks.transactionid IS NOT DISTINCT FROM blocked_locks.transactionid
269+
AND blocking_locks.classid IS NOT DISTINCT FROM blocked_locks.classid
270+
AND blocking_locks.objid IS NOT DISTINCT FROM blocked_locks.objid
271+
AND blocking_locks.objsubid IS NOT DISTINCT FROM blocked_locks.objsubid
272+
AND blocking_locks.pid != blocked_locks.pid
273+
JOIN pg_catalog.pg_stat_activity blocking_activity ON blocking_activity.pid = blocking_locks.pid
274+
WHERE NOT blocked_locks.granted
275275
metrics:
276276
- total:
277277
usage: "GAUGE"
@@ -309,16 +309,17 @@ monitoringQueriesConfigMap:
309309
description: "Time at which postgres started (based on epoch)"
310310
311311
pg_replication:
312-
query: "SELECT CASE WHEN (
313-
NOT pg_catalog.pg_is_in_recovery()
314-
OR pg_catalog.pg_last_wal_receive_lsn() = pg_catalog.pg_last_wal_replay_lsn())
315-
THEN 0
316-
ELSE GREATEST (0,
317-
EXTRACT(EPOCH FROM (now() - pg_catalog.pg_last_xact_replay_timestamp())))
318-
END AS lag,
319-
pg_catalog.pg_is_in_recovery() AS in_recovery,
320-
EXISTS (TABLE pg_stat_wal_receiver) AS is_wal_receiver_up,
321-
(SELECT count(*) FROM pg_catalog.pg_stat_replication) AS streaming_replicas"
312+
query: |
313+
SELECT CASE WHEN (
314+
NOT pg_catalog.pg_is_in_recovery()
315+
OR pg_catalog.pg_last_wal_receive_lsn() = pg_catalog.pg_last_wal_replay_lsn())
316+
THEN 0
317+
ELSE GREATEST (0,
318+
EXTRACT(EPOCH FROM (now() - pg_catalog.pg_last_xact_replay_timestamp())))
319+
END AS lag,
320+
pg_catalog.pg_is_in_recovery() AS in_recovery,
321+
EXISTS (TABLE pg_stat_wal_receiver) AS is_wal_receiver_up,
322+
(SELECT count(*) FROM pg_catalog.pg_stat_replication) AS streaming_replicas
322323
metrics:
323324
- lag:
324325
usage: "GAUGE"
@@ -374,6 +375,9 @@ monitoringQueriesConfigMap:
374375
, COALESCE(CAST(CAST('x'||pg_catalog.right(pg_catalog.split_part(last_failed_wal, '.', 1), 16) AS pg_catalog.bit(64)) AS pg_catalog.int8), -1) AS last_failed_wal_start_lsn
375376
, EXTRACT(EPOCH FROM stats_reset) AS stats_reset_time
376377
FROM pg_catalog.pg_stat_archiver
378+
predicate_query: |
379+
SELECT NOT pg_catalog.pg_is_in_recovery()
380+
OR pg_catalog.current_setting('archive_mode') = 'always'
377381
metrics:
378382
- archived_count:
379383
usage: "COUNTER"
@@ -586,20 +590,20 @@ monitoringQueriesConfigMap:
586590
pg_stat_replication:
587591
primary: true
588592
query: |
589-
SELECT usename
590-
, COALESCE(application_name, '') AS application_name
591-
, COALESCE(client_addr::text, '') AS client_addr
592-
, COALESCE(client_port::text, '') AS client_port
593-
, EXTRACT(EPOCH FROM backend_start) AS backend_start
594-
, COALESCE(pg_catalog.age(backend_xmin), 0) AS backend_xmin_age
595-
, pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_current_wal_lsn(), sent_lsn) AS sent_diff_bytes
596-
, pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_current_wal_lsn(), write_lsn) AS write_diff_bytes
597-
, pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_current_wal_lsn(), flush_lsn) AS flush_diff_bytes
598-
, COALESCE(pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_current_wal_lsn(), replay_lsn),0) AS replay_diff_bytes
599-
, COALESCE((EXTRACT(EPOCH FROM write_lag)),0)::float AS write_lag_seconds
600-
, COALESCE((EXTRACT(EPOCH FROM flush_lag)),0)::float AS flush_lag_seconds
601-
, COALESCE((EXTRACT(EPOCH FROM replay_lag)),0)::float AS replay_lag_seconds
602-
FROM pg_catalog.pg_stat_replication
593+
SELECT usename
594+
, COALESCE(application_name, '') AS application_name
595+
, COALESCE(client_addr::text, '') AS client_addr
596+
, COALESCE(client_port::text, '') AS client_port
597+
, EXTRACT(EPOCH FROM backend_start) AS backend_start
598+
, COALESCE(pg_catalog.age(backend_xmin), 0) AS backend_xmin_age
599+
, pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_current_wal_lsn(), sent_lsn) AS sent_diff_bytes
600+
, pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_current_wal_lsn(), write_lsn) AS write_diff_bytes
601+
, pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_current_wal_lsn(), flush_lsn) AS flush_diff_bytes
602+
, COALESCE(pg_catalog.pg_wal_lsn_diff(pg_catalog.pg_current_wal_lsn(), replay_lsn),0) AS replay_diff_bytes
603+
, COALESCE((EXTRACT(EPOCH FROM write_lag)),0)::float AS write_lag_seconds
604+
, COALESCE((EXTRACT(EPOCH FROM flush_lag)),0)::float AS flush_lag_seconds
605+
, COALESCE((EXTRACT(EPOCH FROM replay_lag)),0)::float AS replay_lag_seconds
606+
FROM pg_catalog.pg_stat_replication
603607
metrics:
604608
- usename:
605609
usage: "LABEL"
@@ -659,13 +663,13 @@ monitoringQueriesConfigMap:
659663
pg_extensions:
660664
query: |
661665
SELECT
662-
current_database() as datname,
663-
name as extname,
664-
default_version,
665-
installed_version,
666-
CASE
667-
WHEN default_version = installed_version THEN 0
668-
ELSE 1
666+
current_database() as datname,
667+
name as extname,
668+
default_version,
669+
installed_version,
670+
CASE
671+
WHEN default_version = installed_version THEN 0
672+
ELSE 1
669673
END AS update_available
670674
FROM pg_catalog.pg_available_extensions
671675
WHERE installed_version IS NOT NULL

‎packages/system/postgres-operator/values.yaml‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,13 @@
11
cloudnative-pg:
22
crds:
33
create: true
4-
image:
5-
tag: "1.27.3"
4+
# Image tag intentionally left to the chart's appVersion (1.28.1). Do NOT pin
5+
# image.tag to a version the vendored chart's CRDs don't match: an operator
6+
# newer than its CRDs writes status fields the CRD prunes (e.g.
7+
# status.instanceID.sessionID, added in CNPG 1.27.3), which makes the operator
8+
# believe the instance manager restarted and fail EVERY backup cluster-wide
9+
# ("instance manager was restarted during backup"). Bump the chart version in
10+
# the Makefile to move the operator + CRDs together.
611

712
# CloudNativePG Barman Cloud Plugin (github.com/cloudnative-pg/plugin-barman-cloud).
813
# Native spec.backup.barmanObjectStore is deprecated in CNPG 1.27 and removed in 1.29,

0 commit comments

Comments
 (0)