Skip to content

Commit 7204992

Browse files
fix(backups): gate every default Strategy CR on its CRD
The chart ships its strategy CRDs as ordinary templates, and Helm maps every document of a release through the RESTMapper before it applies any. A default Strategy CR gated only on the bucket name therefore fails Build() with "no matches for kind" whenever the bucket name resolves before the CRD is served, the release applies nothing, the CRDs are never created, and every retry renders the same manifest. That happens on the upgrade that first ships a kind: the Kafka, Rabbitmq and Redis CRs leave a v1.6 cluster's HelmRelease stuck in UpgradeFailed on the way to v1.7. It also happens on the first render of an external-S3 install (provisionBucket=false), where the bucket name comes from values and every older default CR renders next to its absent CRD. MongoDB already gates on crdEstablished for this reason. Give every default Strategy the same gate. On an installed cluster the CRDs are established and the render is unchanged; otherwise the first revision creates the CRDs and skips the CRs, and DefaultObjectsGate, which reads the cozy-default BackupClass routes, forces the follow-up upgrade that renders them once the kinds are served. Assisted-by: LLM Signed-off-by: Myasnikov Daniil <[email protected]>
1 parent e05c565 commit 7204992

12 files changed

Lines changed: 215 additions & 26 deletions

‎docs/operations/backup-classes.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -209,6 +209,8 @@ The same trap sits one release earlier, on the Secret everything else depends on
209209

210210
Convergence is driven instead by the controller's default-objects gate (`backupStorage.reconcileDefaultObjects`, on by default), which repairs both, in order. While the credentials Secret is absent or carries no bucket name, the gate stamps `reconcile.fluxcd.io/forceAt` + `requestedAt` on the **bucket's** `bucket-<name>-system` HelmRelease. Once the bucket name resolves it checks that every object `cozy-default` routes to exists and stamps the same pair on the **`backupstrategy-controller`** HelmRelease, forcing the real Helm upgrade that re-runs the lookups. The two are throttled independently (one force per 5 minutes each), so the second is not delayed by the first. Expect the objects within a minute or two of the bucket becoming ready.
211211

212+
Each default `Strategy` CR also waits for its own CRD. The chart ships the `strategy.backups.cozystack.io` CRDs as templates in the same release, and Helm resolves every object's kind before it applies any, so a CR rendered next to a CRD the apiserver does not serve yet would fail the whole release, CRDs included. Each strategy template therefore renders only once its CRD is `Established`. That holds the CRs back on the first render of every install, including an external-S3 one where the bucket name is known up front, and on the upgrade that first ships a new strategy kind: that revision creates the CRDs, and the gate's forced upgrade above renders the CRs.
213+
212214
The gate does not force a suspended HelmRelease — helm-controller ignores both annotations while `spec.suspend` is true, so it logs the skip and leaves the force counter alone. A release left suspended (`cozyhr suspend`) is therefore never repaired until it is resumed.
213215

214216
Watch it with:

‎packages/system/backupstrategy-controller/templates/strategy-altinity-default.yaml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,8 @@
1212
inner expression at chart install time and fail.
1313
*/}}
1414
{{- $bucketName := include "backupstrategy-controller.bucketName" . -}}
15-
{{- if $bucketName -}}
15+
{{- $crdEstablished := include "backupstrategy-controller.crdEstablished" (dict "root" . "crd" "altinities.strategy.backups.cozystack.io") -}}
16+
{{- if and $bucketName $crdEstablished -}}
1617
apiVersion: strategy.backups.cozystack.io/v1alpha1
1718
kind: Altinity
1819
metadata:

‎packages/system/backupstrategy-controller/templates/strategy-cnpg-default.yaml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
{{- $bucketName := include "backupstrategy-controller.bucketName" . -}}
2-
{{- if $bucketName -}}
2+
{{- $crdEstablished := include "backupstrategy-controller.crdEstablished" (dict "root" . "crd" "cnpgs.strategy.backups.cozystack.io") -}}
3+
{{- if and $bucketName $crdEstablished -}}
34
apiVersion: strategy.backups.cozystack.io/v1alpha1
45
kind: CNPG
56
metadata:

‎packages/system/backupstrategy-controller/templates/strategy-etcd-default.yaml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
{{- $bucketName := include "backupstrategy-controller.bucketName" . -}}
2-
{{- if $bucketName -}}
2+
{{- $crdEstablished := include "backupstrategy-controller.crdEstablished" (dict "root" . "crd" "etcds.strategy.backups.cozystack.io") -}}
3+
{{- if and $bucketName $crdEstablished -}}
34
apiVersion: strategy.backups.cozystack.io/v1alpha1
45
kind: Etcd
56
metadata:

‎packages/system/backupstrategy-controller/templates/strategy-foundationdb-default.yaml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,8 @@
22
{{- $endpointHost := $endpoint | trimPrefix "https://" | trimPrefix "http://" -}}
33
{{- $secureConn := ternary "1" "0" (hasPrefix "https://" $endpoint) -}}
44
{{- $bucketName := include "backupstrategy-controller.bucketName" . -}}
5-
{{- if $bucketName -}}
5+
{{- $crdEstablished := include "backupstrategy-controller.crdEstablished" (dict "root" . "crd" "foundationdbs.strategy.backups.cozystack.io") -}}
6+
{{- if and $bucketName $crdEstablished -}}
67
apiVersion: strategy.backups.cozystack.io/v1alpha1
78
kind: FoundationDB
89
metadata:

‎packages/system/backupstrategy-controller/templates/strategy-kafka-default.yaml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,8 @@
1717
docs/operations/backup-classes.md and the example README.
1818
*/}}
1919
{{- $bucketName := include "backupstrategy-controller.bucketName" . -}}
20-
{{- if $bucketName -}}
20+
{{- $crdEstablished := include "backupstrategy-controller.crdEstablished" (dict "root" . "crd" "kafkas.strategy.backups.cozystack.io") -}}
21+
{{- if and $bucketName $crdEstablished -}}
2122
apiVersion: strategy.backups.cozystack.io/v1alpha1
2223
kind: Kafka
2324
metadata:

‎packages/system/backupstrategy-controller/templates/strategy-mariadb-default.yaml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
11
{{- $endpoint := include "backupstrategy-controller.endpoint" . -}}
22
{{- $endpointHost := $endpoint | trimPrefix "https://" | trimPrefix "http://" -}}
33
{{- $bucketName := include "backupstrategy-controller.bucketName" . -}}
4-
{{- if $bucketName -}}
4+
{{- $crdEstablished := include "backupstrategy-controller.crdEstablished" (dict "root" . "crd" "mariadbs.strategy.backups.cozystack.io") -}}
5+
{{- if and $bucketName $crdEstablished -}}
56
apiVersion: strategy.backups.cozystack.io/v1alpha1
67
kind: MariaDB
78
metadata:

‎packages/system/backupstrategy-controller/templates/strategy-rabbitmq-default.yaml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,8 @@
2020
a full-data backup is a Velero volume snapshot, not this strategy.
2121
*/}}
2222
{{- $bucketName := include "backupstrategy-controller.bucketName" . -}}
23-
{{- if $bucketName -}}
23+
{{- $crdEstablished := include "backupstrategy-controller.crdEstablished" (dict "root" . "crd" "rabbitmqs.strategy.backups.cozystack.io") -}}
24+
{{- if and $bucketName $crdEstablished -}}
2425
apiVersion: strategy.backups.cozystack.io/v1alpha1
2526
kind: Rabbitmq
2627
metadata:

‎packages/system/backupstrategy-controller/templates/strategy-redis-default.yaml‎

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -11,13 +11,12 @@
1111
directly or Helm will try to evaluate the inner expression at chart
1212
install time and fail.
1313

14-
Gate on a resolved $bucketName: an ungated Strategy CR has no RESTMapping
15-
on a fresh install, Helm Build() fails "no matches for kind", and the
16-
release applies zero objects (including the CRDs). See
17-
strategy-mongodb-default.yaml for the full rationale.
14+
Gated, like every default Strategy, on a resolved $bucketName and on its
15+
CRD being established; see the crdEstablished helper.
1816
*/}}
1917
{{- $bucketName := include "backupstrategy-controller.bucketName" . -}}
20-
{{- if $bucketName -}}
18+
{{- $crdEstablished := include "backupstrategy-controller.crdEstablished" (dict "root" . "crd" "redis.strategy.backups.cozystack.io") -}}
19+
{{- if and $bucketName $crdEstablished -}}
2120
apiVersion: strategy.backups.cozystack.io/v1alpha1
2221
kind: Redis
2322
metadata:

‎packages/system/backupstrategy-controller/templates/strategy-velero-vmdisk-default.yaml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
{{- $bucketName := include "backupstrategy-controller.bucketName" . -}}
2-
{{- if and .Values.velero.bslEnabled $bucketName -}}
2+
{{- $crdEstablished := include "backupstrategy-controller.crdEstablished" (dict "root" . "crd" "veleroes.strategy.backups.cozystack.io") -}}
3+
{{- if and .Values.velero.bslEnabled $bucketName $crdEstablished -}}
34
apiVersion: strategy.backups.cozystack.io/v1alpha1
45
kind: Velero
56
metadata:

0 commit comments

Comments
 (0)