Skip to content

Commit 7b65cf2

Browse files
authored
feat(platform): migrate to etcd-operator v1alpha2 (#2859)
<!-- Thank you for making a contribution! Here are some tips for you: - Use Conventional Commits for the PR title: `type(scope): description` - Types: feat, fix, docs, style, refactor, perf, test, build, ci, chore - Scopes are not an exhaustive list — pick the most specific scope for the change and extend the list when a genuinely new area appears. Examples: - System components: dashboard, platform, operator, cilium, kube-ovn, linstor, fluxcd, cluster-api - Managed apps: postgres, mariadb, redis, kafka, clickhouse, virtual-machine, kubernetes - Development and maintenance: api, hack, tests, ci, docs, maintenance - Breaking changes: append `!` after type/scope (`feat(api)!: ...`) or add a `BREAKING CHANGE:` footer - If it's a work in progress, consider creating this PR as a draft. - Don't hesistate to ask for opinion and review in the community chats, even if it's still a draft. - Add the label `backport` if it's a bugfix that needs to be backported to a previous version. --> ## What this PR does ### Screenshots <!-- REQUIRED for UI changes: attach screenshots or screen recordings demonstrating the visual impact of your changes. PRs with UI changes without screenshots will not be merged. --> ### Release note <!-- Write a release note: - Explain what has changed internally and for users. - Start with the same `type(scope):` prefix as in the PR title - Follow the guidelines at https://github.com/kubernetes/community/blob/master/contributors/guide/release-notes.md. --> ```release-note ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added support for the newer etcd snapshot workflow (BackupClass/BackupJob/RestoreJob) with in-place restore. * Updated etcd chart/operator packaging and migration flow for the v1alpha2 operator release. * **Bug Fixes** * Switched readiness/condition handling to **Available**. * Improved waits to cover all etcd member pods during restore and scenario steps. * **Documentation** * Refreshed etcd backup/restore docs and example scenarios to match the v1alpha2 APIs and parameters. * **Tests** * Updated e2e coverage to validate v1alpha2 operator runtime contracts and replaced legacy backup-block tests with a snapshot round-trip test. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 parents a0db444 + d0740ed commit 7b65cf2

87 files changed

Lines changed: 7403 additions & 3978 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.gitignore‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,3 +93,4 @@ packages/core/platform/.build-revision
9393
examples/backups/*/.bucket-info.env
9494
examples/backups/*/.sentinel.env
9595
examples/backups/*/.backup-name.env
96+
packages/core/platform/images/migrations/etcd-operator-crds/

‎api/backups/strategy/v1alpha1/etcd_types.go‎

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -29,16 +29,16 @@ const (
2929
// +kubebuilder:resource:scope=Cluster
3030

3131
// Etcd defines a backup strategy that delegates execution to the
32-
// etcd-operator (etcd.aenix.io). The strategy carries a templated backup
33-
// destination configuration; the driver materialises one
34-
// etcd.aenix.io/v1alpha1 EtcdBackup per Cozystack BackupJob and surfaces
32+
// etcd-operator (etcd-operator.cozystack.io). The strategy carries a templated
33+
// backup destination configuration; the driver materialises one
34+
// etcd-operator.cozystack.io/v1alpha2 EtcdSnapshot per Cozystack BackupJob and surfaces
3535
// it as a Cozystack Backup artefact.
3636
//
3737
// Restore is in-place only: the driver suspends the source Etcd
3838
// HelmRelease, snapshots the live EtcdCluster spec, deletes it, and
3939
// re-creates the EtcdCluster with spec.bootstrap.restore.source
4040
// pointing at the Backup's S3 coordinates, then resumes the HR after
41-
// the cluster reaches Ready.
41+
// the cluster becomes Available.
4242
//
4343
// To-copy restore is NOT supported: RestoreJob.spec.targetApplicationRef
4444
// is a TypedLocalObjectReference (no namespace field), so there is no
@@ -68,7 +68,7 @@ type EtcdList struct {
6868

6969
// EtcdSpec specifies the desired etcd-operator-driven backup strategy.
7070
type EtcdSpec struct {
71-
// Template carries the templated EtcdBackup destination configuration
71+
// Template carries the templated EtcdSnapshot destination configuration
7272
// applied per BackupJob (and re-rendered against the same
7373
// .Application / .Parameters at restore time). String fields support
7474
// Helm-style Go templating with two top-level values:
@@ -80,19 +80,19 @@ type EtcdSpec struct {
8080
Template EtcdTemplate `json:"template"`
8181
}
8282

83-
// EtcdTemplate describes the templated EtcdBackup-shaped configuration the
83+
// EtcdTemplate describes the templated EtcdSnapshot-shaped configuration the
8484
// driver renders per BackupJob (and on the restore path stamps onto
8585
// EtcdCluster.spec.bootstrap.restore.source).
8686
type EtcdTemplate struct {
8787
// Destination defines where the etcd snapshot will be stored. Field
88-
// semantics mirror etcd.aenix.io/v1alpha1 EtcdBackup.spec.destination.
88+
// semantics mirror etcd-operator.cozystack.io/v1alpha2 EtcdSnapshot.spec.destination.
8989
Destination EtcdDestinationTemplate `json:"destination"`
9090
}
9191

92-
// EtcdDestinationTemplate mirrors etcd.aenix.io/v1alpha1
93-
// EtcdBackup.spec.destination, RESTRICTED to S3.
92+
// EtcdDestinationTemplate mirrors etcd-operator.cozystack.io/v1alpha2
93+
// EtcdSnapshot.spec.destination, RESTRICTED to S3.
9494
//
95-
// The upstream etcd-operator EtcdBackup.spec.destination accepts either
95+
// The upstream etcd-operator EtcdSnapshot.spec.destination accepts either
9696
// S3 or PVC, but the operator's filename convention is asymmetric
9797
// between backup and restore on the PVC path:
9898
//
@@ -118,7 +118,7 @@ type EtcdDestinationTemplate struct {
118118
}
119119

120120
// EtcdS3Template is a typed, kubebuilder-validated mirror of
121-
// etcd.aenix.io/v1alpha1 EtcdBackup.spec.destination.s3. Restores reuse
121+
// etcd-operator.cozystack.io/v1alpha2 EtcdSnapshot.spec.destination.s3. Restores reuse
122122
// the same shape: the driver re-renders the strategy and stamps the
123123
// resulting block onto EtcdCluster.spec.bootstrap.restore.source.s3 of the
124124
// target Etcd application.

‎examples/backups/etcd/00-helpers.sh‎

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ export BOLD='\033[1m'
1515
# Default settings (override via environment).
1616
export NAMESPACE="${NAMESPACE:-tenant-root}"
1717
# The chart pins the Helm release name (and the operator-side
18-
# etcd.aenix.io/EtcdCluster name) to "etcd" via
18+
# etcd-operator.cozystack.io/EtcdCluster name) to "etcd" via
1919
# templates/check-release-name.yaml, so the apps.cozystack.io/Etcd CR
2020
# MUST be named "etcd" per namespace. To-copy in the same namespace is
2121
# unsupported by design (chart constraint).
@@ -44,13 +44,20 @@ print_header() {
4444
}
4545

4646
# Wait until a JSONPath value on a resource matches the desired string.
47+
# Optional 7th arg is a TERMINAL failure value: once the field reaches it the
48+
# wait returns 1 immediately instead of polling to the timeout. BackupJob and
49+
# RestoreJob settle on a terminal phase=Failed that never becomes Succeeded, so
50+
# waiting the full timeout only wastes wall-clock AND risks the snapshot Job's
51+
# Pod being TTL-reaped (TTLSecondsAfterFinished=600s) before diagnostics can read
52+
# its log. Failing fast on the terminal phase keeps that log in reach.
4753
wait_for_field() {
4854
local resource_type="$1"
4955
local resource_name="$2"
5056
local jsonpath="$3"
5157
local desired="$4"
5258
local namespace="${5:-}"
5359
local timeout="${6:-300}"
60+
local fail_value="${7:-}"
5461

5562
log_substep "Waiting for $resource_type/$resource_name $jsonpath to become '$desired'..."
5663
local elapsed=0
@@ -64,6 +71,10 @@ wait_for_field() {
6471
log_success "$resource_type/$resource_name reached '$desired'"
6572
return 0
6673
fi
74+
if [[ -n "$fail_value" && "$current" == "$fail_value" ]]; then
75+
log_error "$resource_type/$resource_name reached terminal '$current' (expected '$desired')"
76+
return 1
77+
fi
6778
if [[ $elapsed -ge $timeout ]]; then
6879
log_error "Timeout waiting for $resource_type/$resource_name (current: '$current', expected: '$desired')"
6980
return 1
@@ -109,7 +120,7 @@ spec:
109120
restartPolicy: Never
110121
containers:
111122
- name: etcdctl
112-
image: quay.io/coreos/etcd:v3.5.12
123+
image: quay.io/coreos/etcd:v3.6.11
113124
command:
114125
- etcdctl
115126
- "--cacert=/etc/etcd/pki/client/ca.crt"

‎examples/backups/etcd/01-create-strategy.sh‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
#!/bin/bash
22
# Step 01: Apply the cluster-scoped Etcd strategy. Admin operation -
33
# tenants do not author the strategy CR directly. The strategy carries
4-
# the templated EtcdBackup destination shape; the BackupClass created in
4+
# the templated EtcdSnapshot destination shape; the BackupClass created in
55
# step 02 supplies bucket/endpoint/credentials per tenant via parameters.
66
set -euo pipefail
77

‎examples/backups/etcd/02-create-bucket.sh‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ for v in ETCD_ACCESS_KEY ETCD_SECRET_KEY ETCD_ENDPOINT ETCD_REGION ETCD_BUCKET;
4545
[[ -n "${!v}" && "${!v}" != "null" ]] || { log_error "BucketInfo missing required field: ${v}"; exit 1; }
4646
done
4747

48-
log_substep "Materialising the per-app credentials Secret consumed by the rendered EtcdBackup..."
48+
log_substep "Materialising the per-app credentials Secret consumed by the rendered EtcdSnapshot..."
4949
# The strategy template references this Secret by name:
5050
# "{{ .Application.metadata.name }}-etcd-backup-creds"
5151
# The etcd-operator's snapshot Job mounts the Secret and exposes

‎examples/backups/etcd/03-create-etcd-src.sh‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
#!/bin/bash
22
# Step 03: Apply the source Etcd application, wait for the operator-side
3-
# EtcdCluster to reach Ready, then write a sentinel key that the backup
3+
# EtcdCluster to become Available, then write a sentinel key that the backup
44
# captures and the restore round-trips. The sentinel is the in-cluster
55
# witness that the snapshot actually round-tripped through S3.
66
set -euo pipefail
@@ -27,13 +27,13 @@ EOF
2727

2828
log_substep "Waiting for HelmRelease 'etcd' to be Ready..."
2929
kubectl -n "$NAMESPACE" wait hr "${ETCD_NAME}" --for=condition=ready --timeout=300s
30-
log_substep "Waiting for etcd.aenix.io/EtcdCluster 'etcd' to be Ready..."
31-
kubectl -n "$NAMESPACE" wait etcdcluster.etcd.aenix.io etcd \
32-
--for=jsonpath='{.status.conditions[?(@.type=="Ready")].status}'=True \
30+
log_substep "Waiting for etcd-operator.cozystack.io/EtcdCluster 'etcd' to be Available..."
31+
kubectl -n "$NAMESPACE" wait etcdcluster.etcd-operator.cozystack.io etcd \
32+
--for=jsonpath='{.status.conditions[?(@.type=="Available")].status}'=True \
3333
--timeout=300s
3434

35-
log_substep "Waiting for etcd-0 pod to be Ready..."
36-
kubectl -n "$NAMESPACE" wait pod etcd-0 --for=condition=ready --timeout=300s
35+
log_substep "Waiting for etcd member pods to be Ready..."
36+
kubectl -n "$NAMESPACE" wait pod -l app.kubernetes.io/instance=etcd --for=condition=ready --timeout=300s
3737

3838
SENTINEL_KEY="__cozystack_e2e_sentinel"
3939
SENTINEL_VAL="pristine-$(date -u +%Y%m%dT%H%M%SZ)"

‎examples/backups/etcd/04-create-backupjob.sh‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ spec:
2525
EOF
2626

2727
wait_for_field backupjobs.backups.cozystack.io "$BACKUPJOB_NAME" \
28-
'{.status.phase}' Succeeded "$NAMESPACE" 1200
28+
'{.status.phase}' Succeeded "$NAMESPACE" 1200 Failed
2929

3030
BACKUP_NAME=$(kubectl -n "$NAMESPACE" get backupjobs.backups.cozystack.io "$BACKUPJOB_NAME" \
3131
-o jsonpath='{.status.backupRef.name}')

‎examples/backups/etcd/05-restore-in-place.sh‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -43,10 +43,10 @@ spec:
4343
EOF
4444

4545
wait_for_field restorejobs.backups.cozystack.io "$RESTOREJOB_INPLACE_NAME" \
46-
'{.status.phase}' Succeeded "$NAMESPACE" 1800
46+
'{.status.phase}' Succeeded "$NAMESPACE" 1800 Failed
4747

48-
log_substep "Waiting for etcd-0 to be Ready again after restore..."
49-
kubectl -n "$NAMESPACE" wait pod etcd-0 --for=condition=ready --timeout=300s
48+
log_substep "Waiting for etcd member pods to be Ready again after restore..."
49+
kubectl -n "$NAMESPACE" wait pod -l app.kubernetes.io/instance=etcd --for=condition=ready --timeout=300s
5050

5151
# Helm's next apply re-renders the chart's bootstrap-less EtcdCluster
5252
# spec, which evicts the bootstrap.restore block the driver stamped on
@@ -58,7 +58,7 @@ kubectl -n "$NAMESPACE" wait pod etcd-0 --for=condition=ready --timeout=300s
5858
log_substep "Waiting for Helm to evict bootstrap.restore from the live EtcdCluster..."
5959
elapsed=0
6060
while true; do
61-
BOOTSTRAP=$(kubectl -n "$NAMESPACE" get etcdcluster.etcd.aenix.io etcd \
61+
BOOTSTRAP=$(kubectl -n "$NAMESPACE" get etcdcluster.etcd-operator.cozystack.io etcd \
6262
-o jsonpath='{.spec.bootstrap}' 2>/dev/null || true)
6363
if [[ -z "$BOOTSTRAP" || "$BOOTSTRAP" == "{}" || "$BOOTSTRAP" == "null" ]]; then
6464
log_success "bootstrap.restore evicted by Helm reconcile"

‎examples/backups/etcd/90-scenario-admin-prepare.md‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,9 @@ prerequisite hand-off ends at "tenants can author `BackupJob` /
77

88
## 1. Install the operator + driver
99

10-
- `packages/system/etcd-operator` ships the upstream `etcd-operator`
11-
(CRDs: `EtcdCluster`, `EtcdBackup`, `EtcdBackupSchedule`).
10+
- `packages/system/etcd-operator` ships the cozystack-authored `etcd-operator`
11+
(`etcd-operator.cozystack.io/v1alpha2`; CRDs: `EtcdCluster`, `EtcdMember`,
12+
`EtcdSnapshot`).
1213
- `packages/system/backupstrategy-controller` ships the
1314
`strategy.backups.cozystack.io/v1alpha1` API and the controllers that
1415
dispatch `BackupJob` / `RestoreJob` to the per-app driver. The

‎examples/backups/etcd/91-scenario-user-backup.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ coordinates baked into `parameters`.
1818
## 2. Deploy the source Etcd (`03-create-etcd-src.sh`)
1919

2020
The tenant applies an `apps.cozystack.io/Etcd` named `etcd`. The
21-
HelmRelease materialises an `etcd.aenix.io/EtcdCluster` and three
21+
HelmRelease materialises an `etcd-operator.cozystack.io/EtcdCluster` and three
2222
members. The script writes a sentinel key under `etcdctl` so step 05
2323
can witness the round-trip.
2424

@@ -27,7 +27,7 @@ can witness the round-trip.
2727
A `BackupJob` references the `BackupClass` and the source `Etcd`. The
2828
driver resolves the strategy, renders its template against the live
2929
application and class parameters, then materialises one
30-
`etcd.aenix.io/EtcdBackup` CR per `BackupJob`. The etcd-operator's
30+
`etcd-operator.cozystack.io/EtcdSnapshot` CR per `BackupJob`. The etcd-operator's
3131
controller runs a one-shot Job that snapshots etcd and uploads the
3232
file to S3. On `phase=Complete`, the driver creates a Cozystack
3333
`Backup` artefact with the S3 coordinates baked into
@@ -36,7 +36,7 @@ file to S3. On `phase=Complete`, the driver creates a Cozystack
3636

3737
## What the tenant does NOT do
3838

39-
- Author `EtcdBackup` CRs by hand. The driver owns the lifecycle.
39+
- Author `EtcdSnapshot` CRs by hand. The driver owns the lifecycle.
4040
- Manage the underlying S3 credentials. The driver consumes them from
4141
the per-app Secret the bucket flow already produced.
4242
- Track snapshot retention via cron. A `Plan` resource (not part of

0 commit comments

Comments
 (0)