Skip to content

Commit c0449ee

Browse files
authored
feat(backups): back up Valkey with the Redis strategy (#4898)
## What this PR does Adds backup and restore for the managed Valkey app. Valkey runs on the same spotahome RedisFailover operator as Redis (`spec.engine: Valkey`), with the same sentinel topology, replication protocol and RDB format. So instead of adding a new strategy kind, the existing Redis strategy now serves both apps: - **Driver.** `applicationRef.kind` may be `Redis` or `Valkey`. The template context gets `.Release.Prefix`, which the driver derives from the app's kind (`redis-` or `valkey-`). The chart uses that prefix for every name it builds: the sentinel Service `rfs-<prefix><app>`, the auth Secret `<prefix><app>-auth` and the CA Secret. In cleanup mode the prefix comes from the Backup's `applicationRef`, so it works after the app is gone. A kind with no known prefix fails the render instead of silently producing an empty prefix. Error messages name the real kind. - **Upgrade window.** The shipped template reads `{{ .Release.Prefix | default "redis-" }}`. The strategy CR and the controller ship in one release, and until the new controller takes the lease, the previous one (which has no `Prefix` and serves only Redis) renders the new CR; a bare reference would print `<no value>` into the Secret and sentinel names. - **Same-kind restore only.** A restore whose target kind differs from the backup's source kind fails before any Job is created. Replaying a Redis dump into Valkey, or the reverse, is not something the strategy can promise across engine releases. - **Chart.** `cozy-default-redis` builds names from `.Release.Prefix`. `cozy-default` routes `apps.cozystack.io/Valkey` to `cozy-default-redis`; the DefaultObjectsGate already dedupes repeated strategy refs. No new image: the `redis:8` base loads the RDB v11 that both shipped Valkey versions write. The Valkey chart has no TLS option, so the optional CA mount stays empty and the Job connects in plaintext. - **CRD.** Only the `.Release.Prefix` description in the Redis strategy CRD changes (regenerated with controller-gen v0.16.4; every other strategy CRD regenerates byte-identical). - **Docs.** `docs/operations/backup-classes.md` has the Valkey route and a paragraph on how the shared strategy behaves. - **E2E.** The Redis backup demo takes `APP_KIND=Valkey`, which switches the app kind and the name prefix. Its BackupClass routes Valkey too. The valkey suite gains `valkey-backup-roundtrip`, which seeds a marker, backs it up, restores it in place and into a copy. In `hack/select-e2e.sh`, a seaweedfs change now selects the valkey suite (enforced by `select-e2e_test.bats`), an edit to the shared `examples/backups/redis/` harness selects both `redis` and `valkey` (pinned by a new test), and `hack/select-install.sh` keeps the backup stack in the reduced install for the valkey suite (pinned in `select-install_test.bats`). ### Validation - `go test ./internal/backupcontroller/...` passes. New tests: a Valkey BackupJob renders `--prefix=valkey-`; a Redis backup restored into a Valkey target fails without creating a Job; the ref validator accepts Valkey; a cleanup for a kind with no prefix is released without a Job. The existing Redis render test now also asserts `redis-`. - The `default "redis-"` fallback, run through the controller's own template engine with no `Prefix` in the context, renders `rfs-redis-cache`; the bare reference renders `rfs-<no value>cache`. - `helm unittest` on backupstrategy-controller: 26/26, with `cozy-default` at eleven routes. `helm lint` is clean. - `hack/redis-backup-strategy.bats` executes the shipped script and adds a Valkey case: it must query `rfs-valkey-<app>` and never `rfs-redis-`. The fixture now also fails if any `{{ }}` placeholder survives substitution. All 62 tests in `hack/select-e2e_test.bats` pass. - Engine compatibility, checked locally with the strategy image `redis:8-alpine@sha256:ba6e…` against `valkey/valkey` 8.1.8 and 7.2.13 using the shipped script's exact steps: `redis-cli --rdb` from a password-protected Valkey master, load into the throwaway `redis-server`, emit the RESP base AOF, `FLUSHALL`, then `--pipe` into a fresh Valkey. On both versions every check held: the db0/db1/db3 boundaries, a stream with its consumer group, a key's TTL and the marker all survived, and FLUSHALL wiped the stale target key. - I have not run this on a live cluster. The new chainsaw round-trip is what exercises it end to end. Out of scope, and already true before this PR: the website's backup pages in `docs/next` don't list Redis, RabbitMQ or Kafka yet, so none of the triggers in the downstream map fire here. ### Screenshots N/A — no UI changes. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: ### Release note ```release-note feat(backups): Valkey apps can now be backed up and restored. The platform `cozy-default` BackupClass routes `apps.cozystack.io/Valkey` to the existing `cozy-default-redis` strategy, which dumps an RDB snapshot to the system bucket and restores it in place or into another Valkey app. A Valkey backup restores only into a Valkey app. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added support for backing up and restoring Valkey applications using the existing Redis backup strategy. * Updated the backup walkthrough to support both Redis and Valkey, with resource names selected for the chosen application type. * **Bug Fixes** * Prevented restores when the backup and target application types differ. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 parents 7f454d8 + 43b02bd commit c0449ee

24 files changed

Lines changed: 391 additions & 80 deletions

‎api/backups/strategy/v1alpha1/redis_types.go‎

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -29,11 +29,11 @@ const (
2929
// +kubebuilder:subresource:status
3030
// +kubebuilder:resource:scope=Cluster
3131

32-
// Redis defines a backup strategy for a spotahome RedisFailover app. The
33-
// spotahome operator ships no backup CRD, so the strategy runs a one-shot Job
34-
// that dumps an RDB snapshot to object storage (and replays it on restore) -
35-
// the same PodTemplateSpec model as the Job/Altinity strategies rather than an
36-
// operator-CR delegation.
32+
// Redis defines a backup strategy for a spotahome RedisFailover app, Redis or
33+
// Valkey. The spotahome operator ships no backup CRD, so the strategy runs a
34+
// one-shot Job that dumps an RDB snapshot to object storage (and replays it on
35+
// restore) - the same PodTemplateSpec model as the Job/Altinity strategies
36+
// rather than an operator-CR delegation.
3737
type Redis struct {
3838
metav1.TypeMeta `json:",inline"`
3939
metav1.ObjectMeta `json:"metadata,omitempty"`
@@ -61,6 +61,10 @@ type RedisSpec struct {
6161
// `.Application.spec.replicas`).
6262
// - `.Release.Name` and `.Release.Namespace` — the application's name
6363
// and namespace.
64+
// - `.Release.Prefix` — the prefix the application's chart puts before
65+
// the name of every resource it creates: `redis-` for a Redis
66+
// application, `valkey-` for a Valkey one. Both run on the same
67+
// RedisFailover operator, so one strategy serves either kind.
6468
// - `.Mode` — `"backup"`, `"restore"`, or `"cleanup"` (delete the object
6569
// of a removed Backup), identifying the run.
6670
// - `.ObjectKey` — the object-storage key for this run: unique per backup,

‎docs/operations/backup-classes.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,9 +18,12 @@ Tenants reference `cozy-default` from `BackupJob`, `Plan`, and `RestoreJob` reso
1818
| `apps.cozystack.io/Etcd` | etcd-operator snapshot | `strategy.backups.cozystack.io/Etcd` `cozy-default-etcd` |
1919
| `apps.cozystack.io/RabbitMQ` | RabbitMQ definitions (management API) | `strategy.backups.cozystack.io/Rabbitmq` `cozy-default-rabbitmq` |
2020
| `apps.cozystack.io/Redis` | RDB dump Job (sentinel-discovered master) | `strategy.backups.cozystack.io/Redis` `cozy-default-redis` |
21+
| `apps.cozystack.io/Valkey` | RDB dump Job (sentinel-discovered master) | `strategy.backups.cozystack.io/Redis` `cozy-default-redis` |
2122
| `apps.cozystack.io/VMInstance` | Velero + kubevirt-velero-plugin | `strategy.backups.cozystack.io/Velero` `cozy-default-velero-vminstance` |
2223
| `apps.cozystack.io/VMDisk` | Velero | `strategy.backups.cozystack.io/Velero` `cozy-default-velero-vmdisk` |
2324

25+
Valkey runs on the same RedisFailover operator as Redis, so `cozy-default-redis` serves both: the driver hands the template `.Release.Prefix` (`redis-` or `valkey-`, after the app's Kind), from which it builds the Sentinel Service and the auth and CA Secret names. A Valkey backup restores only into a Valkey app, a Redis backup only into a Redis app. The Valkey chart has no TLS option, so a Valkey dump always connects in plaintext.
26+
2427
### Shipped but NOT bound (admin opt-in required)
2528

2629
| Application Kind | Driver | Strategy CR |

‎examples/backups/redis/00-helpers.sh‎

Lines changed: 19 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,16 @@ export BOLD='\033[1m'
1717
# reaches lives in tenant-root, and the default e2e tenant-test is egress-
1818
# isolated from it).
1919
export NAMESPACE="${NAMESPACE:-tenant-root}"
20+
# APP_KIND=Valkey runs the same round-trip against a Valkey app: it runs on the
21+
# same RedisFailover operator, and its chart names every resource valkey-<app>
22+
# where the redis chart uses redis-<app>.
23+
export APP_KIND="${APP_KIND:-Redis}"
24+
case "$APP_KIND" in
25+
Redis) APP_PREFIX=redis-; APP_RESOURCE=redis ;;
26+
Valkey) APP_PREFIX=valkey-; APP_RESOURCE=valkeys.apps.cozystack.io ;;
27+
*) echo "APP_KIND must be Redis or Valkey, got '$APP_KIND'" >&2; exit 1 ;;
28+
esac
29+
export APP_PREFIX APP_RESOURCE
2030
export REDIS_NAME="${REDIS_NAME:-redis-test}"
2131
export REDIS_RESTORE_NAME="${REDIS_RESTORE_NAME:-redis-restore}"
2232
export BACKUPJOB_NAME="${BACKUPJOB_NAME:-redis-backup-job}"
@@ -70,22 +80,23 @@ print_header() {
7080
source "$(dirname "${BASH_SOURCE[0]}")/../_lib/wait-helpers.sh"
7181

7282
# The cozystack redis chart names its RedisFailover (hence the operator's rfr-/
73-
# rfs- Services and the -auth Secret) redis-<app>, mirroring the strategy driver
74-
# that derives the same base from .Release.Name. So app X's password lives in
75-
# redis-X-auth. The chart renders that Secret only with authEnabled, so its
76-
# absence reads as no password; any other error still fails the read.
83+
# rfs- Services and the -auth Secret) redis-<app>, the valkey chart valkey-<app>,
84+
# mirroring the strategy driver that derives the same base from .Release.Prefix
85+
# and .Release.Name. So app X's password lives in <prefix>X-auth. The chart
86+
# renders that Secret only with authEnabled, so its absence reads as no
87+
# password; any other error still fails the read.
7788
redis_pw() {
78-
kubectl -n "$NAMESPACE" get secret "redis-$1-auth" --ignore-not-found -o jsonpath='{.data.password}' | base64 -d
89+
kubectl -n "$NAMESPACE" get secret "${APP_PREFIX}$1-auth" --ignore-not-found -o jsonpath='{.data.password}' | base64 -d
7990
}
8091

8192
# Run a redis-cli command against an application's CURRENT master. Master
82-
# discovery asks a sentinel of the operator's rfs-redis-<app> Deployment
93+
# discovery asks a sentinel of the operator's rfs-<prefix><app> Deployment
8394
# (master name "mymaster"), so a write always lands on the writable node even
8495
# after a failover. Args after the app name are passed verbatim to redis-cli.
8596
# redis_cmd redis-test SET sentinel:marker hello
8697
#
8798
# The command runs by `kubectl exec` in a sentinel Pod, whose image carries
88-
# redis-cli. A throwaway `kubectl run -i` Pod is not usable here: its stdout
99+
# redis-cli (the valkey image ships it as a link to valkey-cli). A throwaway `kubectl run -i` Pod is not usable here: its stdout
89100
# comes back over an attach, which only carries what the container writes after
90101
# the attach is established, and kubectl falls back to the Pod log only when
91102
# the attach fails. A redis-cli that finishes in between comes back as an
@@ -98,7 +109,7 @@ redis_pw() {
98109
redis_cmd() {
99110
local app="$1" pw; shift
100111
pw=$(redis_pw "$app")
101-
printf '%s\n' "$pw" | kubectl -n "$NAMESPACE" exec -i "deploy/rfs-redis-$app" -c sentinel -- sh -c '
112+
printf '%s\n' "$pw" | kubectl -n "$NAMESPACE" exec -i "deploy/rfs-${APP_PREFIX}$app" -c sentinel -- sh -c '
102113
IFS= read -r pw || true
103114
addr=$(redis-cli -p 26379 sentinel get-master-addr-by-name mymaster) || exit 1
104115
h=$(echo "$addr" | sed -n 1p); p=$(echo "$addr" | sed -n 2p)

‎examples/backups/redis/01-create-redis.sh‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ print_header "Step 01: Provision Redis '${REDIS_NAME}'"
1010

1111
kubectl apply -f - <<EOF
1212
apiVersion: apps.cozystack.io/v1alpha1
13-
kind: Redis
13+
kind: ${APP_KIND}
1414
metadata:
1515
name: ${REDIS_NAME}
1616
namespace: ${NAMESPACE}

‎examples/backups/redis/02-create-backupjob.sh‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ metadata:
1919
spec:
2020
applicationRef:
2121
apiGroup: apps.cozystack.io
22-
kind: Redis
22+
kind: ${APP_KIND}
2323
name: ${REDIS_NAME}
2424
backupClassName: ${BACKUPCLASS_NAME}
2525
EOF

‎examples/backups/redis/04-restore-to-copy.sh‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ print_header "Step 04: To-copy restore into '${REDIS_RESTORE_NAME}'"
1313

1414
kubectl apply -f - <<EOF
1515
apiVersion: apps.cozystack.io/v1alpha1
16-
kind: Redis
16+
kind: ${APP_KIND}
1717
metadata:
1818
name: ${REDIS_RESTORE_NAME}
1919
namespace: ${NAMESPACE}
@@ -38,7 +38,7 @@ spec:
3838
name: ${BACKUPJOB_NAME}
3939
targetApplicationRef:
4040
apiGroup: apps.cozystack.io
41-
kind: Redis
41+
kind: ${APP_KIND}
4242
name: ${REDIS_RESTORE_NAME}
4343
EOF
4444

‎examples/backups/redis/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@ Or step through them:
2828

2929
## Notes
3030

31+
- **Valkey.** `APP_KIND=Valkey` runs the same walkthrough against a Valkey app, which runs on the same RedisFailover operator; every `redis-<name>` resource below is then `valkey-<name>`.
3132
- **Master discovery** goes through the operator's sentinel Service `rfs-redis-<name>` (master name `mymaster`), so writes and the dump/replay always target the current master even after a failover.
3233
- **Faithful restore.** The restore loads the RDB into a throwaway local redis and replays what a real engine emits, so all logical databases (`SELECT` per db), stream last-ids and consumer groups survive — a merge onto db0 does not happen.
3334
- **Endpoint TLS.** The projected `endpoint` in `cozy-backups-creds` is always scheme-stripped, so the strategy prepends `https://` and the transport is always TLS (there is no `http://` path). It verifies against the image's system CA store by default — the platform bucket's ACME cert needs nothing more — honours a private CA from `backupStorage.endpointCASecretName` when set, and skips verification only when `backupStorage.insecureSkipTLSVerify: true` is set explicitly.

‎examples/backups/redis/backupclass.yaml‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# Maps Kind=Redis to this demo's Redis strategy. BackupClass is cluster-scoped,
1+
# Maps Kind=Redis and Kind=Valkey to this demo's Redis strategy. BackupClass is cluster-scoped,
22
# so a single instance covers every namespace. Named redis-default so it does
33
# not collide with the platform cozy-default class, which routes Redis to the
44
# shipped cozy-default-redis strategy.
@@ -16,3 +16,10 @@ spec:
1616
apiGroup: strategy.backups.cozystack.io
1717
kind: Redis
1818
name: redis-strategy-default
19+
- application:
20+
apiGroup: apps.cozystack.io
21+
kind: Valkey
22+
strategyRef:
23+
apiGroup: strategy.backups.cozystack.io
24+
kind: Redis
25+
name: redis-strategy-default

‎examples/backups/redis/cleanup.sh‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -16,8 +16,8 @@ kubectl -n "$NAMESPACE" delete restorejob "$RESTOREJOB_TOCOPY_NAME" --ignore-not
1616
kubectl -n "$NAMESPACE" delete restorejob "$RESTOREJOB_INPLACE_NAME" --ignore-not-found
1717
kubectl -n "$NAMESPACE" delete backupjob "$BACKUPJOB_NAME" --ignore-not-found
1818
kubectl -n "$NAMESPACE" delete backup "$BACKUPJOB_NAME" --ignore-not-found
19-
kubectl -n "$NAMESPACE" delete redis "$REDIS_RESTORE_NAME" --ignore-not-found
20-
kubectl -n "$NAMESPACE" delete redis "$REDIS_NAME" --ignore-not-found
19+
kubectl -n "$NAMESPACE" delete "$APP_RESOURCE" "$REDIS_RESTORE_NAME" --ignore-not-found
20+
kubectl -n "$NAMESPACE" delete "$APP_RESOURCE" "$REDIS_NAME" --ignore-not-found
2121

2222
# Demo-owned backups plumbing: the BackupClass and strategy CR are cluster-scoped
2323
# (no -n). Deleting the Bucket releases its S3 objects and the projected keys.

‎examples/backups/redis/redis-strategy.yaml‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ spec:
2828
# presence — mirrors the shipped strategy.
2929
- name: redis-ca
3030
secret:
31-
secretName: "redis-{{ .Release.Name }}.ca-cert"
31+
secretName: '{{ .Release.Prefix | default "redis-" }}{{ .Release.Name }}.ca-cert'
3232
optional: true
3333
- name: s3-ca
3434
secret:
@@ -51,7 +51,7 @@ spec:
5151
- name: REDIS_PASSWORD
5252
valueFrom:
5353
secretKeyRef:
54-
name: "redis-{{ .Release.Name }}-auth"
54+
name: '{{ .Release.Prefix | default "redis-" }}{{ .Release.Name }}-auth'
5555
key: password
5656
optional: true
5757
# S3 coordinates come from this demo's Bucket: the access keys are
@@ -137,7 +137,7 @@ spec:
137137
RTLS="--tls --cacert /etc/redis-ca/ca.crt"
138138
fi
139139
140-
SENTINEL="rfs-redis-{{ .Release.Name }}"
140+
SENTINEL="rfs-{{ .Release.Prefix | default "redis-" }}{{ .Release.Name }}"
141141
addr=$(redis-cli ${RTLS} -h "${SENTINEL}" -p 26379 sentinel get-master-addr-by-name mymaster)
142142
MHOST=$(echo "${addr}" | sed -n '1p')
143143
MPORT=$(echo "${addr}" | sed -n '2p')

0 commit comments

Comments
 (0)