Skip to content

Commit 95c2d85

Browse files
lexfreilexfrei
authored
fix(etcd): leave compaction to etcd and pass the real quota to defrag (#4873)
## What this PR does The hourly defrag CronJob ran `etcd-defrag` with its defaults, and two of them do not fit the cluster the chart creates. - `--compaction` (default `true`) compacts to the latest revision before every defrag. The EtcdCluster already compacts itself (`autoCompactionMode: periodic`, `autoCompactionRetention: 5m`), so the extra compaction either fails with `mvcc: required revision has been compacted` when nothing was written since the last auto-compaction, which is what every defrag Job log shows on our clusters, or, on a busy cluster, drops the history the retention keeps for watchers to resume from. - `--etcd-storage-quota-bytes` defaults to 2 GiB, so `dbQuotaUsage` in the defrag rule is measured against 2 GiB rather than the `quotaBackendBytes` the chart sets (95% of `size`, about 3.8 GiB for the default `4Gi`). The CronJob now passes `--compaction=false` and the chart's own quota through the existing `calculateQuotaBackendBytes` helper. The new unit tests check both arguments and that the defrag quota equals the EtcdCluster's `quotaBackendBytes` for a non-default size. Without the template change all three fail. The same templates are on `release-1.6` and the change applies there cleanly. ### Screenshots Not a UI change. ### Downstream repositories - [x] No downstream repository is affected by this change - [ ] [cozystack/website](https://github.com/cozystack/website) - follow-up: - [ ] [cozystack/terraform-provider-cozystack](https://github.com/cozystack/terraform-provider-cozystack) - follow-up: - [ ] [cozystack/ansible-cozystack](https://github.com/cozystack/ansible-cozystack) - follow-up: - [ ] [cozystack/ccp](https://github.com/cozystack/ccp) - follow-up: - [ ] [cozystack/talm](https://github.com/cozystack/talm) - follow-up: - [ ] [cozystack/cozyhr](https://github.com/cozystack/cozyhr) - follow-up: - [ ] [cozystack/cozy-proxy](https://github.com/cozystack/cozy-proxy) - follow-up: - [ ] [cozystack/cozystack-telemetry-server](https://github.com/cozystack/cozystack-telemetry-server) - follow-up: - [ ] [cozystack/external-apps-example](https://github.com/cozystack/external-apps-example) - follow-up: - [ ] [cozystack/examples](https://github.com/cozystack/examples) - follow-up: - [ ] [cozystack/community](https://github.com/cozystack/community) - follow-up: The diff is two template arguments and a unit test in `packages/extra/etcd`, with no change to values, schema, defaults or package names, so nothing in the trigger map applies. ### Release note ```release-note fix(etcd): the defrag job no longer compacts on top of etcd's periodic auto-compaction, and it evaluates its 80% quota threshold against the cluster's real quotaBackendBytes instead of 2 GiB. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Improvements** * The etcd defragmentation job now uses the cluster’s configured backend storage quota and skips compaction before defragmenting. This keeps the defragmentation operation aligned with the cluster’s storage settings. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 parents 33ca8b6 + fe59a6b commit 95c2d85

2 files changed

Lines changed: 51 additions & 0 deletions

File tree

‎packages/extra/etcd/templates/etcd-defrag.yaml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,14 @@ spec:
4444
- --cert=/etc/etcd/pki/client/cert/tls.crt
4545
- --key=/etc/etcd/pki/client/cert/tls.key
4646
- --cluster
47+
# etcd compacts itself (periodic, 5m retention, see etcd-cluster.yaml).
48+
# Compacting again to the latest revision either fails because nothing
49+
# was written since, or drops the history that retention keeps for
50+
# watchers to resume from.
51+
- --compaction=false
52+
# dbQuotaUsage below is dbSize over this value, and the tool assumes
53+
# 2 GiB unless told the quota the cluster actually runs with.
54+
- --etcd-storage-quota-bytes={{ include "calculateQuotaBackendBytes" .Values.size }}
4755
- --defrag-rule
4856
- "dbQuotaUsage > 0.8 || dbSize - dbSizeInUse > 200*1024*1024"
4957
volumeMounts:

‎packages/extra/etcd/tests/defrag_test.yaml‎

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
suite: etcd defrag CronJob leaves compaction to etcd and knows the real quota
2+
3+
release:
4+
name: etcd
5+
namespace: tenant-test
6+
7+
templates:
8+
- templates/etcd-defrag.yaml
9+
- templates/etcd-cluster.yaml
10+
11+
set:
12+
_cluster: {}
13+
14+
tests:
15+
- it: does not compact before defragmenting
16+
template: templates/etcd-defrag.yaml
17+
asserts:
18+
- contains:
19+
path: spec.jobTemplate.spec.template.spec.containers[0].args
20+
content: --compaction=false
21+
22+
- it: evaluates dbQuotaUsage against the default 4Gi cluster quota
23+
template: templates/etcd-defrag.yaml
24+
asserts:
25+
- contains:
26+
path: spec.jobTemplate.spec.template.spec.containers[0].args
27+
content: --etcd-storage-quota-bytes=4080218931
28+
29+
- it: passes the defrag tool the same quota the EtcdCluster gets
30+
set:
31+
size: 10Gi
32+
asserts:
33+
- contains:
34+
path: spec.jobTemplate.spec.template.spec.containers[0].args
35+
content: --etcd-storage-quota-bytes=10200547328
36+
template: templates/etcd-defrag.yaml
37+
- equal:
38+
path: spec.options.quotaBackendBytes
39+
value: 10200547328
40+
template: templates/etcd-cluster.yaml
41+
documentSelector:
42+
path: kind
43+
value: EtcdCluster

0 commit comments

Comments
 (0)