Skip to content

Commit 33d2ac7

Browse files
committed
feat(seaweedfs): expose the master volume size limit on the app chart
The constant that decides how many buckets an instance can serve lives in the data-plane chart, and an operator has no way to reach it. That chart is pulled by ExternalArtifact, and the component publishing it declares no install section, so the reconciler drops a Package-level values override for it before any merge happens. Editing the chart was the only route, on a value whose right setting depends on the PVC size the operator chose. The HelmRelease this chart renders now sends the limit on every release, so the value in the data-plane chart becomes the fallback a standalone render gets and the effective default is the one here. The arithmetic behind the number stays next to the growth count it multiplies against, which nothing overrides, and each side points at the other. The volume server's own -max was already in that position and had no test on the install path. This chart writes maxVolumes into dataDirs on every release, and because dataDirs is a list Helm replaces the data-plane chart's copy wholesale, so the zero that makes the slot count auto-derived at all comes from here. Losing it drops the server to the subchart's fixed seven, and both suites stayed green through that, so it is pinned beside the limit it divides, for the default volume and for a pool. The field is bounded at both ends, because the schema types it only as an integer and either end takes an instance down. Zero is not a smaller limit but an off switch: the volume server leaves its slot calculation untouched at zero, keeps reporting no capacity, and nothing is ever grown. Past 30000 the master does not start at all. The bound lives in the template because the schema cannot hold one, being rewritten from the annotations on every generate with no numeric bounds emitted, and a case pins each edge along with the value just outside it. The growth count is deliberately not exposed beside the limit. The master picks which of its four growth counts to apply from the replication the filer requests, and the filer takes that from replicationFactor, so a field wired to COPY_2 would do nothing at 1 or 3 copies while still appearing to be a setting. A knob that is silently inert for two of the three supported values is worse than no knob. Making it track the replication factor is a feature with its own matrix, and it would rest on a path where the master's own default replication is separately broken. The size limit has no such dependency: it is a flat master flag that divides disk capacity into slots whatever the replication is, so lowering it raises the ceiling at every supported replicationFactor. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <[email protected]>
1 parent c2c2af9 commit 33d2ac7

6 files changed

Lines changed: 260 additions & 66 deletions

File tree

‎packages/extra/seaweedfs/README.md‎

Lines changed: 65 additions & 64 deletions
Large diffs are not rendered by default.

‎packages/extra/seaweedfs/templates/seaweedfs.yaml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,23 @@
88
{{- if lt (int .Values.replicationFactor) 1 }}
99
{{- fail "Invalid value for .Values.replicationFactor. Must be at least 1." }}
1010
{{- end }}
11+
{{- /* Both ends take an instance down, so both are refused. Zero is not a
12+
smaller limit but an off switch: this value is the divisor the volume
13+
server derives its slot count with, and it leaves that calculation
14+
untouched at 0, so it keeps reporting no capacity, nothing is grown and
15+
every write fails. Above 30000 the master does not start at all, exiting
16+
on "volumeSizeLimitMB should be smaller than 30000". That ceiling is
17+
VolumeSizeLimitGB in the standard SeaweedFS build, which is the one the
18+
image carries; a large-disk build raises it, so read that constant there
19+
before widening this. The bound lives in the template because the schema
20+
cannot hold one: cozyvalues-gen rewrites values.schema.json on every
21+
make generate and emits no numeric bounds, so a hand-added minimum is
22+
erased on the next regenerate, and the ApplicationDefinition's
23+
openAPISchema is derived from that same file. Moving it there loses it
24+
silently. */}}
25+
{{- if or (lt (int .Values.master.volumeSizeLimitMB) 1) (gt (int .Values.master.volumeSizeLimitMB) 30000) }}
26+
{{- fail "Invalid value for .Values.master.volumeSizeLimitMB. Must be between 1 and 30000." }}
27+
{{- end }}
1128
{{- if eq .Values.topology "MultiZone" }}
1229
{{- if (eq (len .Values.volume.zones) 0) }}
1330
{{- fail "Zones must be defined for MultiZone topology." }}
@@ -234,6 +251,7 @@ spec:
234251
defaultReplicaPlacement: "{{ sub .Values.replicationFactor 1 }}00"
235252
{{- end }}
236253
replicas: {{ .Values.master.replicas }}
254+
volumeSizeLimitMB: {{ .Values.master.volumeSizeLimitMB }}
237255
resources: {{- include "cozy-lib.resources.defaultingSanitize" (list .Values.master.resourcesPreset .Values.master.resources $) | nindent 10 }}
238256
volume:
239257
{{ if eq .Values.topology "MultiZone" }}
Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
suite: seaweedfs master volume size limit reaches the data plane
2+
3+
# This limit is one of the two numbers the bucket ceiling divides by, so it has
4+
# to arrive at the master or the ceiling is whatever the data-plane chart
5+
# defaults to. The arithmetic behind that is in
6+
# packages/system/seaweedfs/values.yaml, next to the pair it governs.
7+
# It is not reachable any other way: the system chart is pulled by
8+
# ExternalArtifact, and the component that publishes it declares no install
9+
# section, so a Package-level values override for it is dropped before any
10+
# merge.
11+
#
12+
# The overridden case matters as much as the default. Asserting only the
13+
# default passes just as well against a template that ignores the value and
14+
# writes the same number as a literal, which is the state this suite exists to
15+
# rule out.
16+
17+
templates:
18+
- templates/seaweedfs.yaml
19+
20+
release:
21+
name: seaweedfs
22+
namespace: tenant-root
23+
24+
tests:
25+
- it: carries the default volume size limit to the master
26+
set: &platform
27+
_namespace:
28+
host: example.org
29+
ingress: tenant-root
30+
_cluster:
31+
issuer-name: letsencrypt-prod
32+
solver: http01
33+
documentSelector:
34+
path: metadata.name
35+
value: seaweedfs-system
36+
asserts:
37+
- equal:
38+
path: spec.values.seaweedfs.master.volumeSizeLimitMB
39+
value: 500
40+
41+
- it: carries an operator's volume size limit to the master
42+
set:
43+
<<: *platform
44+
master:
45+
volumeSizeLimitMB: 2000
46+
documentSelector:
47+
path: metadata.name
48+
value: seaweedfs-system
49+
asserts:
50+
- equal:
51+
path: spec.values.seaweedfs.master.volumeSizeLimitMB
52+
value: 2000
53+
54+
# The slot figures quoted beside the values in packages/system/seaweedfs are
55+
# for a 10Gi volume, which is this chart's default. Nothing else pins that
56+
# default, so raising it would leave those figures stale with no test to say
57+
# so; this case couples the two, and reddens when the default moves.
58+
- it: ships a 10Gi volume by default, the size those slot figures assume
59+
set: *platform
60+
documentSelector:
61+
path: metadata.name
62+
value: seaweedfs-system
63+
asserts:
64+
- equal:
65+
path: spec.values.seaweedfs.volume.dataDirs[0].size
66+
value: "10Gi"
67+
68+
# The divisor is only half the arithmetic: the volume server derives a slot
69+
# count at all because its -max is zero, and this template writes that zero
70+
# into dataDirs on every release. dataDirs is a list, so Helm replaces the
71+
# data-plane chart's copy wholesale and this literal is what installs run.
72+
# Losing it is a silent capacity regression rather than a render error: the
73+
# server falls back to the subchart's fixed 7, and before these two cases
74+
# existed both suites stayed green through it.
75+
#
76+
# The template writes the same literal at four sites, one per render path:
77+
# the default volume, a pool, a zone, and a pool inside a zone. All four are
78+
# covered, here and in the MultiZone case at the end.
79+
- it: keeps the volume server slot count auto-derived on every release
80+
set: *platform
81+
documentSelector:
82+
path: metadata.name
83+
value: seaweedfs-system
84+
asserts:
85+
- equal:
86+
path: spec.values.seaweedfs.volume.dataDirs[0].maxVolumes
87+
value: 0
88+
89+
- it: keeps the slot count auto-derived for a storage pool too
90+
set:
91+
<<: *platform
92+
volume:
93+
pools:
94+
fast:
95+
diskType: ssd
96+
documentSelector:
97+
path: metadata.name
98+
value: seaweedfs-system
99+
asserts:
100+
- equal:
101+
path: spec.values.seaweedfs.volumes.fast.dataDirs[0].maxVolumes
102+
value: 0
103+
104+
# Both ends of the range take an instance down while satisfying the schema,
105+
# which types the field only as an integer. Zero turns the slot calculation
106+
# off, so nothing is grown and every write fails. Past 30000 the master
107+
# refuses to start, so the instance never comes up at all.
108+
- it: refuses a volume size limit of zero
109+
set:
110+
<<: *platform
111+
master:
112+
volumeSizeLimitMB: 0
113+
asserts:
114+
- failedTemplate:
115+
errorPattern: "master.volumeSizeLimitMB. Must be between 1 and 30000"
116+
117+
- it: refuses a volume size limit past what the master accepts
118+
set:
119+
<<: *platform
120+
master:
121+
volumeSizeLimitMB: 30001
122+
asserts:
123+
- failedTemplate:
124+
errorPattern: "master.volumeSizeLimitMB. Must be between 1 and 30000"
125+
126+
- it: allows the largest volume size limit the master accepts
127+
set:
128+
<<: *platform
129+
master:
130+
volumeSizeLimitMB: 30000
131+
documentSelector:
132+
path: metadata.name
133+
value: seaweedfs-system
134+
asserts:
135+
- equal:
136+
path: spec.values.seaweedfs.master.volumeSizeLimitMB
137+
value: 30000
138+
139+
# MultiZone renders its own dataDirs per zone, and per pool within a zone, so
140+
# the zero has four sites in this template rather than two. replicationFactor
141+
# has to come down to the zone count, which the preflight enforces.
142+
- it: keeps the slot count auto-derived for a zone and a pool inside it
143+
set:
144+
<<: *platform
145+
topology: MultiZone
146+
replicationFactor: 1
147+
volume:
148+
zones:
149+
east:
150+
replicas: 2
151+
size: 10Gi
152+
pools:
153+
fast:
154+
diskType: ssd
155+
documentSelector:
156+
path: metadata.name
157+
value: seaweedfs-system
158+
asserts:
159+
- equal:
160+
path: spec.values.seaweedfs.volumes.east.dataDirs[0].maxVolumes
161+
value: 0
162+
- equal:
163+
path: spec.values.seaweedfs.volumes.east-fast.dataDirs[0].maxVolumes
164+
value: 0

‎packages/extra/seaweedfs/values.schema.json‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -236,6 +236,11 @@
236236
"xlarge",
237237
"2xlarge"
238238
]
239+
},
240+
"volumeSizeLimitMB": {
241+
"description": "Size in MB at which a volume stops taking writes, and the divisor that sets how many buckets a volume server can serve. Lower it to fit more buckets on the same disk, at the cost of more volume files for the same stored data; on a large volume it is worth raising instead.",
242+
"type": "integer",
243+
"default": 500
239244
}
240245
}
241246
},

‎packages/extra/seaweedfs/values.yaml‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,12 +92,18 @@ db:
9292
## @field {int} [replicas] - Number of master replicas.
9393
## @field {Resources} [resources] - Explicit CPU and memory configuration. When omitted, the preset defined in `resourcesPreset` is applied.
9494
## @field {ResourcesPreset} [resourcesPreset] - Default sizing preset used when `resources` is omitted.
95+
## @field {int} [volumeSizeLimitMB] - Size in MB at which a volume stops taking writes, and the divisor that sets how many buckets a volume server can serve. Lower it to fit more buckets on the same disk, at the cost of more volume files for the same stored data; on a large volume it is worth raising instead.
9596

9697
## @param {Master} [master] - Master service configuration.
9798
master:
9899
replicas: 3
99100
resources: {}
100101
resourcesPreset: "t1.small"
102+
# Effective default for every release: the template sends it, so it wins over
103+
# the data-plane chart's own value. What 500 buys and what it costs is worked
104+
# out beside that one, in packages/system/seaweedfs/values.yaml, together with
105+
# the growth count it multiplies against.
106+
volumeSizeLimitMB: 500
101107

102108
## @typedef {struct} Filer - Filer service configuration.
103109
## @field {int} [replicas] - Number of filer replicas.

0 commit comments

Comments
 (0)