Skip to content

Commit 85b7e44

Browse files
committed
test(seaweedfs): cover the bucket count a volume server can serve
A volume server started with -max 0 derives its slot count from free disk divided by the master's volume size limit, and each collection claims a batch of those slots on every volume server the replication spans. The number of buckets an instance can hold therefore has a ceiling well below what the free bytes suggest, and crossing it leaves every COSI object reporting ready while the S3 PUT times out. The existing case creates one bucket, which sits inside that ceiling at any sizing, so nothing here told a working ceiling from an exhausted one. This case creates three beside the platform's own backup bucket and requires each to accept an object, which is the only signal the exhaustion produces. Three rather than two because the count has to clear the ceiling without depending on how many slots the unnamed default collection holds, which this suite cannot observe. It runs last in the suite so its buckets are reclaimed immediately after it rather than occupying slots for the suites that follow. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <[email protected]>
1 parent 070dcb1 commit 85b7e44

2 files changed

Lines changed: 147 additions & 0 deletions

File tree

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
apiVersion: apps.cozystack.io/v1alpha1
2+
kind: Bucket
3+
metadata:
4+
name: cap1
5+
spec:
6+
users:
7+
admin: {}
8+
---
9+
apiVersion: apps.cozystack.io/v1alpha1
10+
kind: Bucket
11+
metadata:
12+
name: cap2
13+
spec:
14+
users:
15+
admin: {}
16+
---
17+
apiVersion: apps.cozystack.io/v1alpha1
18+
kind: Bucket
19+
metadata:
20+
name: cap3
21+
spec:
22+
users:
23+
admin: {}

‎hack/e2e-chainsaw/bucket/chainsaw-test.yaml‎

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -134,3 +134,127 @@ spec:
134134
($error == null): true
135135
# The Bucket CR was applied by chainsaw and is deleted automatically during
136136
# cleanup — no explicit teardown step needed.
137+
---
138+
# Slot exhaustion leaves every COSI object green and fails only the write, so
139+
# readiness alone cannot see it: the master answers `assign` with a grow that
140+
# returns no volume, and the S3 gateway times the PUT out. Each bucket here
141+
# therefore has to take an object, not just report ready.
142+
#
143+
# Three buckets, because the count has to exceed what the old volume sizing
144+
# could serve without depending on how many slots the unnamed default
145+
# collection holds — a number this suite cannot observe. At the old sizing the
146+
# platform's own cozy-backups collection plus one more exhausts the per-server
147+
# slots whether or not that default collection took any, so the third bucket
148+
# cannot be served under either reading.
149+
#
150+
# Runs last in the suite so its buckets are reclaimed immediately after it
151+
# rather than sitting through the suites that follow.
152+
apiVersion: chainsaw.kyverno.io/v1alpha1
153+
kind: Test
154+
metadata:
155+
name: bucket-1-capacity
156+
spec:
157+
catch:
158+
- events: {}
159+
- get:
160+
apiVersion: objectstorage.k8s.io/v1alpha1
161+
kind: Bucket
162+
format: yaml
163+
- podLogs:
164+
namespace: tenant-root
165+
selector: app.kubernetes.io/component=master
166+
tail: 200
167+
steps:
168+
- name: create-buckets
169+
try:
170+
- apply:
171+
file: buckets-capacity.yaml
172+
173+
- name: wait-buckets-ready
174+
try:
175+
- assert:
176+
timeout: 6m
177+
resource:
178+
apiVersion: helm.toolkit.fluxcd.io/v2
179+
kind: HelmRelease
180+
metadata:
181+
name: bucket-cap1
182+
status:
183+
(conditions[?type == 'Ready']):
184+
- status: "True"
185+
- assert:
186+
timeout: 6m
187+
resource:
188+
apiVersion: helm.toolkit.fluxcd.io/v2
189+
kind: HelmRelease
190+
metadata:
191+
name: bucket-cap2
192+
status:
193+
(conditions[?type == 'Ready']):
194+
- status: "True"
195+
- assert:
196+
timeout: 6m
197+
resource:
198+
apiVersion: helm.toolkit.fluxcd.io/v2
199+
kind: HelmRelease
200+
metadata:
201+
name: bucket-cap3
202+
status:
203+
(conditions[?type == 'Ready']):
204+
- status: "True"
205+
- assert:
206+
timeout: 2m
207+
resource:
208+
apiVersion: objectstorage.k8s.io/v1alpha1
209+
kind: BucketClaim
210+
metadata:
211+
name: bucket-cap1
212+
status:
213+
bucketReady: true
214+
- assert:
215+
timeout: 2m
216+
resource:
217+
apiVersion: objectstorage.k8s.io/v1alpha1
218+
kind: BucketClaim
219+
metadata:
220+
name: bucket-cap2
221+
status:
222+
bucketReady: true
223+
- assert:
224+
timeout: 2m
225+
resource:
226+
apiVersion: objectstorage.k8s.io/v1alpha1
227+
kind: BucketClaim
228+
metadata:
229+
name: bucket-cap3
230+
status:
231+
bucketReady: true
232+
233+
- name: verify-every-bucket-takes-a-write
234+
try:
235+
- script:
236+
timeout: 5m
237+
content: |
238+
set -eu
239+
workdir=$(mktemp -d)
240+
trap '[ -n "${pf_pid:-}" ] && kill "$pf_pid" 2>/dev/null || true; rm -rf "$workdir"' EXIT
241+
242+
kubectl port-forward service/seaweedfs-s3 -n tenant-root 8333:8333 >/dev/null 2>&1 &
243+
pf_pid=$!
244+
timeout 30 sh -ec 'until nc -z localhost 8333; do sleep 1; done'
245+
246+
echo "capacity probe" > "$workdir/probe.txt"
247+
248+
for name in cap1 cap2 cap3; do
249+
kubectl -n "$NAMESPACE" get secret "bucket-${name}-admin" \
250+
-o jsonpath='{.data.BucketInfo}' | base64 -d > "$workdir/${name}.json"
251+
ACCESS_KEY=$(jq -r '.spec.secretS3.accessKeyID' "$workdir/${name}.json")
252+
SECRET_KEY=$(jq -r '.spec.secretS3.accessSecretKey' "$workdir/${name}.json")
253+
BUCKET_NAME=$(jq -r '.spec.bucketName' "$workdir/${name}.json")
254+
255+
mc alias set "$name" https://localhost:8333 "$ACCESS_KEY" "$SECRET_KEY" --insecure
256+
mc cp --insecure "$workdir/probe.txt" "$name/$BUCKET_NAME/probe.txt"
257+
mc rm --insecure "$name/$BUCKET_NAME/probe.txt"
258+
done
259+
check:
260+
($error == null): true

0 commit comments

Comments
 (0)