Skip to content

Commit c402f95

Browse files
kvapsclaude
authored andcommitted
fix(seaweedfs): restore S3 TLS and COSI provisioner SA after 4.31 bump
The 4.31 chart bump (#2834) broke E2E for every PR: tenant-root seaweedfs-system never becomes ready. 1. S3 served plaintext. Until 4.05 the -cert.file/-key.file args were rendered whenever enableSecurity was set, so weed s3 served TLS on the main port (8333); 4.31 nested them under 'if s3.httpsPort' and we ship httpsPort=0, so s3 spoke HTTP while the HTTPS probes, the ingress backend-protocol and the COSI sidecar endpoint all expected HTTPS ('server gave HTTP response to HTTPS client'). Restore the 4.05 behaviour via patch. 2. COSI provisioner ServiceAccount mismatch. cosi-service-account.yaml and the ClusterRoleBinding name the SA <serviceAccountName>-objectstorage-provisioner, but the Deployment referenced componentName (<release>-objectstorage-provisioner). When serviceAccountName != release name (tenant-root-seaweedfs vs seaweedfs-system) the Deployment could not create pods ('serviceaccount not found'). Align the Deployment with the SA + binding. Both fixes are vendored as patches re-applied by 'make update', and covered by helm-unittest. Co-Authored-By: Claude <[email protected]> Signed-off-by: Andrei Kvapil <[email protected]> Signed-off-by: Ivan Okhotnikov <[email protected]>
1 parent 93e5db8 commit c402f95

6 files changed

Lines changed: 109 additions & 2 deletions

File tree

‎packages/system/seaweedfs/Makefile‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,5 +14,7 @@ update:
1414
tar xzvf - --strip 3 -C charts seaweedfs-$${version}/k8s/charts/seaweedfs
1515
patch --no-backup-if-mismatch -p4 < patches/resize-api-server-annotation.diff
1616
patch --no-backup-if-mismatch -p4 < patches/disable-ca-key-rotation.patch
17+
patch --no-backup-if-mismatch -p4 < patches/s3-tls-main-port.patch
18+
patch --no-backup-if-mismatch -p4 < patches/cosi-provisioner-sa-name.patch
1719
#patch --no-backup-if-mismatch -p4 < patches/retention-policy-delete.yaml
1820

‎packages/system/seaweedfs/charts/seaweedfs/templates/cosi/cosi-deployment.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ spec:
5858
priorityClassName: {{ .Values.cosi.priorityClassName | quote }}
5959
{{- end }}
6060
enableServiceLinks: false
61-
serviceAccountName: {{ include "seaweedfs.componentName" (list . "objectstorage-provisioner") }}
61+
serviceAccountName: {{ .Values.global.seaweedfs.serviceAccountName }}-objectstorage-provisioner
6262
{{- if .Values.cosi.initContainers }}
6363
initContainers:
6464
{{ tpl .Values.cosi.initContainers . | nindent 8 | trim }}

‎packages/system/seaweedfs/charts/seaweedfs/templates/s3/s3-deployment.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -127,8 +127,8 @@ spec:
127127
{{- if .Values.global.seaweedfs.enableSecurity }}
128128
{{- if .Values.s3.httpsPort }}
129129
-port.https={{ .Values.s3.httpsPort }} \
130-
{{- include "seaweedfs.s3.tlsArgs" (dict "root" . "prefix" "") | nindent 14 }}
131130
{{- end }}
131+
{{- include "seaweedfs.s3.tlsArgs" (dict "root" . "prefix" "") | nindent 14 }}
132132
{{- end }}
133133
{{- if .Values.s3.domainName }}
134134
-domainName={{ .Values.s3.domainName }} \
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
diff --git a/packages/system/seaweedfs/charts/seaweedfs/templates/cosi/cosi-deployment.yaml b/packages/system/seaweedfs/charts/seaweedfs/templates/cosi/cosi-deployment.yaml
2+
index aefc9d369..d485e0678 100644
3+
--- a/packages/system/seaweedfs/charts/seaweedfs/templates/cosi/cosi-deployment.yaml
4+
+++ b/packages/system/seaweedfs/charts/seaweedfs/templates/cosi/cosi-deployment.yaml
5+
@@ -58,7 +58,7 @@ spec:
6+
priorityClassName: {{ .Values.cosi.priorityClassName | quote }}
7+
{{- end }}
8+
enableServiceLinks: false
9+
- serviceAccountName: {{ include "seaweedfs.componentName" (list . "objectstorage-provisioner") }}
10+
+ serviceAccountName: {{ .Values.global.seaweedfs.serviceAccountName }}-objectstorage-provisioner
11+
{{- if .Values.cosi.initContainers }}
12+
initContainers:
13+
{{ tpl .Values.cosi.initContainers . | nindent 8 | trim }}
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
diff --git a/packages/system/seaweedfs/charts/seaweedfs/templates/s3/s3-deployment.yaml b/packages/system/seaweedfs/charts/seaweedfs/templates/s3/s3-deployment.yaml
2+
index db231fc96..efb967bb2 100644
3+
--- a/packages/system/seaweedfs/charts/seaweedfs/templates/s3/s3-deployment.yaml
4+
+++ b/packages/system/seaweedfs/charts/seaweedfs/templates/s3/s3-deployment.yaml
5+
@@ -127,8 +127,8 @@ spec:
6+
{{- if .Values.global.seaweedfs.enableSecurity }}
7+
{{- if .Values.s3.httpsPort }}
8+
-port.https={{ .Values.s3.httpsPort }} \
9+
- {{- include "seaweedfs.s3.tlsArgs" (dict "root" . "prefix" "") | nindent 14 }}
10+
{{- end }}
11+
+ {{- include "seaweedfs.s3.tlsArgs" (dict "root" . "prefix" "") | nindent 14 }}
12+
{{- end }}
13+
{{- if .Values.s3.domainName }}
14+
-domainName={{ .Values.s3.domainName }} \
Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
suite: seaweedfs s3 TLS on main port + COSI provisioner SA name
2+
3+
# Pins two regressions introduced by the 4.31 chart bump:
4+
#
5+
# 1. S3 TLS. Until 4.05 the `-cert.file`/`-key.file` args were rendered whenever
6+
# enableSecurity was on, so `weed s3` served TLS on the main port (8333) and
7+
# our HTTPS readiness/liveness probes (and the ingress backend-protocol=HTTPS
8+
# and the COSI sidecar's https:// endpoint) worked. 4.31 nested those args
9+
# inside `if .Values.s3.httpsPort`, and we ship httpsPort=0, so s3 served
10+
# plaintext and every probe failed with "server gave HTTP response to HTTPS
11+
# client". The s3-tls-main-port.patch restores the 4.05 behaviour.
12+
#
13+
# 2. COSI provisioner ServiceAccount. cosi-service-account.yaml and the
14+
# ClusterRoleBinding both name the SA <serviceAccountName>-objectstorage-provisioner,
15+
# but the Deployment referenced componentName (<release>-objectstorage-provisioner).
16+
# Whenever serviceAccountName != release name the Deployment could not create
17+
# pods ("serviceaccount not found"). The cosi-provisioner-sa-name.patch aligns
18+
# the Deployment with the SA + binding.
19+
20+
templates:
21+
- charts/seaweedfs/templates/s3/s3-deployment.yaml
22+
- charts/seaweedfs/templates/cosi/cosi-deployment.yaml
23+
24+
release:
25+
name: seaweedfs
26+
namespace: cozy-seaweedfs
27+
28+
tests:
29+
- it: renders s3 TLS cert/key args under enableSecurity even when httpsPort is 0
30+
template: charts/seaweedfs/templates/s3/s3-deployment.yaml
31+
set:
32+
global.seaweedfs.enableSecurity: true
33+
seaweedfs.s3.enabled: true
34+
seaweedfs.s3.httpsPort: 0
35+
asserts:
36+
- matchRegex:
37+
path: spec.template.spec.containers[0].command[2]
38+
pattern: "-cert\\.file=/usr/local/share/ca-certificates/client/tls\\.crt"
39+
- matchRegex:
40+
path: spec.template.spec.containers[0].command[2]
41+
pattern: "-key\\.file=/usr/local/share/ca-certificates/client/tls\\.key"
42+
- notMatchRegex:
43+
path: spec.template.spec.containers[0].command[2]
44+
pattern: "-port\\.https"
45+
46+
- it: still gates the cert args on enableSecurity
47+
template: charts/seaweedfs/templates/s3/s3-deployment.yaml
48+
set:
49+
global.seaweedfs.enableSecurity: false
50+
seaweedfs.s3.enabled: true
51+
asserts:
52+
- notMatchRegex:
53+
path: spec.template.spec.containers[0].command[2]
54+
pattern: "-cert\\.file"
55+
56+
- it: renders -port.https alongside the cert args when httpsPort is set
57+
template: charts/seaweedfs/templates/s3/s3-deployment.yaml
58+
set:
59+
global.seaweedfs.enableSecurity: true
60+
seaweedfs.s3.enabled: true
61+
seaweedfs.s3.httpsPort: 8334
62+
asserts:
63+
- matchRegex:
64+
path: spec.template.spec.containers[0].command[2]
65+
pattern: "-port\\.https=8334"
66+
- matchRegex:
67+
path: spec.template.spec.containers[0].command[2]
68+
pattern: "-cert\\.file=/usr/local/share/ca-certificates/client/tls\\.crt"
69+
70+
- it: COSI provisioner Deployment uses the serviceAccountName-based SA, not componentName
71+
template: charts/seaweedfs/templates/cosi/cosi-deployment.yaml
72+
set:
73+
seaweedfs.cosi.enabled: true
74+
global.seaweedfs.serviceAccountName: custom-sa
75+
asserts:
76+
- equal:
77+
path: spec.template.spec.serviceAccountName
78+
value: custom-sa-objectstorage-provisioner

0 commit comments

Comments
 (0)