Skip to content

Commit b25f294

Browse files
scooby87claude
andcommitted
fix(harbor): select kept PVCs by release label so the trivy PVC is also cleaned
The post-delete cleanup hook selected PVCs by app.kubernetes.io/instance=<release>-system, a label only the jobservice PVC carries (via the chart's "harbor.labels" helper). The trivy PVC is a StatefulSet volumeClaimTemplate labeled via "harbor.legacy.labels" with a selector using "harbor.matchLabels" — neither sets app.kubernetes.io/instance — so the sweep silently missed it. Trivy is enabled by default, so a default Harbor delete left the trivy PVC orphaned, contradicting the PR's own claim. Select on release=<release>-system instead: both kept PVCs carry it, while CNPG (db) and Redis PVCs use different label schemes and the registry is on S3, so only the two intended PVCs match. The unit test now asserts the corrected selector and guards against regressing to the instance label. Also drop the by-name delete of Orders/Challenges (their hash-suffixed names make a by-name delete a no-op; the Certificate owner-ref cascade plus the host-scoped sweep handle them), narrow the hook RBAC to challenges only, and note the Gateway-mode no-op. Assisted-By: Claude <[email protected]> Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]> Signed-off-by: Алексей Артамонов <[email protected]>
1 parent d113273 commit b25f294

2 files changed

Lines changed: 45 additions & 13 deletions

File tree

‎packages/apps/harbor/templates/hooks/cleanup.yaml‎

Lines changed: 26 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -48,26 +48,39 @@ spec:
4848
- -c
4949
- |
5050
set -u
51-
# Harbor's jobservice PVC is annotated with
51+
# Harbor's jobservice and trivy PVCs are annotated with
5252
# helm.sh/resource-policy: keep by the upstream subchart
5353
# (persistence.resourcePolicy: keep), so Helm/Flux deliberately
54-
# leaves it behind on uninstall. The subchart runs as the
55-
# "{{ .Release.Name }}-system" HelmRelease, so its PVCs carry
56-
# app.kubernetes.io/instance={{ .Release.Name }}-system.
54+
# leave them behind on uninstall. The subchart runs as the
55+
# "{{ .Release.Name }}-system" HelmRelease, so both PVCs carry the
56+
# label release={{ .Release.Name }}-system: jobservice via the
57+
# chart's "harbor.labels" helper, and trivy's StatefulSet PVC via
58+
# "harbor.legacy.labels". We must select on "release" (not
59+
# app.kubernetes.io/instance) because the trivy PVC does NOT carry
60+
# app.kubernetes.io/instance, so an instance-label sweep would
61+
# silently miss trivy (enabled by default). CNPG (db) and Redis
62+
# PVCs use different label schemes and the registry is on S3, so
63+
# this selector matches only those two kept PVCs.
5764
echo "Deleting orphaned PVCs for {{ .Release.Name }}-system..."
5865
kubectl delete pvc -n {{ .Release.Namespace }} \
59-
-l app.kubernetes.io/instance={{ .Release.Name }}-system --ignore-not-found \
66+
-l release={{ .Release.Name }}-system --ignore-not-found \
6067
|| echo "WARNING: PVC cleanup failed for {{ .Release.Name }}; orphaned PVCs may remain" >&2
6168
62-
# Remove cert-manager ACME objects owned by this release's
69+
# Remove the cert-manager Certificate owned by this release's
6370
# Ingress so an in-flight http01 challenge does not leave a
6471
# dangling cm-acme-http-solver-* Pod/Service/Ingress behind.
6572
# Deleting the release's Certificate (deterministic name
6673
# "{{ .Release.Name }}-ingress-tls") triggers cert-manager to
6774
# cascade-delete the Orders, Challenges and solver
68-
# Pod/Service/Ingress objects it owns for this release.
69-
echo "Deleting ACME objects for {{ .Release.Name }}-ingress..."
70-
kubectl delete certificate,order,challenge -n {{ .Release.Namespace }} \
75+
# Pod/Service/Ingress objects it owns via owner references.
76+
# We delete only the Certificate by name: Orders/Challenges have
77+
# hash-suffixed names, so a by-name delete of those kinds would
78+
# be a no-op (the host-scoped sweep below covers them instead).
79+
# In Gateway mode no per-app Ingress or "-ingress-tls"
80+
# Certificate is rendered, so this delete is simply a no-op there
81+
# and the dnsName-scoped sweep below still handles http01.
82+
echo "Deleting ACME Certificate for {{ .Release.Name }}-ingress..."
83+
kubectl delete certificate -n {{ .Release.Namespace }} \
7184
{{ .Release.Name }}-ingress-tls --ignore-not-found \
7285
|| echo "WARNING: cert-manager cleanup failed for {{ .Release.Name }}; orphaned ACME resources may remain" >&2
7386
@@ -132,8 +145,11 @@ rules:
132145
- apiGroups: ["cert-manager.io"]
133146
resources: ["certificates"]
134147
verbs: ["get", "list", "delete"]
148+
# Only Challenges are touched directly (the host-scoped sweep lists and
149+
# deletes them); Orders are cascade-deleted by cert-manager itself when the
150+
# Certificate is removed, so the hook needs no RBAC for them.
135151
- apiGroups: ["acme.cert-manager.io"]
136-
resources: ["orders", "challenges"]
152+
resources: ["challenges"]
137153
verbs: ["get", "list", "delete"]
138154
# We list ACME http01 solver Ingresses (filtered by this release's host)
139155
# and delete the orphaned ones by name.

‎packages/apps/harbor/tests/cleanup_hook_test.yaml‎

Lines changed: 19 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -29,13 +29,20 @@ tests:
2929
path: spec.template.spec.serviceAccountName
3030
value: harbor-test-cleanup
3131

32-
- it: targets the jobservice PVC label (release-system instance) and ACME objects
32+
- it: targets both kept PVCs (jobservice + trivy) via the shared release label and ACME objects
3333
documentIndex: 0
3434
asserts:
3535
# The upstream Harbor subchart runs as the "<release>-system"
36-
# HelmRelease, so its kept jobservice PVC is labeled
37-
# app.kubernetes.io/instance=<release>-system, not <release>.
36+
# HelmRelease. Its kept PVCs must be selected by release=<release>-system:
37+
# jobservice carries it via "harbor.labels", and trivy's StatefulSet PVC
38+
# carries it via "harbor.legacy.labels". The trivy PVC does NOT carry
39+
# app.kubernetes.io/instance, so selecting on that label would silently
40+
# miss trivy (which is enabled by default).
3841
- matchRegex:
42+
path: spec.template.spec.containers[0].command[2]
43+
pattern: kubectl delete pvc[\s\S]*-l release=harbor-test-system
44+
# Guard against regressing to the instance label, which misses trivy.
45+
- notMatchRegex:
3946
path: spec.template.spec.containers[0].command[2]
4047
pattern: app.kubernetes.io/instance=harbor-test-system
4148
- matchRegex:
@@ -114,7 +121,16 @@ tests:
114121
apiGroups: ["cert-manager.io"]
115122
resources: ["certificates"]
116123
verbs: ["get", "list", "delete"]
124+
# Only "challenges" are granted: the host-scoped sweep lists/deletes
125+
# them, while Orders are cascade-deleted by cert-manager itself when the
126+
# Certificate is removed, so the hook needs no RBAC for orders.
117127
- contains:
128+
path: rules
129+
content:
130+
apiGroups: ["acme.cert-manager.io"]
131+
resources: ["challenges"]
132+
verbs: ["get", "list", "delete"]
133+
- notContains:
118134
path: rules
119135
content:
120136
apiGroups: ["acme.cert-manager.io"]

0 commit comments

Comments
 (0)