Skip to content

Commit 1e5f538

Browse files
fix(kamaji): backport datastore unused-deletion deadlock fix (clastix/kamaji#1122) (#3146)
## What Backports the upstream Kamaji fix [clastix/kamaji#1122](clastix/kamaji#1122) (which fixes clastix/kamaji#1115) as a source patch on top of the pinned `26.3.6-edge` build. No version bump, no CRD/RBAC changes. This lands the fix on `main`; the `backport` / `backport-previous` labels fan it out to `release-1.5` and `release-1.4` via the backport automation. Supersedes the per-release PRs #3021 (release-1.5) and #3020 (release-1.4). ## Why Deleting a Tenant whose etcd `DataStore` has **0 referencing `TenantControlPlane`s** — e.g. a tenant deleted before any Kubernetes app / TCP is ever created in it — leaves the `DataStore` stuck `Terminating` with finalizer `kamaji.clastix.io/TenantControlPlane`. That hangs `helm uninstall <etcd>` (which waits by default), freezes the etcd HelmRelease at `status=uninstalling`, and wedges the tenant namespace in `Terminating` indefinitely. Tracked in #3018. Root cause: in `26.3.x-edge`, DataStore deletion drives an `AllowedDeletion` condition that only transitions `False → True`. With 0 referencing TCPs at deletion time the condition is never initialized, so the reconcile falls through to logging `DataStore can be safely deleted` and the deferred finalizer-removal (gated on `IsStatusConditionTrue`) never runs — with no requeue to retry. The pinned `26.3.6-edge` predates the upstream fix (merged 2026-04-14, first released in `26.4.4-edge`). ## Change Adds `packages/system/kamaji/images/kamaji/patches/fix-datastore-unused-deletion.diff`, applied by the existing `RUN git apply /patches/*.diff` step in the kamaji image build. The one-line guard adds the missing 0-TCP path: ```go - if meta.IsStatusConditionFalse(..., DataStoreConditionAllowedDeletionType) { + if meta.IsStatusConditionFalse(..., DataStoreConditionAllowedDeletionType) || len(tcpList.Items) == 0 { ``` ## Test - `git apply --check` confirms the patch applies cleanly to the `26.3.6-edge` source pinned on `main`; after applying, the guard is present at `controllers/datastore_controller.go`. - Reproduced the deadlock on a v1.4.x cluster (tenant deleted before any TCP existed) and confirmed the manual unstick; this patch removes the need for it. Workaround for an already-stuck DataStore (0 TCP refs): `kubectl patch datastore <name> --type=merge -p '{"metadata":{"finalizers":null}}'`. Relates to #2885 (full kamaji bump for CVEs — supersedes this patch once it lands). ```release-note Fix tenant namespace stuck `Terminating` when an etcd DataStore with no referencing TenantControlPlanes is deleted (Kamaji datastore unused-deletion deadlock). ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Data stores are now recognized as unused even when no matching tenant control plane resources are found, improving deletion handling in that case. * This prevents inactive data stores from being left behind when there are no associated matches. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 parents 5b261b9 + 47638f8 commit 1e5f538

1 file changed

Lines changed: 12 additions & 0 deletions

File tree

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
diff --git a/controllers/datastore_controller.go b/controllers/datastore_controller.go
2+
--- a/controllers/datastore_controller.go
3+
+++ b/controllers/datastore_controller.go
4+
@@ -151,7 +151,7 @@ func (r *DataStore) Reconcile(ctx context.Context, request reconcile.Request) (reconcile.Result, error) {
5+
return reconcile.Result{}, nil
6+
}
7+
8+
- if meta.IsStatusConditionFalse(ds.Status.Conditions, kamajiv1alpha1.DataStoreConditionAllowedDeletionType) {
9+
+ if meta.IsStatusConditionFalse(ds.Status.Conditions, kamajiv1alpha1.DataStoreConditionAllowedDeletionType) || len(tcpList.Items) == 0 {
10+
logger.Info("DataStore is not used by any TenantControlPlane object")
11+
12+
meta.SetStatusCondition(&ds.Status.Conditions, metav1.Condition{

0 commit comments

Comments
 (0)