Repository navigation
Commit 1e5f538
authored
fix(kamaji): backport datastore unused-deletion deadlock fix (clastix/kamaji#1122) (#3146)
## What
Backports the upstream Kamaji fix
[clastix/kamaji#1122](clastix/kamaji#1122)
(which fixes clastix/kamaji#1115) as a source patch on top of the pinned
`26.3.6-edge` build. No version bump, no CRD/RBAC changes.
This lands the fix on `main`; the `backport` / `backport-previous`
labels fan it out to `release-1.5` and `release-1.4` via the backport
automation. Supersedes the per-release PRs #3021 (release-1.5) and #3020
(release-1.4).
## Why
Deleting a Tenant whose etcd `DataStore` has **0 referencing
`TenantControlPlane`s** — e.g. a tenant deleted before any Kubernetes
app / TCP is ever created in it — leaves the `DataStore` stuck
`Terminating` with finalizer `kamaji.clastix.io/TenantControlPlane`.
That hangs `helm uninstall <etcd>` (which waits by default), freezes the
etcd HelmRelease at `status=uninstalling`, and wedges the tenant
namespace in `Terminating` indefinitely. Tracked in #3018.
Root cause: in `26.3.x-edge`, DataStore deletion drives an
`AllowedDeletion` condition that only transitions `False → True`. With 0
referencing TCPs at deletion time the condition is never initialized, so
the reconcile falls through to logging `DataStore can be safely deleted`
and the deferred finalizer-removal (gated on `IsStatusConditionTrue`)
never runs — with no requeue to retry. The pinned `26.3.6-edge` predates
the upstream fix (merged 2026-04-14, first released in `26.4.4-edge`).
## Change
Adds
`packages/system/kamaji/images/kamaji/patches/fix-datastore-unused-deletion.diff`,
applied by the existing `RUN git apply /patches/*.diff` step in the
kamaji image build. The one-line guard adds the missing 0-TCP path:
```go
- if meta.IsStatusConditionFalse(..., DataStoreConditionAllowedDeletionType) {
+ if meta.IsStatusConditionFalse(..., DataStoreConditionAllowedDeletionType) || len(tcpList.Items) == 0 {
```
## Test
- `git apply --check` confirms the patch applies cleanly to the
`26.3.6-edge` source pinned on `main`; after applying, the guard is
present at `controllers/datastore_controller.go`.
- Reproduced the deadlock on a v1.4.x cluster (tenant deleted before any
TCP existed) and confirmed the manual unstick; this patch removes the
need for it.
Workaround for an already-stuck DataStore (0 TCP refs): `kubectl patch
datastore <name> --type=merge -p '{"metadata":{"finalizers":null}}'`.
Relates to #2885 (full kamaji bump for CVEs — supersedes this patch once
it lands).
```release-note
Fix tenant namespace stuck `Terminating` when an etcd DataStore with no referencing TenantControlPlanes is deleted (Kamaji datastore unused-deletion deadlock).
```
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Data stores are now recognized as unused even when no matching tenant
control plane resources are found, improving deletion handling in that
case.
* This prevents inactive data stores from being left behind when there
are no associated matches.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->1 file changed
Lines changed: 12 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
0 commit comments