Skip to content

fix(ds): allow deletion for unused datastores - #1122

Merged
prometherion merged 1 commit into
clastix:masterfrom
prometherion:issues/1115
Apr 14, 2026
Merged

prometherion merged 1 commit into
clastix:masterfrom
prometherion:issues/1115

Conversation

@prometherion

Copy link
Copy Markdown
Member

Closes #1115.

@netlify

netlify Bot commented Apr 13, 2026 •

Copy link
Copy Markdown

✅ Deploy Preview for kamaji-documentation canceled.

Name Link
🔨 Latest commit 3d088c1
🔍 Latest deploy log https://app.netlify.com/projects/kamaji-documentation/deploys/69ddf9bedaaf6f0008350676

@prometherion
prometherion merged commit 83c99ab into clastix:master Apr 14, 2026
12 checks passed
@prometherion
prometherion deleted the issues/1115 branch April 14, 2026 09:32
myasnikovdaniil added a commit to cozystack/cozystack that referenced this pull request Jul 3, 2026
…/kamaji#1122)

Deleting a Tenant whose etcd DataStore has 0 referencing TenantControlPlanes left the DataStore stuck Terminating (finalizer kamaji.clastix.io/TenantControlPlane), hanging the etcd helm uninstall and wedging the tenant namespace. Backports clastix/kamaji#1122 (fixes clastix/kamaji#1115) as a source patch on the pinned 26.3.6-edge build. Refs #3018.

Signed-off-by: Myasnikov Daniil <[email protected]>
myasnikovdaniil added a commit to cozystack/cozystack that referenced this pull request Jul 3, 2026
…/kamaji#1122) (#3146)

## What

Backports the upstream Kamaji fix
[clastix/kamaji#1122](clastix/kamaji#1122)
(which fixes clastix/kamaji#1115) as a source patch on top of the pinned
`26.3.6-edge` build. No version bump, no CRD/RBAC changes.

This lands the fix on `main`; the `backport` / `backport-previous`
labels fan it out to `release-1.5` and `release-1.4` via the backport
automation. Supersedes the per-release PRs #3021 (release-1.5) and #3020
(release-1.4).

## Why

Deleting a Tenant whose etcd `DataStore` has **0 referencing
`TenantControlPlane`s** — e.g. a tenant deleted before any Kubernetes
app / TCP is ever created in it — leaves the `DataStore` stuck
`Terminating` with finalizer `kamaji.clastix.io/TenantControlPlane`.
That hangs `helm uninstall <etcd>` (which waits by default), freezes the
etcd HelmRelease at `status=uninstalling`, and wedges the tenant
namespace in `Terminating` indefinitely. Tracked in #3018.

Root cause: in `26.3.x-edge`, DataStore deletion drives an
`AllowedDeletion` condition that only transitions `False → True`. With 0
referencing TCPs at deletion time the condition is never initialized, so
the reconcile falls through to logging `DataStore can be safely deleted`
and the deferred finalizer-removal (gated on `IsStatusConditionTrue`)
never runs — with no requeue to retry. The pinned `26.3.6-edge` predates
the upstream fix (merged 2026-04-14, first released in `26.4.4-edge`).

## Change

Adds
`packages/system/kamaji/images/kamaji/patches/fix-datastore-unused-deletion.diff`,
applied by the existing `RUN git apply /patches/*.diff` step in the
kamaji image build. The one-line guard adds the missing 0-TCP path:

```go
- if meta.IsStatusConditionFalse(..., DataStoreConditionAllowedDeletionType) {
+ if meta.IsStatusConditionFalse(..., DataStoreConditionAllowedDeletionType) || len(tcpList.Items) == 0 {
```

## Test

- `git apply --check` confirms the patch applies cleanly to the
`26.3.6-edge` source pinned on `main`; after applying, the guard is
present at `controllers/datastore_controller.go`.
- Reproduced the deadlock on a v1.4.x cluster (tenant deleted before any
TCP existed) and confirmed the manual unstick; this patch removes the
need for it.

Workaround for an already-stuck DataStore (0 TCP refs): `kubectl patch
datastore <name> --type=merge -p '{"metadata":{"finalizers":null}}'`.

Relates to #2885 (full kamaji bump for CVEs — supersedes this patch once
it lands).

```release-note
Fix tenant namespace stuck `Terminating` when an etcd DataStore with no referencing TenantControlPlanes is deleted (Kamaji datastore unused-deletion deadlock).
```


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Data stores are now recognized as unused even when no matching tenant
control plane resources are found, improving deletion handling in that
case.
* This prevents inactive data stores from being left behind when there
are no associated matches.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
myasnikovdaniil added a commit to cozystack/cozystack that referenced this pull request Jul 3, 2026
…n deadlock fix (clastix/kamaji#1122) (#3188)

# Description
Backport of #3146 to `release-1.4`.
myasnikovdaniil added a commit to cozystack/cozystack that referenced this pull request Jul 3, 2026
…n deadlock fix (clastix/kamaji#1122) (#3187)

# Description
Backport of #3146 to `release-1.5`.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

DataStore finalizer never removed when TenantControlPlane is deleted before DataStore deletion is triggered

1 participant