Skip to content

Commit 609c219

Browse files
committed
docs(compute-plane): derive release names; kubeconfig Secret becomes computeplane-cluster-admin-kubeconfig
Follow-up from the cozystack/cozystack#3280 review: the aggregated API rebuilds HelmRelease specs without spec.releaseName, so neither the module nor the cluster HelmRelease may set it — release names derive from object names. The stable external contract is therefore the cluster HelmRelease object name computeplane-cluster, and the Kamaji admin-kubeconfig Secret the placement: ComputePlane injection consumes is computeplane-cluster-admin-kubeconfig. Assisted-By: Claude Signed-off-by: Andrei Kvapil <[email protected]>
1 parent a3749c0 commit 609c219

1 file changed

Lines changed: 8 additions & 9 deletions

File tree

‎design-proposals/compute-plane/README.md‎

Lines changed: 8 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -79,7 +79,7 @@ The platform already isolates *tenant-owned* untrusted compute (a managed `kind:
7979
Reuse the `kubernetes` app as the substrate, unchanged. A ComputePlane is delivered as a new Tenant module, `packages/extra/computeplane`, of the same shape as `extra/etcd` / `extra/seaweedfs`. The packaging is **two-level** — the module chart is the wrapper, and the wrapper is what carries the operator-fixed values:
8080

8181
1. When a tenant has the module enabled, the **tenant chart** renders the module `HelmRelease` (name `computeplane`, in the tenant namespace on the management cluster) whose `chartRef` points at the **`extra/computeplane`** chart's `ExternalArtifact`. Its own Helm release takes the `-module` suffix so the canonical release name `computeplane` stays reserved for the cluster release below.
82-
2. The **`extra/computeplane` chart** in turn renders the cluster `HelmRelease` (object name `computeplane-cluster`, `spec.releaseName: computeplane`) whose `chartRef` points at the **source-only re-sourced `apps/kubernetes`** `ExternalArtifact`, carrying the operator-fixed values. The release name `computeplane` is what makes Kamaji write the admin kubeconfig to the `computeplane-admin-kubeconfig` Secret — the contract `placement: ComputePlane` apps consume.
82+
2. The **`extra/computeplane` chart** in turn renders the cluster `HelmRelease` (object name `computeplane-cluster`) whose `chartRef` points at the **source-only re-sourced `apps/kubernetes`** `ExternalArtifact`, carrying the operator-fixed values. Neither `HelmRelease` sets `spec.releaseName` — the aggregated API rebuilds `HelmRelease` specs without that field, so both releases must derive their names from object names. The derived cluster release name `computeplane-cluster` is what makes Kamaji write the admin kubeconfig to the `computeplane-cluster-admin-kubeconfig` Secret — the contract `placement: ComputePlane` apps consume.
8383

8484
```yaml
8585
# 1) tenant chart → module HelmRelease (illustrative)
@@ -89,7 +89,6 @@ metadata:
8989
name: computeplane
9090
namespace: tenant-<name> # management cluster, tenant namespace
9191
spec:
92-
releaseName: computeplane-module # frees the canonical name for the cluster release
9392
chartRef:
9493
kind: ExternalArtifact
9594
name: cozystack-computeplane-application-kubevirt-computeplane # extra/computeplane
@@ -105,7 +104,7 @@ metadata:
105104
name: computeplane-cluster
106105
namespace: tenant-<name>
107106
spec:
108-
releaseName: computeplane # → Kamaji Secret computeplane-admin-kubeconfig
107+
# no releaseName: derived from the object name → Kamaji Secret computeplane-cluster-admin-kubeconfig
109108
chartRef:
110109
kind: ExternalArtifact
111110
name: cozystack-computeplane-application-kubevirt-kubernetes # apps/kubernetes, re-sourced (Design §2)
@@ -118,7 +117,7 @@ spec:
118117
name: cozystack-values
119118
```
120119
121-
**The release name `computeplane` is a stable external contract.** For every other tenant module (`etcd`, `monitoring`, `seaweedfs`) the Helm release name is *internal identity* — nothing outside the module resolves it by name. Here it is consumed by objects the module never sees: `apps/kubernetes` derives the Kamaji admin-kubeconfig Secret from its release name (`<release>-admin-kubeconfig`, key `super-admin.svc`), and every `placement: ComputePlane` HelmRelease references `computeplane-admin-kubeconfig` by that fixed name. No future repackaging, migration, or generic module-provisioning path may rename this release. A rename would, at best, dangle every routed app's `secretRef` (fail-closed, consistent with the missing-credential behaviour below); at worst, Helm treats the cluster as a *different release* and uninstalls the old one — a live Kamaji control plane plus KubeVirt-VM workers and their PVCs, i.e. tenant data. The implementation pins the release name and Secret name in helm-unittest; this paragraph is the *why* behind those pins.
120+
**The cluster `HelmRelease` object name `computeplane-cluster` is a stable external contract.** For every other tenant module (`etcd`, `monitoring`, `seaweedfs`) the inner release identity is *internal* — nothing outside the module resolves it by name. Here it is consumed by objects the module never sees: `apps/kubernetes` derives the Kamaji admin-kubeconfig Secret from the Helm release name (`<release>-admin-kubeconfig`, key `super-admin.svc`); the release name derives from the object name because **neither `HelmRelease` may set `spec.releaseName`** — the aggregated `cozystack-api` rebuilds `HelmRelease` specs on every write and cannot represent that field, so an explicit release name would be silently dropped on the first tenant edit and the release renamed out from under Helm. Every `placement: ComputePlane` HelmRelease therefore references `computeplane-cluster-admin-kubeconfig` by that derived name, and the module's `ApplicationDefinition` withholds the same name from tenant-visible secrets. No future repackaging, migration, or generic module-provisioning path may rename the `computeplane-cluster` object. A rename would, at best, dangle every routed app's `secretRef` (fail-closed, consistent with the missing-credential behaviour below); at worst it re-exposes the credential (the secrets exclusion matches by literal name) or has Helm treat the cluster as a *different release* and uninstall the old one — a live Kamaji control plane plus KubeVirt-VM workers and their PVCs, i.e. tenant data. The implementation pins the object name, the derived Secret name and the secrets exclusion in helm-unittest and a bats cross-check; this paragraph is the *why* behind those pins.
122121
123122
Why a module wrapping the app, rather than a new kind or new fields on the app:
124123
@@ -134,7 +133,7 @@ flowchart TB
134133
subgraph mgmt["Management / infra cluster (control plane)"]
135134
MOD["extra/computeplane module (enabled on tenant)<br/>→ HelmRelease chartRef: apps/kubernetes<br/>+ operator-fixed values"]
136135
AD["ApplicationDefinition (kind: JupyterHub)<br/>placement: ComputePlane"]
137-
HR["HelmRelease<br/>kubeConfig.secretRef → computeplane-admin-kubeconfig"]
136+
HR["HelmRelease<br/>kubeConfig.secretRef → computeplane-cluster-admin-kubeconfig"]
138137
IAP["ingress / identity-aware proxy"]
139138
end
140139
subgraph cp["ComputePlane = kind: Kubernetes (single-tenant, operator-controlled)"]
@@ -157,21 +156,21 @@ The module is registered by a `PackageSource` (`packages/core/platform/sources/c
157156
- `kubernetes` → `path: apps/kubernetes` — **source-only**, a deliberate **duplicate** of the component already in `kubernetes-application`, so the wrapped chart's `ExternalArtifact` is materialized within this package's own scope and the module is self-contained (it does not depend on the `kubernetes-application` source being independently enabled).
158157
- `computeplane-rd` → `path: system/computeplane-rd` with `install:` — the resource-definition chart carrying the module's `cozyrd` (its `ApplicationDefinition` / dashboard presentation and the tenant-values wiring), the same role `seaweedfs-rd` / `kubernetes-rd` play.
159158

160-
The tenant chart gains a `computeplane` bool toggle alongside the existing module toggles (a profile-name string is a possible later extension if the operator ships more than one module variant); when set by the parent tenant, `packages/apps/tenant/templates/computeplane.yaml` renders the module `HelmRelease` (level 1 above). Kamaji writes the cluster's admin kubeconfig to `computeplane-admin-kubeconfig` (key `super-admin.svc`), which `placement: ComputePlane` apps consume (Design §4).
159+
The tenant chart gains a `computeplane` bool toggle alongside the existing module toggles (a profile-name string is a possible later extension if the operator ships more than one module variant); when set by the parent tenant, `packages/apps/tenant/templates/computeplane.yaml` renders the module `HelmRelease` (level 1 above). Kamaji writes the cluster's admin kubeconfig to `computeplane-cluster-admin-kubeconfig` (key `super-admin.svc`), which `placement: ComputePlane` apps consume (Design §4).
161160

162161
### 3. What the operator exposes vs fixes
163162

164163
Everything security-relevant is **fixed** in the module chart and unreachable by the tenant: the hardened PSA/admission profile, the deny-egress-to-management-kube-API policy, the scoped per-service egress contract (Design §5), the withheld-kubeconfig posture, and the addon set. The operator may **expose** a curated, safe subset as tenant-settable knobs through the module's `values.yaml` (e.g. GPU on/off and type, cluster size / autoscaling bounds, Kubernetes version) — the same "high-level API, operator-controlled" contract every other managed service follows. There is deliberately **no** general `valuesOverride` passthrough onto the wrapped `kind: Kubernetes`: that would re-open the tamper surface and re-create the two-sources-of-truth problem. Different postures (e.g. a future *trusted*/`cluster-meshed` variant) are separate module variants or separate `extra` charts, not a field on the app.
165164

166165
### 4. `placement` routes a catalog app onto the ComputePlane
167166

168-
`ApplicationDefinition` gains `placement` = `ManagementPlane` (default) | `ComputePlane`. `ManagementPlane` deploys into the tenant namespace on the management cluster, as today. `ComputePlane` routes the generated `HelmRelease` onto the tenant's ComputePlane by injecting `spec.kubeConfig.secretRef` → `computeplane-admin-kubeconfig` (the module's fixed secret) and `spec.install.createNamespace: true`. Because the module is a single per-tenant enabler, `ComputePlane` resolves unambiguously to *that* tenant's cluster — no cluster name to thread through the app.
167+
`ApplicationDefinition` gains `placement` = `ManagementPlane` (default) | `ComputePlane`. `ManagementPlane` deploys into the tenant namespace on the management cluster, as today. `ComputePlane` routes the generated `HelmRelease` onto the tenant's ComputePlane by injecting `spec.kubeConfig.secretRef` → `computeplane-cluster-admin-kubeconfig` (the module's fixed secret) and `spec.install.createNamespace: true`. Because the module is a single per-tenant enabler, `ComputePlane` resolves unambiguously to *that* tenant's cluster — no cluster name to thread through the app.
169168

170169
*(Optional, advanced — deferred.)* The same routing generalizes to `placement: <named cluster>` for a tenant that deliberately runs its own `kind: Kubernetes` and wants apps placed there; that path reuses the identical `kubeConfig` injection with the cluster's own `<name>-admin-kubeconfig`. It is not the default delivery surface and is out of scope for v1.
171170

172171
### 5. Connectivity, remote apply, and inbound access
173172

174-
- **Remote apply (into the cluster).** For a `placement: ComputePlane` app, `cozystack-api` converts it to a `HelmRelease` on the management cluster carrying `spec.kubeConfig.secretRef` → `computeplane-admin-kubeconfig`; Flux applies the chart **into the ComputePlane**, never into the tenant namespace on management. This is exactly what the first revision implemented.
173+
- **Remote apply (into the cluster).** For a `placement: ComputePlane` app, `cozystack-api` converts it to a `HelmRelease` on the management cluster carrying `spec.kubeConfig.secretRef` → `computeplane-cluster-admin-kubeconfig`; Flux applies the chart **into the ComputePlane**, never into the tenant namespace on management. This is exactly what the first revision implemented.
175174
- **Connectivity to tenant services (`sandbox` data-plane contract).** A notebook/LLM/n8n flow needs the tenant's data ("my Jupyter → my managed Postgres"), which runs in the tenant namespace on the management cluster. The guarantee is "no kube-API access / no creds to escalate," **not** "no packets ever." No mesh is required: the ComputePlane's KubeVirt-VM node pods sit on the management Cilium pod network, so reachability is a **scoped per-service `CiliumNetworkPolicy`** — allow → the tenant's Postgres Service, deny → kube-apiserver (same shape as the existing `policy.cozystack.io/allow-to-apiserver` label). Per-service egress is narrower-by-construction than a node mesh — which matters because the consumer is untrusted code.
176175
- **Inbound access.** Workloads expose themselves via Ingress/Gateway on the ComputePlane; the management ingress proxies to the cluster's `exposeMethod: Proxied` NodePort (or a kubevirt-ccm `Service type: LoadBalancer`). Inbound data path only — no reverse kube-API path, and the tenant never receives cluster credentials.
177176

@@ -228,7 +227,7 @@ Tenant *visibility* is a separable UX/operability default, not a security mechan
228227

229228
## Failure and edge cases
230229

231-
- **ComputePlane not ready when a `placement: ComputePlane` app is created** → the `HelmRelease` waits on the `computeplane-admin-kubeconfig` Secret; Flux surfaces not-ready, as with any dependency ordering.
230+
- **ComputePlane not ready when a `placement: ComputePlane` app is created** → the `HelmRelease` waits on the `computeplane-cluster-admin-kubeconfig` Secret; Flux surfaces not-ready, as with any dependency ordering.
232231
- **kubeconfig Secret missing/rotated** → remote apply fails closed (no fallback to local apply); the security-correct behavior.
233232
- **`placement: ComputePlane` app but the module is disabled on the tenant** → reject at admission. Never climb to an ancestor's ComputePlane (Design §7).
234233
- **GPU exhaustion** → cluster-autoscaler adds GPU nodes up to the configured `maxReplicas`; beyond that the workload pends.

0 commit comments

Comments
 (0)