|
| 1 | +--- |
| 2 | +title: "OIDC authentication for kubectl" |
| 3 | +linkTitle: "OIDC authentication" |
| 4 | +description: "Give tenant users per-identity kubectl access to a Managed Kubernetes cluster with per-cluster audience isolation." |
| 5 | +weight: 45 |
| 6 | +--- |
| 7 | + |
| 8 | +Tenant Kubernetes clusters can authenticate `kubectl` users through OIDC instead of the shared static admin kubeconfig. Each user then has their own identity, per-user audit, and RBAC that can be revoked by disabling the account — not by rotating a shared certificate. |
| 9 | + |
| 10 | +The identity model is deliberately **per-cluster** rather than per-tenant: each tenant Kubernetes cluster gets its own OIDC audience, and a token minted for cluster A is rejected by cluster B's apiserver. That gives you cross-cluster isolation without provisioning a Keycloak realm per tenant. The full rationale (why per-cluster audience and not per-tenant realm; how it relates to Keycloak Organizations; what BYO-OIDC looks like) is in the [design proposal](https://github.com/cozystack/community/pull/24). |
| 11 | + |
| 12 | +{{% alert color="info" %}} |
| 13 | +The static `kubernetes-<cluster>-admin-kubeconfig` Secret in the tenant namespace stays available as a break-glass path regardless of whether OIDC is enabled. |
| 14 | +{{% /alert %}} |
| 15 | + |
| 16 | +## Modes |
| 17 | + |
| 18 | +`spec.oidc.mode` picks the identity source: |
| 19 | + |
| 20 | +- **`None`** — the default. No OIDC; only the static admin kubeconfig works. Existing clusters render identically to before. |
| 21 | +- **`System`** — trust the platform `cozy` Keycloak realm via a per-cluster public client and audience binding. Users are the ones a Cozystack platform admin already provisioned in `cozy`; the tenant does not manage a directory of its own. |
| 22 | +- **`CustomConfig`** — trust a tenant-supplied issuer directly (BYO IdP: Okta, Auth0, a customer's own Keycloak). `cozy` is not in the path. |
| 23 | + |
| 24 | +## Enable OIDC — `System` mode |
| 25 | + |
| 26 | +```yaml |
| 27 | +apiVersion: apps.cozystack.io/v1alpha1 |
| 28 | +kind: Kubernetes |
| 29 | +metadata: |
| 30 | + name: prod |
| 31 | + namespace: tenant-acme |
| 32 | +spec: |
| 33 | + oidc: |
| 34 | + mode: System |
| 35 | + users: |
| 36 | + |
| 37 | + role: admin # binds to ClusterRole/cluster-admin |
| 38 | + |
| 39 | + role: view # binds to ClusterRole/view |
| 40 | + # ... |
| 41 | +``` |
| 42 | + |
| 43 | +Cozystack provisions: |
| 44 | + |
| 45 | +- A per-cluster `KeycloakClient` in the `cozy` realm, with `clientId` set to `<namespace>-kubernetes-<cluster-name>` (for the CR above: `tenant-acme-kubernetes-prod`). `public: true`, PKCE required, redirect URIs locked to `localhost:8000` and `localhost:18000` (the `kubectl oidc-login` defaults). |
| 46 | +- A per-cluster `KeycloakClientScope` whose audience mapper pins the token's `aud` claim to that same `clientId`. |
| 47 | +- A structured `AuthenticationConfiguration` (`apiserver.config.k8s.io/v1beta1`) on the tenant kube-apiserver, pointing at the `cozy` issuer and the per-cluster audience. |
| 48 | +- One `ClusterRoleBinding` inside the tenant cluster for each `users[]` entry — `admin` → `cluster-admin`, `view` → `view`. The chart uses your `users[].email` value as the `User:` subject and matches it against the token's `email` claim. |
| 49 | + |
| 50 | +Removing a user from `users[]` prunes their `ClusterRoleBinding` on the next reconcile. |
| 51 | + |
| 52 | +### Prerequisite |
| 53 | + |
| 54 | +`System` mode requires the platform-level OIDC feature (`authentication.oidc.enabled` at the Cozystack platform values). If the flag is off, the chart hard-fails the render with a clear message. Ask a Cozystack platform admin to enable it, or use `CustomConfig`. |
| 55 | + |
| 56 | +## Enable OIDC — `CustomConfig` mode |
| 57 | + |
| 58 | +Bring your own issuer. Two supply paths, **mutually exclusive**: |
| 59 | + |
| 60 | +```yaml |
| 61 | +spec: |
| 62 | + oidc: |
| 63 | + mode: CustomConfig |
| 64 | + customConfig: |
| 65 | + config: | |
| 66 | + apiVersion: apiserver.config.k8s.io/v1beta1 |
| 67 | + kind: AuthenticationConfiguration |
| 68 | + jwt: |
| 69 | + - issuer: |
| 70 | + url: https://idp.acme.example |
| 71 | + certificateAuthority: | |
| 72 | + -----BEGIN CERTIFICATE----- |
| 73 | + ... |
| 74 | + -----END CERTIFICATE----- |
| 75 | + audiences: |
| 76 | + - cozystack-prod |
| 77 | + claimMappings: |
| 78 | + username: |
| 79 | + claim: email |
| 80 | + prefix: "" |
| 81 | + groups: |
| 82 | + claim: groups |
| 83 | + prefix: "" |
| 84 | + users: |
| 85 | + |
| 86 | + role: admin |
| 87 | +``` |
| 88 | +
|
| 89 | +…or via a pre-existing Secret in the tenant namespace (you create it separately, e.g. under your own GitOps repository, so the AuthenticationConfiguration does not live inside the `Kubernetes` CR): |
| 90 | + |
| 91 | +```yaml |
| 92 | +spec: |
| 93 | + oidc: |
| 94 | + mode: CustomConfig |
| 95 | + customConfig: |
| 96 | + secretRef: |
| 97 | + name: acme-byo-authn-config # Secret with a `config.yaml` key holding the AuthenticationConfiguration |
| 98 | +``` |
| 99 | + |
| 100 | +Setting both `config` and `secretRef.name` (or neither) fails the render. In `CustomConfig` mode no Keycloak objects are provisioned in `cozy`; the tenant apiserver trusts the tenant-supplied issuer directly. |
| 101 | + |
| 102 | +Ensure your BYO issuer emits the `email` claim in the JWT. Every conformant OIDC provider does when the client requests the `email` scope. If you distribute a hand-crafted kubeconfig instead of using the chart-generated one, remember to include `--oidc-extra-scope=email` in the `kubectl oidc-login` exec block. |
| 103 | + |
| 104 | +## Get the kubeconfig |
| 105 | + |
| 106 | +In `System` mode, Cozystack writes a ready-to-use kubeconfig into a `kubernetes-<cluster>-oidc-kubeconfig` Secret in the tenant namespace (the same namespace where the `Kubernetes` resource lives). It's exposed to the dashboard alongside the admin kubeconfig, and you can also fetch it directly: |
| 107 | + |
| 108 | +```shell |
| 109 | +kubectl --namespace tenant-acme get secret kubernetes-prod-oidc-kubeconfig \ |
| 110 | + --output=jsonpath='{.data.kubeconfig}' | base64 -d > prod.kubeconfig |
| 111 | +``` |
| 112 | + |
| 113 | +The file contains the tenant CA (extracted from the Kamaji-issued admin kubeconfig at reconcile time), the external apiserver URL, and a `kubectl oidc-login` exec block wired to your per-cluster client. |
| 114 | + |
| 115 | +In `CustomConfig` mode no kubeconfig Secret is generated — you distribute the OIDC kubeconfig out-of-band from your own IdP configuration. |
| 116 | + |
| 117 | +## Sign in |
| 118 | + |
| 119 | +Install the `oidc-login` kubectl plugin once: |
| 120 | + |
| 121 | +```shell |
| 122 | +kubectl krew install oidc-login |
| 123 | +``` |
| 124 | + |
| 125 | +Then use the OIDC kubeconfig — the first request triggers the browser flow: |
| 126 | + |
| 127 | +```shell |
| 128 | +kubectl --kubeconfig prod.kubeconfig get pods --all-namespaces |
| 129 | +``` |
| 130 | + |
| 131 | +`kubectl oidc-login` opens Keycloak's login page on `localhost:8000` (falling back to `localhost:18000`), captures the token, and caches it locally. Subsequent calls are silent until the token expires. |
| 132 | + |
| 133 | +## Toggling OIDC off |
| 134 | + |
| 135 | +Setting `spec.oidc.mode` back to `None` — or deleting the `Kubernetes` resource entirely — reconciles a cleanup pass that removes the tenant apiserver's `--authentication-config` flag, deletes the chart-owned OIDC Secrets, and drops the per-cluster Keycloak client and audience scope. `ClusterRoleBindings` labelled by the release are also removed from the tenant cluster (best-effort during pre-delete, since the tenant apiserver may already be tearing down). |
| 136 | + |
| 137 | +## Prerequisites and gotchas |
| 138 | + |
| 139 | +- **Don't mix with legacy `--oidc-*` flags.** The tenant kube-apiserver refuses to boot if both `--authentication-config` (injected by `spec.oidc`) and any legacy `--oidc-*` flag are set. If you previously wired OIDC by hand through `controlPlane.apiServer.extraArgs`, remove those flags before enabling `spec.oidc`. The chart fails the render with a pointer to this migration. |
| 140 | +- **`oidc-login` plugin required.** Without `kubectl krew install oidc-login` the exec block errors out client-side. The plugin is a documented prerequisite. |
| 141 | +- **`emailVerified: true` when provisioning Keycloak users.** Phase 1 does not add a `claimValidationRules` entry to the rendered `AuthenticationConfiguration` — so `email_verified` is not chart-enforced. Set `emailVerified: true` on the `KeycloakRealmUser` (or complete the email-verify flow through the Keycloak UI) so the identity holding a given `users[].email` is guaranteed authentic. The `cozy` realm's default `duplicateEmails: false` prevents a second account from claiming an already-registered address. If the issuer explicitly emits `email_verified: false` on a token the apiserver rejects it (k8s upstream behaviour); a missing claim is treated as verified. CEL `claimValidationRules` to make this a hard gate is a follow-up hardening path. |
| 142 | +- **Custom issuer with a self-signed CA.** In `CustomConfig` mode you can supply the CA inline under `issuer.certificateAuthority`. The legacy `--oidc-*` flag path could not. |
| 143 | + |
| 144 | +## What's out of scope for this feature |
| 145 | + |
| 146 | +- **Per-tenant Keycloak realms.** Managed multi-tenant identity (a hosted directory the tenant self-administers) is a separate proposal, evaluated against Keycloak Organizations. Track it in the [community proposal](https://github.com/cozystack/community/pull/24). |
| 147 | +- **Federating an external IdP into the platform `cozy` realm.** BYO-for-Cozystack-itself is a distinct problem — this feature is BYO-for-a-managed-service. |
| 148 | +- **Cross-cluster SSO inside one tenant.** By design: each cluster has its own audience, which is the per-cluster isolation primitive. |
| 149 | +- **RFC 8693 token exchange.** Possible future optimisation; not required for the per-cluster client + audience model. |
0 commit comments