Add K8s support with SA Trust architecture#12434
Conversation
New auth flow for k8s SA-trust Can run single Gateway node with autoscaled number of securely connected worker nodes Helm chart for gateway + worker deployment
| import { readFileSync, existsSync } from "node:fs"; | ||
|
|
||
| /** |
There was a problem hiding this comment.
CA cert ignored
validateToken() reads the cluster CA (const _caCert = ...) but never uses it, while fetch() will still enforce TLS validation. In-cluster, this will fail against clusters where the API server cert isn’t trusted by Node’s default CA store unless the process is configured with NODE_EXTRA_CA_CERTS (not enforced anywhere in this PR). This makes k8s-sa-trust non-functional by default in many Kubernetes environments.
Consider wiring the CA into the HTTP client (e.g., Node fetch dispatcher/agent) or explicitly documenting/enforcing the required env/config so the gateway can reliably call TokenReview.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/gateway/k8s-auth.ts
Line: 1:3
Comment:
**CA cert ignored**
`validateToken()` reads the cluster CA (`const _caCert = ...`) but never uses it, while `fetch()` will still enforce TLS validation. In-cluster, this will fail against clusters where the API server cert isn’t trusted by Node’s default CA store unless the process is configured with `NODE_EXTRA_CA_CERTS` (not enforced anywhere in this PR). This makes `k8s-sa-trust` non-functional by default in many Kubernetes environments.
Consider wiring the CA into the HTTP client (e.g., Node fetch dispatcher/agent) or explicitly documenting/enforcing the required env/config so the gateway can reliably call TokenReview.
How can I resolve this? If you propose a fix, please make it concise.| isIdentityAllowed(namespace: string, serviceAccount: string, requestedRole: string): boolean { | ||
| const identities = this.config.allowedIdentities; | ||
| if (!identities || identities.length === 0) { |
There was a problem hiding this comment.
Allowlist defaults to allow-all
isIdentityAllowed() returns true when allowedIdentities is missing/empty. That means enabling gateway.auth.k8sTrust.enabled without an allowlist will accept any authenticated ServiceAccount in the cluster (including from other namespaces), and skipPairing will then be granted. If this is meant to be secure-by-default, this should deny-by-default (or require an explicit wildcard config).
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/gateway/k8s-auth.ts
Line: 150:152
Comment:
**Allowlist defaults to allow-all**
`isIdentityAllowed()` returns `true` when `allowedIdentities` is missing/empty. That means enabling `gateway.auth.k8sTrust.enabled` without an allowlist will accept *any* authenticated ServiceAccount in the cluster (including from other namespaces), and `skipPairing` will then be granted. If this is meant to be secure-by-default, this should deny-by-default (or require an explicit wildcard config).
How can I resolve this? If you propose a fix, please make it concise.| authMethod = "device-token"; | ||
| } | ||
| } |
There was a problem hiding this comment.
K8s auth enables pairing bypass
skipPairing is set whenever authMethod === "k8s-sa-trust", regardless of k8sTrust.ephemeralSessions (which is documented as controlling persistence/pairing behavior). As written, enabling k8s-sa-trust always bypasses pairing even if an operator wanted TokenReview as an additional auth factor but still require device pairing. Either honor ephemeralSessions here or rename/remove the config knob to avoid a config that looks like it’s enforced but isn’t.
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/gateway/server/ws-connection/message-handler.ts
Line: 670:672
Comment:
**K8s auth enables pairing bypass**
`skipPairing` is set whenever `authMethod === "k8s-sa-trust"`, regardless of `k8sTrust.ephemeralSessions` (which is documented as controlling persistence/pairing behavior). As written, enabling `k8s-sa-trust` always bypasses pairing even if an operator wanted TokenReview as an *additional* auth factor but still require device pairing. Either honor `ephemeralSessions` here or rename/remove the config knob to avoid a config that looks like it’s enforced but isn’t.
How can I resolve this? If you propose a fix, please make it concise.|
Got the points, will get back with fixes |
bfc1ccb to
f92900f
Compare
|
This pull request has been automatically marked as stale due to inactivity. |
|
Thanks for the work on this. I’m closing this in favor of #34492 for the upstream Kubernetes install/docs path. There is some overlap here in the Kubernetes docs and deployment artifacts, but this PR is really centered on SA-trust auth and broader gateway/worker architecture changes. We'll keep the initial Kubernetes path simpler and avoid coupling it to new auth and scaling behavior. The SA-trust work here is valuable, but I think it would be better proposed separately as a new focused PR on top of the simpler Kubernetes baseline. Thanks again for the work. |
Summary
Adds a Kubernetes Service Account Trust authentication method that enables workers to authenticate with the gateway using their pod SA token. This allows zero-touch scaling of worker nodes in Kubernetes — new pods authenticate automatically via the K8s TokenReview API, bypassing interactive device pairing.
Use Cases
kubectl scalespawns workers that authenticate instantly.Behavior Changes
k8s-sa-trustadded to gateway authentication.--k8s-trustflag read their pod SA token and present it on connect.auth.k8sTrustis configured.skipPairing).Existing Functionality Check
docs/install/docker.md— Docker guide exists but no K8s equivalent.Changes
Source code (10 files):
src/gateway/k8s-auth.tsK8sAuthenticatorclass — validates SA tokens via TokenReview API, namespace/SA allowlistingsrc/gateway/k8s-auth.test.tssrc/config/types.gateway.tsk8sTrusttype definition onGatewayAuthConfigsrc/config/zod-schema.tsk8sTrustconfig validationsrc/gateway/auth.ts"k8s-sa-trust"added to auth method union typesrc/gateway/protocol/schema/frames.tsk8sTokenfield inConnectParamssrc/gateway/client.tsk8sServiceAccountTokenoption in auth payloadsrc/gateway/server/ws-connection/message-handler.tsskipPairingfork8s-sa-trustsrc/node-host/runner.tsk8sTrustoption, reads SA token from pod volume mountsrc/cli/node-cli/register.ts--k8s-trustCLI flagDeployment files (22 files):
deploy/k8s/deploy/helm/openclaw/Documentation (1 file + Mintlify nav):
docs/gateway/kubernetes.mddocs/docs.jsongateway/kubernetesto navigation +/kubernetesand/k8sredirectsTests
src/gateway/k8s-auth.test.ts— all passing.pnpm test— full suite passes.Manual Testing
Prerequisites
kubectlaccess.Steps
--k8s-trustflag.kubectl scale deployment openclaw-worker --replicas=5.Evidence
oc-test-1andoc-test-2namespaces viahelm install, both operational.Sign-Off
AI Disclosure
This PR was prepared with assistance from Claude Opus 4. The architecture design, implementation, K8s deployment testing, Helm chart creation, and debugging were developed collaboratively. I understand the code and the changes. Tested via automated tests (855/856 pass) and manual K3s deployment (3 namespaces, dynamic scaling 2→5 workers).
lobster-biscuit
Pre-PR Checklist
pnpm build— passespnpm test— 855/856 pass (1 pre-existing EACCES failure unrelated to this PR)pnpm check— passes (tsgo + lint + format).nozim/to final locationsdocs/docs.jsonnavigation + redirectsmainGreptile Overview
Greptile Summary
This PR adds an opt-in Kubernetes ServiceAccount Trust auth path for gateway↔worker connections. Workers can send their projected pod ServiceAccount JWT (
k8sToken) during the connect handshake, and the gateway validates it via the Kubernetes TokenReview API whengateway.auth.k8sTrustis enabled. On successful validation/allowlisting, the connection is marked ask8s-sa-trustand device pairing is skipped. The PR also adds supporting CLI/runtime flags for node-host, config/zod schema updates, and Kubernetes deployment artifacts (raw manifests + Helm chart) plus documentation updates.Confidence Score: 3/5
(2/5) Greptile learns from your feedback when you react with thumbs up/down!