feat(sdk): add OAuth client credentials authentication across SDKs #2803

User Story

As an operator running CI jobs or long-lived service automation, I want every official OpenShell SDK to acquire, attach, and renew gateway access tokens with the OAuth 2.0 client credentials grant, so that I can use Python, TypeScript, or Go without interactive login, prerequisite CLI invocations, client reconstruction, or custom token-refresh code.

Problem Statement

Client-credentials authentication is inconsistent across the SDKs. Python can attach a callable bearer token, but its built-in active-gateway refresher only supports refresh-token grants. TypeScript accepts only a static OIDC token. Go can perform a one-shot client-credentials exchange and separately refresh arbitrary caller-provided token sources, but it does not connect those capabilities into a renewable client authentication workflow.

Because client-credentials grants normally return no refresh token, all three SDKs lack a complete first-class path that repeats the grant before expiry and keeps the main SDK client authenticated for its lifetime.

Impact / Why This Matters

Today, service automation must shell out to the CLI, manually acquire tokens and recreate clients, or independently implement discovery, exchange, caching, concurrency, renewal, and bearer attachment. The workaround differs by language and duplicates security-sensitive behavior such as issuer validation, redirect refusal, TLS requirements, audience/scope handling, and secret redaction.

This creates portability gaps between official SDKs and makes long-running jobs fail after the initial access token expires even though their service-account credentials remain valid. It also increases the chance that individual applications will leak a client secret, request the wrong claims, stampede an identity provider, or follow a redirect that replays credential-bearing request data.

Technical Context

The gateway is already grant-agnostic: it validates bearer JWT signature, issuer, audience, expiry, roles, and scopes, then applies authorization. No gateway or protobuf change is required. The missing capability belongs at each SDK's auth-provider boundary.

The CLI establishes the canonical exchange semantics: validated OIDC discovery, no redirects, client_secret_post, no implicit interactive scopes, configured scopes, optional audience, and access-token expiry. Gateway metadata already stores issuer, client ID, audience, and scopes. The existing Keycloak E2E proves that a client-credentials service-account subject can access the gateway after receiving the required authorization.

Proposed Design

Expose an idiomatic, explicit client-credentials authentication API in the Python, TypeScript, and Go SDKs and integrate it with each SDK's main client transport. Each implementation should acquire and cache an access token in memory, repeat the client-credentials grant before expiry, coalesce concurrent exchanges, and attach the current bearer without requiring client reconstruction.

All SDKs should accept explicit issuer, client ID, client secret, scopes, and optional audience. SDKs that already consume registered gateway state should also resolve issuer, client ID, audience, and scopes from metadata.json; whether TypeScript gains registered-gateway resolution or remains explicit-only is a design decision. Preserve all existing static-token APIs for backward compatibility.

Keep the client secret only in memory or behind a caller-supplied secret provider. Never write it to token caches, logs, exceptions, or object representations. Pin shared security and expiry semantics with a cross-language conformance matrix or fixtures while allowing language-idiomatic API names and types.

Acceptance Criteria

• Python, TypeScript, and Go each expose a documented public client-credentials authentication API usable by their main SDK client.
• Every SDK supports explicit issuer, client ID, client secret, scopes, and optional audience; registered-gateway construction resolves those fields wherever that SDK supports gateway metadata.
• Each SDK acquires a service-account access token and attaches it as authorization: Bearer <token> to all supported RPC call shapes.
• Long-running clients repeat the client-credentials grant before token expiry without requiring client reconstruction.
• Concurrent requests share one in-flight acquisition or renewal, and cancellation by one waiter does not corrupt shared token state.
• Client-credentials requests do not implicitly add interactive scopes such as openid, profile, or email.
• OIDC discovery validates the configured issuer, and token requests do not follow redirects that could replay the client secret.
• Remote token endpoints require protected transport by default; explicit development exceptions do not silently weaken production behavior.
• Client secrets are never persisted or exposed in errors, logs, request diagnostics, or object representations.
• Invalid credentials, missing configuration, malformed responses, missing or invalid expiry data, and transport failures produce actionable SDK errors without leaking sensitive request data.
• Existing static-token and caller-provided auth-provider APIs remain backward compatible.
• Shared conformance coverage pins request fields, audience/scope behavior, expiry/renewal, concurrency, redirect/TLS posture, and secret handling across all three SDKs.
• SDK-specific unit and integration tests exercise acquisition, automatic renewal, bearer attachment, and failure behavior.
• Published Python, TypeScript, and Go documentation describes the service-account workflow and gateway authorization prerequisites.

Affected Components

Technical Investigation

Architecture Overview

Each SDK already has a usable transport/auth boundary but exposes a different subset of the required lifecycle:

• Python: the gRPC interceptor calls a bearer provider per RPC. The active-gateway provider caches and renews authorization-code tokens with a refresh token, but it cannot repeat a client-credentials grant.
• TypeScript: the Connect interceptor closes over a static oidcToken. Both root and sandbox clients build that transport directly, and the README instructs users to recreate the client after refreshing a token.
• Go: the OIDC package performs a secure one-shot client-credentials exchange. The core SDK accepts per-RPC credentials and has a generic, single-flight RefreshableToken, but ClientCredentials is not a reusable oauth2.TokenSource and is not directly composed into a client. The registered-gateway path rereads a token file rather than re-exchanging client credentials.

The TypeScript public API source explicitly defers an OidcRefresher until a cross-language Python/Go/TypeScript conformance suite exists, providing a natural alignment point for this issue.

Code References

Current Behavior

Python

Callers can pass an already-issued token or write a custom callback. from_active_cluster() reads oidc_token.json; a CLI-created client-credentials access token works until its expiry grace window. Renewal then fails because _OidcRefresher._refresh() requires a refresh token. auto_refresh=False changes this to a fail-closed expiry error but does not reacquire a token.

TypeScript

Callers pass oidcToken to OpenShellClient.connect() or SandboxClient.connect(). The interceptor reuses that string for the client's lifetime. There is no native discovery, exchange, token-provider abstraction, renewal, or registered-gateway/XDG loader. A caller must fetch a new token and construct a new SDK client.

Go

oidc.ClientCredentials() supports explicit or gateway-derived issuer/client ID, validates discovery, sends client_secret_post, applies explicit scopes, refuses redirects, limits response size, and redacts secrets. It returns one token but does not attach or renew it. v1.RefreshableToken() can renew a caller-authored oauth2.TokenSource, but the SDK does not expose a client-credentials token source that composes the two. The gateway-aware client reads cached tokens from disk; it cannot repeat the grant and its metadata model omits audience/scopes.

The Go WithGateway documentation also says tokens are persisted, while ClientCredentials() does not persist them; the revised behavior and documentation should resolve that mismatch.

What Would Need to Change

• Python: add a public client-credentials token provider and integrate it with the existing callable/interceptor boundary, active-gateway selection, close lifecycle, and high-level Sandbox wrapper where applicable.
• TypeScript: add public client-credentials configuration and an async token provider to the Connect interceptor, with discovery, exchange, expiry cache, single-flight renewal, typed failures, and preserved edge/static-token transport guards.
• Go: expose a reusable client-credentials token source or auth-provider composition, connect it cleanly to NewClient, and extend registered-gateway metadata parsing for audience/scopes when using gateway-derived configuration.
• Cross-SDK: define shared fixtures or a conformance matrix for request fields, discovery/redirect/TLS behavior, expiry, renewal, concurrency, cancellation, and secret redaction while retaining language-idiomatic APIs.
• Docs/tests: update each SDK's public docs and add unit plus integration/E2E coverage for the complete acquire-attach-renew lifecycle.

No server, protobuf, gateway TOML, Helm, deployment, compute-driver, sandbox-infrastructure, or LSM-sensitive change is required. Python already depends on httpx; TypeScript can use Node 20's HTTP/fetch primitives or deliberately add a dependency; Go already has the required OAuth/HTTP dependencies.

Alternative Approaches Considered

1. Document each SDK's existing escape hatches. Python callbacks and Go token-source composition can be made to work, but TypeScript remains static and every application must reimplement security-sensitive behavior.
2. Acquire once in every SDK and require client reconstruction. This gives API symmetry but does not satisfy long-running automation and preserves expiry failures.
3. Teach disk-backed refreshers to infer client credentials from a missing refresh token. Absence of a refresh token does not identify the original grant, and the secret is intentionally absent from token bundles.
4. Silently detect OPENSHELL_OIDC_CLIENT_SECRET. This matches CLI convenience but makes outbound token exchanges implicit. Explicit opt-in with an optional environment-backed provider is safer.
5. Persist the client secret with access-token state. Rejected because it expands durable secret exposure and conflicts with the current bundle contract.
6. Require one byte-identical public API across languages. Rejected in favor of a shared behavioral/security contract with idiomatic language surfaces.
7. Shell out to openshell gateway login. This adds process coupling and cannot provide a robust native per-client renewal lifecycle.

Patterns to Follow

• Preserve each SDK's existing auth-provider/interceptor boundary rather than adding OAuth logic to resource methods.
• Match the CLI's client-credentials request semantics: no implicit interactive scopes, optional configured audience, and client_secret_post unless maintainers deliberately support additional token-endpoint auth methods.
• Reuse Python's expiry grace, synchronization, issuer validation, redirect refusal, and deterministic cleanup patterns.
• Reuse TypeScript's token exclusivity, cookie validation, plaintext-remote refusal, SdkError, and shared root/scoped transport.
• Reuse Go's secure one-shot exchange, response bound, typed/sentinel errors, generic singleflight refresh/backoff layer, and AuthProvider attachment.
• Keep secret material out of shared token caches. Preserve existing static-token configuration in every SDK.

Scope Assessment

• Complexity: High — cross-language public APIs, security semantics, tests, docs, and conformance behavior
• Confidence: High — all transport hooks, canonical exchange semantics, and a real Keycloak fixture already exist
• Estimated files to change: 18–25, depending on registered-gateway support and shared fixture design
• Issue type: feat

Risks & Open Questions

• What idiomatic public API should each language expose while preserving one behavioral contract?
• Must TypeScript gain registered-gateway/XDG resolution, or is explicit issuer/client configuration sufficient for parity?
• Should client secrets be accepted only as strings, or also through callbacks/providers for runtime secret rotation? Environment lookup should require explicit opt-in.
• Is client_secret_post sufficient for CLI parity, or should SDKs support client_secret_basic for providers that require it?
• Should renewed access tokens remain memory-only or optionally write back to the shared CLI cache? Client secrets must never be persisted.
• How should missing or invalid expires_in behave so a token is not treated as permanently fresh?
• Should one UNAUTHENTICATED RPC trigger a forced re-exchange in addition to proactive expiry renewal?
• Should a renewal failure fail closed or temporarily return a cached token? Go's generic refresher currently allows stale-token fallback, while Python fails closed.
• How should cancellation interact with a shared in-flight exchange, especially when one waiter cancels but others remain active?
• Should token-endpoint transport require HTTPS except explicit loopback development, independently of the gateway transport settings?
• Is shared access-token cache interoperability in scope? Go's disk source expects RFC3339 expiry, while the canonical Rust bundle uses Unix expires_at; fixing that existing mismatch could expand the issue.
• Service-account JWTs still need configured roles/scopes and workspace membership. SDK authentication cannot manufacture gateway authorization.

Disposition Readiness

• State: state:validated
• Assessment: The per-SDK limitations are directly demonstrated in source and tests. Canonical CLI behavior, the Go one-shot implementation, all three transport hooks, and the Keycloak E2E establish feasibility. Remaining questions are public API and cross-SDK consistency decisions for accepted work, not missing evidence.
• Missing evidence: None

Test Considerations

• Add language-neutral fixtures or an explicit conformance matrix covering discovery documents, exact token request fields, scopes/audience, token responses, expiry/leeway, redirects, TLS policy, malformed responses, and errors that attempt to echo the secret.
• Python: extend current bearer/refresher unit coverage and replace or supplement the manual Keycloak exchange with the public SDK API.
• TypeScript: add auth-provider tests for acquisition, per-request bearer changes, single-flight renewal, cancellation, expiry, typed failures, and preservation of edge/static-token transport guards.
• Go: compose client credentials with the auth provider in tests, cover repeated exchange and audience/gateway metadata, and resolve the stale-token and persistence-documentation semantics.
• Add SDK-specific integration coverage against the existing Keycloak/gateway fixture, or a shared harness that proves acquire, attach, authorize, expire/renew, and call again for all three languages.
• Eventual implementation verification should include mise run test:python, mise run sdk:ts:ci, mise run go:ci, mise run e2e:oidc-python:docker, mise run pre-commit, and mise run ci, plus any new TypeScript/Go OIDC E2E task introduced by the work.

Documentation Impact

Update the published gateway-auth/service-account documentation plus each SDK's own usage surface: Python SDK examples under docs/, sdk/typescript/README.md, sdk/go/README.md, and the generated Go OIDC/auth docs. architecture/gateway.md already describes client-credentials login; update it only if the SDK/cache ownership boundary becomes a stable architectural invariant.

No gateway TOML fields or defaults change, so docs/reference/gateway-config.mdx does not require an update. Helm, compute-driver, deployment, and LSM documentation are unaffected.

Updated by spike investigation. state:validated means the issue is ready for human disposition; state:needs-info means specific evidence is still required. A human applies state:accepted or places the issue on the roadmap if OpenShell should pursue the work. To queue unattended agent planning, a human applies agent:plan-requested; a direct request to an agent can instead invoke build-from-issue.