You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit d870b39
Browse filesBrowse the repository at this point in the historyBrowse files
fix(kubernetes): render the aggregation-layer UID header for tenant OIDC
A tenant kube-apiserver publishes requestheader-uid-headers into the
extension-apiserver-authentication ConfigMap only when the flag is set,
and the chart never set it. Without that key the request-header
authenticator on an aggregated apiserver does not know which header
carries the UID and ignores it, so a caller who has a UID is served
without one. Operators had to add the flag by hand through the
controlPlane.apiServer.extraArgs escape hatch.
This forwards a UID the caller already has and does not create one.
ServiceAccount tokens carry a UID and gain from it. OIDC users
authenticated through spec.oidc.mode: System do not: the
AuthenticationConfiguration the chart generates maps username and
groups only, and with no claimMappings.uid upstream returns an empty
UID (plugin/pkg/authenticator/token/oidc/oidc.go, getUID, read at
v0.35.0), so there is nothing to carry. Mapping a claim would give
those users a UID and change the identity of users who have none
today, which is a separate decision and is tracked separately.
The half read rather than run: that an existing UID is lost without the
published key comes from the controller writing the ConfigMap and the
request-header authenticator consuming it, not from a live cluster. That
OIDC users have no UID is unambiguous in the source named above.
Render --requestheader-uid-headers=X-Remote-Uid whenever OIDC is on,
plus --feature-gates=RemoteRequestHeaderUID=true on v1.32, where the
gate is still alpha and the apiserver rejects the flag without it.
v1.31 gets nothing: the flag does not exist there and an unknown flag
is fatal to the apiserver.
The deciding version is the effective one, not the image tag.
--emulated-version in the same passthrough moves it down, and the gate
resolves at the emulated version through featureSpecAtEmulationVersion
(featureSpecAtEmulationAndMinCompatVersion by v1.35), so an emulated
1.32 needs the gate line even on a v1.35 cluster, and below 1.32 the
gate is PreAlpha and cannot be enabled at all, which leaves nothing
safe to render.
Both flags are new, so every extraArgs shape reaching this code was
rendering before it existed, and turning one of those into a render
error would strand a running tenant on a HelmRelease that stops
converging while its control plane carries on. The chart therefore
yields to an operator entry that already satisfies the requirement,
renders nothing at all when the operator turned RemoteRequestHeaderUID
off, and otherwise merges its gate term into a copy of the operator's
--feature-gates value. Carrying that value into the merged entry
decides the gate whether the tenant control plane collapses duplicate
flags into the last one or hands both to the apiserver, which merges
them in the order it reads them.
Operator entries are read the way the apiserver reads them: header list
elements matched whole and untrimmed, folded as EqualFold folds them,
and refused when one is empty after trimming, as checkForWhiteSpaceOnly
refuses it; gate names compared exactly; gate values resolved as
strconv.ParseBool resolves them; blanket AllAlpha and AllBeta honoured;
an empty value treated as no entry at all. Each flag is judged by its
last entry carrying a value. The apiserver's CSV reader strips quotes
the chart does not parse, so quotes on that entry make the chart render
nothing, while quotes on an earlier entry the collapse drops do not.
The render fails only where a uid-headers list of the operator's
describes a control plane that already refuses to start: one that
omits X-Remote-Uid, one with an element that is empty after trimming,
and any such list while the gate is off. Withholding the chart's own
flag rescues none of them, because the flag the apiserver rejects is
the operator's.
Assisted-by: LLM
Signed-off-by: Aleksei Sviridkin <[email protected]>
Copy file name to clipboardExpand all lines: packages/apps/kubernetes/templates/cluster.yaml
+266Lines changed: 266 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -255,6 +255,51 @@ metadata:
255
255
{{- if and $oidcEnabled (has "authentication-config" $userVolumeNames) }}
256
256
{{- fail "controlPlane.apiServer.extraVolumes: volume name \"authentication-config\" is reserved by the chart for OIDC AuthenticationConfiguration when spec.oidc.mode is not None — rename your volume." }}
257
257
{{- end }}
258
+
{{- /*
259
+
Aggregation-layer UID plumbing. What is rendered per version, why an
260
+
opt-out renders nothing rather than failing, and the migration note live
261
+
in docs/oidc-tenant.md. That table is the one to update; there is
262
+
deliberately no copy of it here.
263
+
264
+
Anchors for the split, read at v1.32.13 and v1.35.6:
265
+
staging/src/k8s.io/apiserver/pkg/features/kube_features.go for the gate's
266
+
versioned spec, and .../server/options/authentication.go Validate() for
267
+
the refusals the guards below mirror. The version that decides is the
268
+
EFFECTIVE one, which --emulated-version moves down.
269
+
270
+
The merged gate entry carries the operator's own value, in their own
271
+
prefix spelling, because that is correct whether duplicate flags collapse
272
+
by name (it is last and wins) or are preserved (featureGate.Set() merges
273
+
it into the earlier one).
274
+
275
+
Two things do depend on the collapse, pinned by packages/system/kamaji,
276
+
whose Makefile takes the newest -edge tag on `make update`. Reading only
277
+
the LAST entry of each flag cuts both ways: handed every copy the apiserver
278
+
merges them, so an earlier explicit setting the chart never saw would reach
279
+
the gate. Failing on a uid-headers list without X-Remote-Uid is the other:
280
+
handed both copies pflag appends, and the union passes caseInsensitiveHas.
281
+
The opt-out path is unaffected, since the chart emits nothing there.
282
+
283
+
Ownership is resolved after the arg loop because the last entry of each
284
+
flag, and the interaction between them, are known only once every arg has
285
+
been seen.
286
+
*/}}
287
+
{{- $uidHeaderUnsupported := false }}
288
+
{{- $uidGateRequired := false }}
289
+
{{- $emulatedKubeVersion := "" }}
290
+
{{- $uidGateOptedOut := false }}
291
+
{{- $uidHeaderArgSeen := "" }}
292
+
{{- $uidHeaderQuoted := false }}
293
+
{{- $emulatedQuoted := false }}
294
+
{{- $uidHeaderConforms := false }}
295
+
{{- $uidHeaderHasEmpty := false }}
296
+
{{- $gateArgValue := "" }}
297
+
{{- $gateUsesKubePrefix := false }}
298
+
{{- $gateArgSeen := "" }}
299
+
{{- $uidGateSetting := "" }}
300
+
{{- $allBetaSetting := "" }}
301
+
{{- $allAlphaSetting := "" }}
302
+
{{- $uidArgs := list }}
258
303
{{- /*
259
304
Legacy `--oidc-*` flags in the user-supplied extraArgs are mutually
260
305
exclusive with `--authentication-config`. The kube-apiserver refuses
@@ -276,6 +321,224 @@ metadata:
276
321
{{- if hasPrefix "--authentication-config" $arg }}
277
322
{{- fail (printf "controlPlane.apiServer.extraArgs contains %q while spec.oidc.mode is %q — the chart already injects --authentication-config= for OIDC. Drop your explicit flag and let spec.oidc.mode: CustomConfig own the AuthenticationConfiguration (customConfig.config or customConfig.secretRef.name), or leave spec.oidc.mode: None and keep managing the flag yourself." $arg $.Values.oidc.mode) }}
278
323
{{- end }}
324
+
{{- if hasPrefix "--requestheader-uid-headers=" $arg }}
325
+
{{- /*
326
+
An empty value is not an opinion: readAsCSV returns an empty slice and
327
+
Validate() only inspects a non-empty list, so the chart reads the last
328
+
entry carrying one. Where the chart renders nothing for that flag the
329
+
operator's empty value survives the collapse and reaches the apiserver as
330
+
a BARE flag, which takes the next sorted argument as its value, and
331
+
Validate() refuses the result: not this code's to repair, and why docs
332
+
tell operators to delete such an entry rather than empty it.
333
+
334
+
Only --flag=value is matched, and the space-separated spelling does not
335
+
survive the collapse, which keys on "=". Written as two elements the flag
336
+
comes back bare and takes whichever argument sorts after it as its value;
337
+
written as one element the whole string becomes the key and reaches pflag
338
+
as an unknown flag. The apiserver refuses to start either way.
339
+
340
+
Match whole ELEMENTS and do NOT trim them: caseInsensitiveHas compares with
341
+
strings.EqualFold and pflag's CSV reader keeps surrounding spaces, so
342
+
"X-Remote-Uid-Extra"and "X-Remote-Uid " both fail there. Being more
343
+
permissive than upstream would skip the chart's flag for a list the
344
+
apiserver rejects. The emptiness test is the one that trims, because
345
+
checkForWhiteSpaceOnly does, and it refuses the list whatever else it holds.
{{- $uidHeaderUnsupported = le $effectiveMinor 31 }}
499
+
{{- $uidGateRequired = eq $effectiveMinor 32 }}
500
+
{{- if eq $uidGateSetting "false" }}
501
+
{{- $uidGateOptedOut = true }}
502
+
{{- else if ne $uidGateSetting "true" }}
503
+
{{- if $uidGateRequired }}
504
+
{{- if eq $allAlphaSetting "false" }}
505
+
{{- $uidGateOptedOut = true }}
506
+
{{- end }}
507
+
{{- else if eq $allBetaSetting "false" }}
508
+
{{- $uidGateOptedOut = true }}
509
+
{{- end }}
510
+
{{- end }}
511
+
{{- if not (or $uidHeaderUnsupported $uidHeaderQuoted $emulatedQuoted) }}
512
+
{{- if $uidGateOptedOut }}
513
+
{{- if ne $uidHeaderArgSeen "" }}
514
+
{{- fail (printf "controlPlane.apiServer.extraArgs contains %q alongside %q, while spec.oidc.mode is %q on Kubernetes %s — the kube-apiserver rejects a --requestheader-uid-headers list while RemoteRequestHeaderUID is disabled, so the control plane would refuse to start. Drop one of the two: your header flag, or the gate setting that turns RemoteRequestHeaderUID off." $uidHeaderArgSeen $gateArgSeen $.Values.oidc.mode $versionForMessage) }}
515
+
{{- end }}
516
+
{{- else }}
517
+
{{- if $uidHeaderHasEmpty }}
518
+
{{- fail (printf "controlPlane.apiServer.extraArgs contains %q while spec.oidc.mode is %q — the kube-apiserver rejects a uid-headers list with an empty or whitespace-only element, so the control plane would refuse to start. Remove the empty element from your entry, or drop the entry and let the chart render the flag." $uidHeaderArgSeen $.Values.oidc.mode) }}
519
+
{{- end }}
520
+
{{- if and (ne $uidHeaderArgSeen "") (not $uidHeaderConforms) }}
521
+
{{- fail (printf "controlPlane.apiServer.extraArgs contains %q while spec.oidc.mode is %q — the chart injects --requestheader-uid-headers=X-Remote-Uid so the authenticated user UID survives the aggregation layer, and the kube-apiserver rejects a uid-headers list that omits X-Remote-Uid. Add X-Remote-Uid to your entry, or drop the entry and let the chart render the flag." $uidHeaderArgSeen $.Values.oidc.mode) }}
0 commit comments