Injector/Operator: use RSA keys for webhook serving certs#9873
Merged
Conversation
The Kubernetes API server on some managed cloud distributions rejects Ed25519 serving certificates when invoking admission webhooks. After the workload-identity key algorithm was switched to Ed25519, the Dapr injector (mutating sidecar webhook) and operator (CRD conversion, validating, and mutating webhooks) could no longer be reached by the apiserver in those environments. The injector and operator each have a single SPIFFE identity that backs both their webhook TLS cert and any other mTLS they perform, so switching the entire identity algorithm is sufficient. Sentry's CA already accepts CSRs of any supported algorithm, so no signing-side changes are needed. This commit plumbs a new spiffe.KeyAlgorithm option through security.Options into spiffe.New (added in dapr/kit) and sets it to ptr.Of(spiffe.KeyAlgorithmRSA) in the injector and operator binaries. All other control plane components (sentry, placement, scheduler) and daprd workloads keep the Ed25519 default, since the new option is nil-defaulted. Signed-off-by: joshvanl <[email protected]>
Contributor
Author
|
/ok-to-test |
Collaborator
Dapr E2E testCommit ref: ebd665f ❌ Infrastructure deployment failed
Please check the logs for details on the failure. ❌ Build failed for linux/amd64Please check the logs for details on the error. ❌ Build failed for windows/amd64Please check the logs for details on the error. |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates Dapr’s Kubernetes control-plane components to use RSA workload keys for webhook-serving TLS certificates (injector/operator) to improve compatibility with Kubernetes API servers that reject Ed25519 serving certs, while keeping Ed25519 as the default elsewhere. It does this by plumbing a key-algorithm option through pkg/security into SPIFFE identity creation and adding integration coverage to assert the resulting serving-certificate key types.
Changes:
- Add
KeyAlgorithmtopkg/security.Optionsand pass it through to the SPIFFE identity provider. - Configure the injector and operator to request RSA workload keys (to back webhook-serving certs).
- Add integration tests + a shared TLS-cert capture helper to assert Ed25519 vs RSA serving certs across components.
Reviewed changes
Copilot reviewed 12 out of 13 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| tests/integration/suite/sentry/servingcerted.go | New integration test asserting Sentry serving cert is Ed25519. |
| tests/integration/suite/scheduler/servingcerted.go | New integration test asserting Scheduler serving cert is Ed25519. |
| tests/integration/suite/placement/servingcerted.go | New integration test asserting Placement serving cert is Ed25519. |
| tests/integration/suite/operator/servingcertrsa.go | New integration test asserting Operator webhook serving cert is RSA. |
| tests/integration/suite/injector/servingcertrsa.go | New integration test asserting Injector webhook serving cert is RSA. |
| tests/integration/suite/daprd/mtls/servingcerted.go | New integration test asserting daprd mTLS serving cert is Ed25519. |
| tests/integration/framework/process/operator/operator.go | Expose operator webhook port so tests can target the webhook TLS listener. |
| tests/integration/framework/cert/cert.go | Add helper to capture a server’s leaf cert and assert its public-key type (Ed25519/RSA). |
| pkg/security/security.go | Add KeyAlgorithm option and forward it into spiffe.New(...). |
| pkg/operator/operator.go | Set KeyAlgorithm to RSA for operator security identity (webhook-serving cert). |
| cmd/injector/app/app.go | Set KeyAlgorithm to RSA for injector security identity (webhook-serving cert). |
| go.mod | Adds a replace directive for github.com/dapr/kit (currently problematic; see PR comment). |
| go.sum | Updates sums consistent with the go.mod replace of github.com/dapr/kit. |
acroca
previously approved these changes
May 5, 2026
| // The injector serves a mutating admission webhook to the Kubernetes | ||
| // API server, which on some cloud distributions rejects Ed25519 | ||
| // serving certs. | ||
| KeyAlgorithm: ptr.Of(spiffe.KeyAlgorithmRSA), |
Member
There was a problem hiding this comment.
I guess we can't know which cert would be allowed, right? It'd be cool to use ed25519 when possible, but I'm not sure we can check if it's supported dynamically.
Contributor
Author
There was a problem hiding this comment.
Yeah it's tricky because as the webhook server it is difficult to tell why the peer didn't connect, and it would be a lot of machinery to rotate with a different key in these scenarios
Signed-off-by: joshvanl <[email protected]>
Contributor
Author
|
/ok-to-test |
Collaborator
Dapr E2E testCommit ref: 2a04932 ❌ Infrastructure deployment failed
Please check the logs for details on the failure. ❌ Build failed for linux/amd64Please check the logs for details on the error. ❌ Build failed for windows/amd64Please check the logs for details on the error. |
Signed-off-by: joshvanl <[email protected]>
JoshVanL
added
autoupdate
Closed
cicoyle
added
the
automerge
JoshVanL
added a commit
that referenced
this pull request
May 5, 2026
) * Injector/Operator: use RSA keys for webhook serving certs The Kubernetes API server on some managed cloud distributions rejects Ed25519 serving certificates when invoking admission webhooks. After the workload-identity key algorithm was switched to Ed25519, the Dapr injector (mutating sidecar webhook) and operator (CRD conversion, validating, and mutating webhooks) could no longer be reached by the apiserver in those environments. The injector and operator each have a single SPIFFE identity that backs both their webhook TLS cert and any other mTLS they perform, so switching the entire identity algorithm is sufficient. Sentry's CA already accepts CSRs of any supported algorithm, so no signing-side changes are needed. This commit plumbs a new spiffe.KeyAlgorithm option through security.Options into spiffe.New (added in dapr/kit) and sets it to ptr.Of(spiffe.KeyAlgorithmRSA) in the injector and operator binaries. All other control plane components (sentry, placement, scheduler) and daprd workloads keep the Ed25519 default, since the new option is nil-defaulted. * Update dapr/kit to origin HEAD * Fix serving cert algo int test --------- (cherry picked from commit dfd4ab3) Signed-off-by: joshvanl <[email protected]> Signed-off-by: dapr-bot <[email protected]> Co-authored-by: Josh van Leeuwen <[email protected]> Co-authored-by: Cassie Coyle <[email protected]>
github-project-automation
Bot
moved this from Backlog
to Done
in v1.18 Release Tracking Board
7 tasks
msfussell
added a commit
to dapr/docs
that referenced
this pull request
May 24, 2026
* sentry: document Ed25519 default workload identity keys Sentry switched workload X.509 certificate key generation from ECDSA P-256 to Ed25519 in Dapr 1.18 (dapr/dapr#9598). Changes: - concepts/security-concept.md: add 'Workload identity key algorithm' subsection under 'Secure Dapr to Dapr communication'. Covers what changed (default: Ed25519 for workload/CA certs, RSA for injector/ operator webhook serving certs, RSA-2048 for JWT/OIDC unchanged), why (smaller keys/sigs, faster ops, cleaner curve design), mixed- version rolling-upgrade compatibility, FIPS callout (Ed25519 is not FIPS-140 approved; BYO-CA path is the workaround), and downgrade floor alert (1.17.7). Also updates stale 'ECDSA private key' sentence in the same section. - operations/security/mtls.md: add 'Workload identity key algorithm' section at the top pointing to the concept page. Adds BYO-CA note explaining that supplied issuer key algorithm is what Sentry uses. Updates stale 'ECDSA PEM encoded' in self-hosted BYO-certs prose. Adds inline note on the Kubernetes openssl example clarifying that RSA keys are also accepted (relevant for FIPS environments). Related: dapr/dapr#9873 (injector/operator RSA webhook certs — tracked separately as gap R5). Targets v1.18. Signed-off-by: Nelson Parente <[email protected]> * sentry: tighten Ed25519 workload identity docs (review pass) Signed-off-by: Nelson Parente <[email protected]> --------- Signed-off-by: Nelson Parente <[email protected]> Co-authored-by: Mark Fussell <[email protected]>
msfussell
added a commit
to dapr/docs
that referenced
this pull request
May 26, 2026
* sentry: document Ed25519 default workload identity keys Sentry switched workload X.509 certificate key generation from ECDSA P-256 to Ed25519 in Dapr 1.18 (dapr/dapr#9598). Changes: - concepts/security-concept.md: add 'Workload identity key algorithm' subsection under 'Secure Dapr to Dapr communication'. Covers what changed (default: Ed25519 for workload/CA certs, RSA for injector/ operator webhook serving certs, RSA-2048 for JWT/OIDC unchanged), why (smaller keys/sigs, faster ops, cleaner curve design), mixed- version rolling-upgrade compatibility, FIPS callout (Ed25519 is not FIPS-140 approved; BYO-CA path is the workaround), and downgrade floor alert (1.17.7). Also updates stale 'ECDSA private key' sentence in the same section. - operations/security/mtls.md: add 'Workload identity key algorithm' section at the top pointing to the concept page. Adds BYO-CA note explaining that supplied issuer key algorithm is what Sentry uses. Updates stale 'ECDSA PEM encoded' in self-hosted BYO-certs prose. Adds inline note on the Kubernetes openssl example clarifying that RSA keys are also accepted (relevant for FIPS environments). Related: dapr/dapr#9873 (injector/operator RSA webhook certs — tracked separately as gap R5). Targets v1.18. * sentry: tighten Ed25519 workload identity docs (review pass) --------- Signed-off-by: Nelson Parente <[email protected]> Co-authored-by: Mark Fussell <[email protected]>
marcduiker
pushed a commit
to marcduiker/dapr-docs
that referenced
this pull request
May 28, 2026
* sentry: document Ed25519 default workload identity keys Sentry switched workload X.509 certificate key generation from ECDSA P-256 to Ed25519 in Dapr 1.18 (dapr/dapr#9598). Changes: - concepts/security-concept.md: add 'Workload identity key algorithm' subsection under 'Secure Dapr to Dapr communication'. Covers what changed (default: Ed25519 for workload/CA certs, RSA for injector/ operator webhook serving certs, RSA-2048 for JWT/OIDC unchanged), why (smaller keys/sigs, faster ops, cleaner curve design), mixed- version rolling-upgrade compatibility, FIPS callout (Ed25519 is not FIPS-140 approved; BYO-CA path is the workaround), and downgrade floor alert (1.17.7). Also updates stale 'ECDSA private key' sentence in the same section. - operations/security/mtls.md: add 'Workload identity key algorithm' section at the top pointing to the concept page. Adds BYO-CA note explaining that supplied issuer key algorithm is what Sentry uses. Updates stale 'ECDSA PEM encoded' in self-hosted BYO-certs prose. Adds inline note on the Kubernetes openssl example clarifying that RSA keys are also accepted (relevant for FIPS environments). Related: dapr/dapr#9873 (injector/operator RSA webhook certs — tracked separately as gap R5). Targets v1.18. Signed-off-by: Nelson Parente <[email protected]> * sentry: tighten Ed25519 workload identity docs (review pass) Signed-off-by: Nelson Parente <[email protected]> --------- Signed-off-by: Nelson Parente <[email protected]> Co-authored-by: Mark Fussell <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Kubernetes API server on some managed cloud distributions rejects Ed25519 serving certificates when invoking admission webhooks. After the workload-identity key algorithm was switched to Ed25519, the Dapr injector (mutating sidecar webhook) and operator (CRD conversion, validating, and mutating webhooks) could no longer be reached by the apiserver in those environments.
The injector and operator each have a single SPIFFE identity that backs both their webhook TLS cert and any other mTLS they perform, so switching the entire identity algorithm is sufficient. Sentry's CA already accepts CSRs of any supported algorithm, so no signing-side changes are needed.
This commit plumbs a new spiffe.KeyAlgorithm option through security.Options into spiffe.New (added in dapr/kit) and sets it to ptr.Of(spiffe.KeyAlgorithmRSA) in the injector and operator binaries. All other control plane components (sentry, placement, scheduler) and daprd workloads keep the Ed25519 default, since the new option is nil-defaulted.