Skip to content

Commit 51c5273

Browse files
authored
feat(platform): ACME wildcard certificate on the default ingress-nginx path (#2988)
## What this PR does On the default ingress-nginx path (`gateway.enabled=false`) every published hostname still minted its own per-host ACME certificate via ingress-shim, even with a DNS-01 solver configured. That hits the Let's Encrypt rate limit (50 certs per registered domain per week) once a deployment exceeds ~50 endpoints. The Gateway API path already issues a single per-apex wildcard in DNS-01 mode; this brings the ingress-nginx path to parity. It implements the remaining ingress-nginx gap that #2400 was narrowed to in its triage comment (DNS-01 is already multi-provider, and the Gateway path already issues wildcards). The new `publishing.certificates.wildcard` toggle is opt-in and OFF by default. When set to `true` with `solver=dns01` and no operator-provided wildcard Secret, the platform issues one shared wildcard `Certificate` for `<root-host>` + `*.<root-host>` and serves the resulting Secret as the publishing controller's `--default-ssl-certificate`. The system service Ingresses (dashboard, grafana, keycloak, harbor, …) then stop requesting a per-host cert. How it works: - `core/platform` `apps.yaml` computes the effective wildcard Secret name and a new `_cluster.wildcard-issue` signal. An operator-provided `wildcardSecretName` (the BYO path) always wins and never triggers issuance; HTTP-01 cannot issue wildcards; and when Gateway API is enabled the TenantGateway controller issues the wildcard instead — so all three are excluded from auto-issuance. - `extra/ingress` renders the wildcard `Certificate` only on the publishing controller (`Release.Namespace == expose-ingress`), because the Secret must be same-namespace for ingress-nginx to read it. The Certificate references the DNS-01 `ClusterIssuer` that `cert-manager-issuers` already renders in `dns01` mode, so no per-tenant Issuer is minted. This reuses the existing wildcard-secret consumption path, so no per-service Ingress template changes were needed — only the issuance side is new. Why opt-in (default off): the chosen wildcard Secret name rides the `_cluster` channel, which every child tenant inherits verbatim, so this is not root-tenant scoped. A single-label wildcard `*.<root-host>` covers `<service>.<root-host>` but not a custom service host outside it (a keycloak `ingress.host`, harbor `host`, or grafana `host` pointed at another domain) nor a child tenant's nested host (`<service>.<tenant>.<root-host>`). Enabling issuance makes every such service drop its per-host ACME cert and fall back to the default certificate, which does not cover it. Leaving it off by default means a dns01 cluster is never silently switched on upgrade; an operator who enables it accepts the same coverage responsibility as the operator-provided wildcard path, which propagates identically. The tradeoff is pinned by contract tests (keycloak custom host, bucket nested child-tenant host). Per-tenant wildcards remain on the Gateway path and are tracked separately. Docs: cozystack/website#588 Closes #2400. Part of #2811. ### Screenshots N/A — no UI changes. ### Release note ```release-note feat(platform): add opt-in shared wildcard certificate issuance on the default ingress-nginx path via publishing.certificates.wildcard (default false). When enabled with a DNS-01 solver, the platform issues one *.<root-host> wildcard for system services instead of a per-host ACME certificate, avoiding Let's Encrypt rate limits at scale. ``` <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Added automatic wildcard certificate issuance support for DNS-01 ACME solver. * New `certificates.wildcard` configuration option to enable shared wildcard TLS certificates. * Ingress rules now properly utilize wildcard certificates when enabled. * **Tests** * Added comprehensive test suites validating wildcard certificate generation and ingress integration across multiple scenarios. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
2 parents ad8367c + 4218e4c commit 51c5273

7 files changed

Lines changed: 410 additions & 5 deletions

File tree

‎packages/core/platform/templates/apps.yaml‎

Lines changed: 26 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -24,16 +24,37 @@ stringData:
2424
solver: {{ .Values.publishing.certificates.solver | quote }}
2525
issuer-name: {{ .Values.publishing.certificates.issuerName | quote }}
2626
{{- /*
27-
Operator-supplied wildcard mode. Only the Secret NAME rides this
27+
Wildcard certificate mode. Only the Secret NAME rides this
2828
channel — never the cert/key material (unlike a private key, the
2929
name is not sensitive). When set, the ingress flow points the
3030
root ingress controller's --default-ssl-certificate at it and
3131
drops per-host ACME, and the gateway chart switches the
32-
TenantGateway to certMode=existingSecret. The Secret itself must
33-
be pre-created by the operator in the publishing namespace
34-
(tenant-root). Empty string = ACME issuance as before.
32+
TenantGateway to certMode=existingSecret.
33+
34+
The name comes from one of two sources, with the operator-
35+
provided (BYO) name always winning:
36+
1. publishing.certificates.wildcardSecretName — a Secret the
37+
operator pre-creates in the publishing namespace
38+
(tenant-root). wildcard-issue stays false: the platform
39+
mints nothing.
40+
2. Otherwise, on the ingress-nginx path (gateway.enabled=false)
41+
with solver=dns01 and publishing.certificates.wildcard=true,
42+
the platform issues its own shared wildcard. wildcard-issue
43+
becomes true and the publishing ingress package renders the
44+
Certificate into this Secret name. http01 cannot issue
45+
wildcards, and the Gateway API path issues its own wildcard
46+
via the TenantGateway controller, so both are excluded.
47+
Empty name + wildcard-issue=false = per-host ACME as before.
3548
*/}}
36-
wildcard-secret-name: {{ .Values.publishing.certificates.wildcardSecretName | default "" | quote }}
49+
{{- $byoWildcard := .Values.publishing.certificates.wildcardSecretName | default "" }}
50+
{{- $wildcardSecretName := $byoWildcard }}
51+
{{- $wildcardIssue := false }}
52+
{{- if and (not $byoWildcard) (eq .Values.publishing.certificates.solver "dns01") .Values.publishing.certificates.wildcard (not .Values.gateway.enabled) }}
53+
{{- $wildcardSecretName = "cozystack-wildcard-tls" }}
54+
{{- $wildcardIssue = true }}
55+
{{- end }}
56+
wildcard-secret-name: {{ $wildcardSecretName | quote }}
57+
wildcard-issue: {{ $wildcardIssue | quote }}
3758
{{- /*
3859
DNS-01 solver wiring. Read only when solver=dns01, but the
3960
keys are written unconditionally so a render-time switch from
Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
suite: apps.yaml ACME wildcard issuance wiring
2+
3+
templates:
4+
- templates/apps.yaml
5+
6+
release:
7+
name: cozy-platform
8+
namespace: cozy-system
9+
10+
# WS2 / #2400: on the default ingress-nginx path (gateway.enabled=false) a
11+
# DNS-01 solver lets the platform issue ONE shared wildcard certificate
12+
# (*.<root-host> + <root-host>) instead of minting a per-host ACME cert
13+
# for every system service. It is OPT-IN (publishing.certificates.wildcard,
14+
# default false): a single-label wildcard does not cover custom service
15+
# hosts outside *.<root-host>, so leaving it off keeps the safe per-host
16+
# behavior and never switches a dns01 cluster silently on upgrade.
17+
# apps.yaml computes the effective _cluster.wildcard-secret-name and an
18+
# _cluster.wildcard-issue signal the publishing ingress package reads to
19+
# render the Certificate. Operator-provided (BYO) wildcard names always
20+
# win and never trigger issuance; http01 cannot issue wildcards; the
21+
# Gateway API path issues its own wildcard via the controller, so issuance
22+
# here is gated to gateway.enabled=false.
23+
24+
tests:
25+
- it: does not issue a wildcard by default (http01)
26+
asserts:
27+
- matchRegex:
28+
path: stringData["values.yaml"]
29+
pattern: 'wildcard-secret-name:\s*""'
30+
- matchRegex:
31+
path: stringData["values.yaml"]
32+
pattern: 'wildcard-issue:\s*"false"'
33+
34+
- it: does not auto-issue on dns01 unless explicitly opted in
35+
set:
36+
publishing.certificates.solver: dns01
37+
asserts:
38+
- matchRegex:
39+
path: stringData["values.yaml"]
40+
pattern: 'wildcard-secret-name:\s*""'
41+
- matchRegex:
42+
path: stringData["values.yaml"]
43+
pattern: 'wildcard-issue:\s*"false"'
44+
45+
- it: issues a shared wildcard when opted in on the dns01 ingress path
46+
set:
47+
publishing.certificates.solver: dns01
48+
publishing.certificates.wildcard: true
49+
asserts:
50+
- matchRegex:
51+
path: stringData["values.yaml"]
52+
pattern: 'wildcard-secret-name:\s*"cozystack-wildcard-tls"'
53+
- matchRegex:
54+
path: stringData["values.yaml"]
55+
pattern: 'wildcard-issue:\s*"true"'
56+
57+
- it: does not auto-issue on the ingress path when Gateway API is enabled
58+
set:
59+
publishing.certificates.solver: dns01
60+
publishing.certificates.wildcard: true
61+
gateway.enabled: true
62+
asserts:
63+
- matchRegex:
64+
path: stringData["values.yaml"]
65+
pattern: 'wildcard-secret-name:\s*""'
66+
- matchRegex:
67+
path: stringData["values.yaml"]
68+
pattern: 'wildcard-issue:\s*"false"'
69+
70+
- it: operator-provided wildcard secret wins and never triggers issuance
71+
set:
72+
publishing.certificates.solver: dns01
73+
publishing.certificates.wildcard: true
74+
publishing.certificates.wildcardSecretName: byo-wildcard-tls
75+
asserts:
76+
- matchRegex:
77+
path: stringData["values.yaml"]
78+
pattern: 'wildcard-secret-name:\s*"byo-wildcard-tls"'
79+
- matchRegex:
80+
path: stringData["values.yaml"]
81+
pattern: 'wildcard-issue:\s*"false"'
82+
83+
- it: http01 never issues a wildcard even when opted in
84+
set:
85+
publishing.certificates.solver: http01
86+
publishing.certificates.wildcard: true
87+
asserts:
88+
- matchRegex:
89+
path: stringData["values.yaml"]
90+
pattern: 'wildcard-secret-name:\s*""'
91+
- matchRegex:
92+
path: stringData["values.yaml"]
93+
pattern: 'wildcard-issue:\s*"false"'

‎packages/core/platform/values.yaml‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -199,6 +199,50 @@ publishing:
199199
certificates:
200200
solver: http01 # "http01" or "dns01"
201201
issuerName: letsencrypt-prod
202+
# Shared wildcard issuance on the default ingress-nginx path
203+
# (gateway.enabled=false). Opt-in, OFF by default. When set to true
204+
# with solver=dns01 and no operator-provided wildcardSecretName, the
205+
# platform issues ONE wildcard Certificate (*.<root-host> +
206+
# <root-host>) via the DNS-01 ClusterIssuer and serves it as the
207+
# ingress controller's default SSL certificate, so system services
208+
# (dashboard, grafana, keycloak, harbor, …) stop minting a per-host
209+
# ACME cert each. This avoids the Let's Encrypt per-domain rate limit
210+
# once a deployment exceeds ~50 endpoints, matching the Gateway API
211+
# path which issues a per-apex wildcard in dns01 mode.
212+
#
213+
# Ignored when solver=http01 (HTTP-01 cannot issue wildcards) and
214+
# when gateway.enabled=true (the TenantGateway controller issues the
215+
# wildcard there).
216+
#
217+
# COVERAGE CAVEAT — same contract as wildcardSecretName below: a
218+
# single-label wildcard *.<root-host> covers <service>.<root-host>
219+
# but NOT a custom service host outside it (e.g. a keycloak
220+
# ingress.host / harbor host / grafana host set to auth.example.com,
221+
# or a nested a.b.<root-host>). When this is true, services whose
222+
# host is not covered fall back to the default SSL certificate and
223+
# would serve the wrong certificate. Only enable it when every
224+
# exposed system service uses a <service>.<root-host> host (the
225+
# default), or provide a covering wildcardSecretName instead. Left
226+
# OFF by default precisely so a dns01 cluster with a custom service
227+
# host is never silently switched on upgrade.
228+
#
229+
# BLAST RADIUS — like wildcardSecretName, the chosen name rides the
230+
# _cluster channel, which every child tenant inherits verbatim. A
231+
# child tenant that inherits the root ingress has nested service
232+
# hosts (<service>.<tenant>.<root-host>) that *.<root-host> does NOT
233+
# cover; enabling this makes those services drop per-host ACME and
234+
# serve the root default certificate too. This is not root-tenant
235+
# scoped — only enable it when child-tenant hosts are also covered
236+
# (or left on the Gateway path). Per-tenant wildcards are tracked
237+
# separately.
238+
#
239+
# FIRST-ISSUANCE WINDOW: when this is turned on, consumers drop their
240+
# per-host certs immediately, but the DNS-01 wildcard takes a few
241+
# minutes to issue. Until the Secret is populated ingress-nginx serves
242+
# its self-signed default cert (browser warning), then self-heals —
243+
# the same delay per-host ACME has on first issue, which an
244+
# operator-provided wildcardSecretName avoids only by pre-existing.
245+
wildcard: false
202246
# Operator-provided wildcard certificate. When set, platform
203247
# services and the root tenant's ingress/Gateway serve under this
204248
# pre-existing TLS Secret instead of minting per-host ACME certs.
Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
{{- $exposeIngress := (index .Values._cluster "expose-ingress") | default "tenant-root" }}
2+
{{- /* toString normalises the _cluster value whether it arrives as the
3+
string "true" (Flux valuesFrom Secret, the real path) or as a bool
4+
(helm --set), mirroring the proxy-protocol handling in
5+
nginx-ingress.yaml. */ -}}
6+
{{- $wildcardIssue := (index .Values._cluster "wildcard-issue") | default "false" | toString }}
7+
{{- $wildcardSecret := (index .Values._cluster "wildcard-secret-name") | default "" }}
8+
{{- $rootHost := (index .Values._cluster "root-host") | default "" }}
9+
{{- $issuerName := (index .Values._cluster "issuer-name") | default "letsencrypt-prod" }}
10+
{{- /*
11+
Shared ACME wildcard for the ingress-nginx path (WS2 / #2400). When the
12+
platform decides to issue a wildcard (_cluster.wildcard-issue=true,
13+
computed in core/platform apps.yaml only on the solver=dns01 ingress
14+
path with gateway.enabled=false), the publishing tenant's ingress
15+
controller renders ONE DNS-01 Certificate for <root-host> +
16+
*.<root-host>. ingress-nginx serves the resulting Secret as its
17+
--default-ssl-certificate (see nginx-ingress.yaml), so the system
18+
services drop their per-host cert-manager annotation — matching the
19+
Gateway API path, which issues a per-apex wildcard via the TenantGateway
20+
controller in dns01 mode.
21+
22+
Only the publishing controller (Release.Namespace == expose-ingress)
23+
renders it: the Secret must live in that namespace for ingress-nginx to
24+
read it, and child-tenant controllers cannot point --default-ssl-certificate
25+
at another namespace. The reference is therefore always same-namespace,
26+
the same constraint the operator-supplied wildcard path observes.
27+
28+
The ClusterIssuer named here (issuer-name, letsencrypt-prod by default)
29+
already carries the DNS-01 solver from packages/system/cert-manager-issuers
30+
whenever solver=dns01, so no per-tenant Issuer is minted on this path.
31+
*/}}
32+
{{- if and (eq $wildcardIssue "true") (eq .Release.Namespace $exposeIngress) }}
33+
{{- if not $wildcardSecret }}
34+
{{- fail "packages/extra/ingress: _cluster.wildcard-issue=true but _cluster.wildcard-secret-name is empty — the platform must supply the target Secret name." }}
35+
{{- end }}
36+
{{- if not $rootHost }}
37+
{{- fail "packages/extra/ingress: _cluster.wildcard-issue=true but _cluster.root-host is empty — cannot build the wildcard dnsNames." }}
38+
{{- end }}
39+
apiVersion: cert-manager.io/v1
40+
kind: Certificate
41+
metadata:
42+
name: cozystack-wildcard
43+
spec:
44+
secretName: {{ $wildcardSecret | quote }}
45+
issuerRef:
46+
kind: ClusterIssuer
47+
name: {{ $issuerName | quote }}
48+
dnsNames:
49+
- {{ $rootHost | quote }}
50+
- {{ printf "*.%s" $rootHost | quote }}
51+
{{- end }}
Lines changed: 115 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,115 @@
1+
suite: ingress-nginx ACME wildcard Certificate
2+
3+
templates:
4+
- templates/wildcard-certificate.yaml
5+
6+
release:
7+
name: ingress
8+
namespace: tenant-root
9+
10+
# WS2 / #2400: when the platform decides to issue a shared wildcard on the
11+
# ingress-nginx path (_cluster.wildcard-issue=true, computed in
12+
# core/platform apps.yaml only for solver=dns01 + gateway disabled), the
13+
# publishing tenant's ingress controller renders ONE DNS-01 Certificate
14+
# for <root-host> + *.<root-host>. ingress-nginx serves the resulting
15+
# Secret as its default SSL certificate, so per-host ACME is unnecessary.
16+
# Only the publishing controller (Release.Namespace == expose-ingress)
17+
# renders it — the Secret must live in that namespace and child-tenant
18+
# controllers cannot read it. The ClusterIssuer named here already carries
19+
# the DNS-01 solver whenever solver=dns01.
20+
21+
tests:
22+
- it: renders a DNS-01 wildcard Certificate on the publishing controller
23+
set:
24+
_cluster:
25+
expose-ingress: tenant-root
26+
wildcard-issue: "true"
27+
wildcard-secret-name: cozystack-wildcard-tls
28+
root-host: example.org
29+
issuer-name: letsencrypt-prod
30+
asserts:
31+
- hasDocuments:
32+
count: 1
33+
- equal:
34+
path: kind
35+
value: Certificate
36+
- equal:
37+
path: metadata.name
38+
value: cozystack-wildcard
39+
- equal:
40+
path: spec.secretName
41+
value: cozystack-wildcard-tls
42+
- equal:
43+
path: spec.issuerRef.kind
44+
value: ClusterIssuer
45+
- equal:
46+
path: spec.issuerRef.name
47+
value: letsencrypt-prod
48+
- equal:
49+
path: spec.dnsNames
50+
value:
51+
- example.org
52+
- "*.example.org"
53+
54+
- it: renders nothing when wildcard-issue is not set
55+
set:
56+
_cluster:
57+
expose-ingress: tenant-root
58+
wildcard-secret-name: ""
59+
root-host: example.org
60+
asserts:
61+
- hasDocuments:
62+
count: 0
63+
64+
- it: does not render on a non-publishing tenant controller
65+
release:
66+
name: ingress
67+
namespace: tenant-foo
68+
set:
69+
_cluster:
70+
expose-ingress: tenant-root
71+
wildcard-issue: "true"
72+
wildcard-secret-name: cozystack-wildcard-tls
73+
root-host: example.org
74+
asserts:
75+
- hasDocuments:
76+
count: 0
77+
78+
- it: honors a custom issuer name
79+
set:
80+
_cluster:
81+
expose-ingress: tenant-root
82+
wildcard-issue: "true"
83+
wildcard-secret-name: cozystack-wildcard-tls
84+
root-host: example.org
85+
issuer-name: letsencrypt-stage
86+
asserts:
87+
- equal:
88+
path: spec.issuerRef.name
89+
value: letsencrypt-stage
90+
91+
# Fail-fast guards: platform-internal invariants that should never trip,
92+
# but must fail loudly (not render a Certificate pointing at a
93+
# nonexistent Secret or with empty dnsNames) if apps.yaml ever regresses
94+
# and sets wildcard-issue=true without the keys it implies.
95+
- it: fails when wildcard-issue is true but the secret name is empty
96+
set:
97+
_cluster:
98+
expose-ingress: tenant-root
99+
wildcard-issue: "true"
100+
wildcard-secret-name: ""
101+
root-host: example.org
102+
asserts:
103+
- failedTemplate:
104+
errorMessage: "packages/extra/ingress: _cluster.wildcard-issue=true but _cluster.wildcard-secret-name is empty — the platform must supply the target Secret name."
105+
106+
- it: fails when wildcard-issue is true but root-host is empty
107+
set:
108+
_cluster:
109+
expose-ingress: tenant-root
110+
wildcard-issue: "true"
111+
wildcard-secret-name: cozystack-wildcard-tls
112+
root-host: ""
113+
asserts:
114+
- failedTemplate:
115+
errorMessage: "packages/extra/ingress: _cluster.wildcard-issue=true but _cluster.root-host is empty — cannot build the wildcard dnsNames."

‎packages/system/bucket/tests/ingress_wildcard_test.yaml‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,3 +51,32 @@ tests:
5151
value: letsencrypt-prod
5252
- exists:
5353
path: spec.tls[0].secretName
54+
55+
# WS2 / #2400 child-tenant blast radius, pinned as a contract: the
56+
# wildcard secret name rides _cluster, which packages/apps/tenant copies
57+
# verbatim into every child tenant. A child inheriting the root ingress
58+
# has nested service hosts (<svc>.<tenant>.<root-host>) that a
59+
# single-label *.<root-host> does NOT cover, yet the service still drops
60+
# its per-host cert and falls back to the root default certificate. This
61+
# is the documented reason shared wildcard issuance is opt-in / off by
62+
# default. Here _namespace.host is the child apex (alice.example.org),
63+
# so the bucket host is two labels deep under the wildcard.
64+
- it: nested child-tenant host drops per-host ACME when wildcard mode is active
65+
set:
66+
bucketName: data
67+
_namespace:
68+
host: alice.example.org
69+
ingress: tenant-root
70+
gateway: ""
71+
_cluster:
72+
issuer-name: letsencrypt-prod
73+
solver: dns01
74+
wildcard-secret-name: cozystack-wildcard-tls
75+
asserts:
76+
- equal:
77+
path: spec.rules[0].host
78+
value: data.alice.example.org
79+
- notExists:
80+
path: metadata.annotations["cert-manager.io/cluster-issuer"]
81+
- notExists:
82+
path: spec.tls[0].secretName

0 commit comments

Comments
 (0)