Skip to content

Commit 4b2507b

Browse files
committed
docs(gateway): align gateway prose with the code
The comment on the shared CEL string said what the hostname requirement is but not what it is worth per kind. Two of the three schemas leave spec.hostnames optional at every version the vendored bundle serves, so for HTTPRoute and GRPCRoute this policy is the only rejection a hostname-less route meets; tlsroutes requires the field at v1 and v1alpha3 and not at v1alpha2, so there the policy closes the gap for as long as v1alpha2 is served. State that where the requirement is stated. The comment on the port-80 allowedRoutes filter credited the cert-manager namespace entry for letting ACME complete, which the paragraph above it retracts: the solver route lives in the tenant namespace, and that is what keeps ACME working. The same sentence claimed the filter keeps app routes off port 80, and the Layer 1 paragraph of the gateway README claimed it too. It does not. A tenant that owns its Gateway is its own gateway namespace, so its app routes are admitted by this very filter; what it excludes is cozy-* and inheriting children. Both copies now say that. Nothing said why naming the apex on the redirect costs no coverage, which is the question a reader has once the route stops matching every host. It holds because of the other half of this change: the port-80 listener admits routes only from the tenant namespace and cert-manager's, and the tightened route policy requires every route there to declare hostnames inside the apex. An inheriting child does not widen that set either, and the comment now says so at the point the hostnames are chosen: the gateway hostname policy compares every listener against the host label of the Gateway's own namespace, so a child apex outside it never gets its listener, the Gateway write carrying it being refused at admission whole, in reconcileGateway and before this route is rendered, while a child apex under this one passes that check and the wildcard already covers it. The empty-apex guard in renderHTTPRedirect, and the test pinning it, both read as though the guard is what spares an operator an unreadable schema error. It is, but the condition stated for it named the number of attached routes, which does not enter into it. In HTTP-01 the listener set is built from route hostnames alone, so reconcileGateway never reads the apex ahead of the guard however many routes attach. DNS-01 and existingSecret render an apex listener, and any tlsPassthroughServices entry builds <service>.<apex>, so in those the operator meets the equivalent error against the Gateway first. State that condition once, at the guard, and let the test defer to it rather than carry a second copy that can drift; the test stays green either way because the fake client does not validate against the CRD schema. The e2e suite explained omitting the TLSRoute empty-list probe by the v1 schema rejecting an empty list. Both halves of that are true and the conclusion does not follow: the probe runs at v1alpha2, which marks spec.hostnames neither required nor minItems, so an empty list reaches admission there exactly as an absent field does. What makes the omission safe is that all three policies render one byte-identical CEL string, so the HTTPRoute pair proves that term already. Assisted-by: LLM Signed-off-by: Aleksei Sviridkin <[email protected]>
1 parent 9007115 commit 4b2507b

5 files changed

Lines changed: 66 additions & 19 deletions

File tree

‎hack/e2e-chainsaw/gateway/chainsaw-test.yaml‎

Lines changed: 13 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -509,13 +509,19 @@ spec:
509509
# pin that the terms are present in the expression — this step is what proves
510510
# the apiserver acts on them.
511511
#
512-
# HTTPRoute carries both cases; TLSRoute and GRPCRoute contribute only the
513-
# absent one. At v1 the TLSRoute CRD schema requires spec.hostnames and
514-
# rejects an empty list on minItems, leaving the deprecated v1alpha2 the
515-
# only version where a hostname-less TLSRoute reaches admission at all.
516-
# Scripted rather than an
517-
# apply/expect pair so the grep can prove the VAP did the denying: a bare
518-
# "the apply failed" assertion would be satisfied by a typo in the manifest.
512+
# HTTPRoute carries both cases. The TLSRoute probe is applied at v1alpha2
513+
# because v1 and v1alpha3 mark spec.hostnames required with minItems: 1,
514+
# leaving v1alpha2 the only version where a hostname-less TLSRoute reaches
515+
# admission at all. It and the GRPCRoute probe carry the absent case only,
516+
# and that is a coverage choice rather than a reachability fact: v1alpha2
517+
# sets neither constraint, and the grpcroutes schema sets neither at any
518+
# served version, so an explicitly empty list reaches admission on both
519+
# just as an absent field does. All three policies render one
520+
# byte-identical CEL string, so the HTTPRoute pair above already proves
521+
# the empty-list term end to end.
522+
# Scripted rather than an apply/expect pair so the grep can prove the VAP
523+
# did the denying: a bare "the apply failed" assertion would be satisfied
524+
# by a typo in the manifest.
519525
- name: hostname-less-route-denied
520526
try:
521527
- script:

‎internal/controller/tenantgateway/reconciler_test.go‎

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1143,15 +1143,24 @@ func TestReconcile_OtherControllerOwnedRouteStillClaimsHostnames(t *testing.T) {
11431143
// hostnames ["", "*."], and both fail the HTTPRoute CRD's minLength and
11441144
// pattern, so the create is rejected and the reconcile dies on an
11451145
// apiserver validation error that says nothing about which field is at
1146-
// fault. Before the redirect carried hostnames an empty apex reached
1147-
// Ready, so this is a narrowing this branch is responsible for closing.
1146+
// fault. Under HTTP-01 an empty apex reached Ready before the redirect
1147+
// carried hostnames, so that much is a narrowing this branch closes.
1148+
// DNS-01 and existingSecret never reached Ready on an empty apex, at
1149+
// merge base or now: renderGateway builds its listener hostnames off
1150+
// the same field and fails ahead of the redirect either way.
11481151
//
11491152
// MinLength=1 on the field is the real guard and it stops this at
11501153
// admission. This check is the second layer, for the window where the
11511154
// CRD in the cluster is older than the controller binary: the two are
11521155
// rolled out separately, so the controller cannot assume the field
11531156
// validation it needs is already present. It must fail on its own terms
11541157
// and name the field.
1158+
//
1159+
// Which layer fires first depends on the cert mode; renderHTTPRedirect
1160+
// states the condition at the guard itself. This case stays green
1161+
// either way because the fake client does not validate against the CRD
1162+
// schema, so read it as pinning the error text, not as evidence of
1163+
// which layer fires.
11551164
func TestReconcile_EmptyApexFailsWithNamedError(t *testing.T) {
11561165
s := newScheme(t)
11571166
tgw := &gatewayv1alpha1.TenantGateway{

‎internal/controller/tenantgateway/renderers.go‎

Lines changed: 33 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -138,14 +138,14 @@ func buildAllowedRoutes(tgw *gatewayv1alpha1.TenantGateway) *gatewayv1.AllowedRo
138138
// itself runs in (see acmeChallengeNamespace).
139139
//
140140
// Why: app HTTPRoutes (harbor, keycloak, dashboard, bucket) attach
141-
// by hostname with no sectionName, so without this narrower filter
142-
// they would also bind to the HTTP listener — Gateway API
143-
// tie-breaks merged routes by creationTimestamp, so an app route
144-
// created before the controller's redirect would silently serve
145-
// plaintext on port 80 and leak credentials. Restricting the HTTP
146-
// listener's allowedRoutes namespaces excludes the cozy-* / tenant-*
147-
// namespaces apps live in, while keeping cert-manager's challenge
148-
// namespace open so ACME still completes.
141+
// by hostname with no sectionName, so any of them the filter admits
142+
// reaches the HTTP listener and can serve plaintext there. The filter
143+
// keeps out the ones published from elsewhere; an app in the Gateway's
144+
// own namespace is admitted and is not covered by this. Restricting the HTTP
145+
// listener's allowedRoutes namespaces keeps out routes from the other
146+
// cozy-* namespaces and from inheriting child tenants. The Gateway's
147+
// own namespace stays open, because the redirect and the ACME solver
148+
// route both live there, and so does acmeChallengeNamespace.
149149
func buildHTTPListenerAllowedRoutes(tgw *gatewayv1alpha1.TenantGateway) *gatewayv1.AllowedRoutes {
150150
values := []string{tgw.Namespace}
151151
if acmeChallengeNamespace != tgw.Namespace {
@@ -485,6 +485,14 @@ func (r *Reconciler) renderHTTPRedirect(tgw *gatewayv1alpha1.TenantGateway) (*ga
485485
// HTTPRoute schema. That surfaces as an apiserver validation error
486486
// naming the route rather than the field, which is a poor thing to
487487
// hand an operator.
488+
//
489+
// This wins the race in HTTP-01 without tlsPassthroughServices: the
490+
// listener set is built from route hostnames alone there, so however
491+
// many routes are attached, reconcileGateway never reads the apex
492+
// ahead of this point. DNS-01 and existingSecret render an apex
493+
// listener, and any tlsPassthroughServices entry builds
494+
// "<service>.<apex>", so in those the operator meets the equivalent
495+
// schema error against the Gateway first.
488496
if tgw.Spec.Apex == "" {
489497
return nil, fmt.Errorf("spec.apex is empty on TenantGateway %s/%s: the http-to-https redirect route derives its hostnames from the apex and cannot be rendered without one", tgw.Namespace, tgw.Name)
490498
}
@@ -513,7 +521,23 @@ func (r *Reconciler) renderHTTPRedirect(tgw *gatewayv1alpha1.TenantGateway) (*ga
513521
// "*.<apex>" covers every subdomain at every depth, but it
514522
// does not cover the bare "<apex>", hence the plain apex
515523
// first.
516-
524+
//
525+
// Naming them costs no coverage, and the other half of
526+
// this change is why: the port-80 listener admits routes
527+
// only from this namespace and cert-manager's, and the
528+
// tightened cozystack-route-hostname-policy requires every
529+
// route here to declare hostnames inside the apex. So any
530+
// host published from this namespace is already one of
531+
// these two.
532+
//
533+
// An inheriting child cannot widen that set either.
534+
// cozystack-gateway-hostname-policy compares every listener
535+
// against the host label of the Gateway's own namespace, so
536+
// a child apex outside it never gets its listener: the
537+
// Gateway write carrying it is refused at admission whole,
538+
// in reconcileGateway and before this route is rendered. A
539+
// child apex under this one passes that check, and the
540+
// wildcard above already covers it.
517541
Hostnames: []gatewayv1.Hostname{
518542
gatewayv1.Hostname(tgw.Spec.Apex),
519543
gatewayv1.Hostname("*." + tgw.Spec.Apex),

‎packages/extra/gateway/README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -103,7 +103,7 @@ The protections below split into three groups by who they defend against:
103103
- **Defense-in-depth** — Layers 1, 2, 5, 6, 7, 8. These do not protect against tenant-user input (tenants don't hold the relevant RBAC). They guard against bugs in cozystack-controller / Flux, supply-chain compromise of an app chart that emits Gateway API or Ingress resources, and confused-deputy mistakes by a cluster admin. Fail-closed via `failurePolicy: Fail` + `validationActions: [Deny]`. Layers 1-7 cover the opt-in Gateway API dataplane; Layer 8 covers the legacy Ingress dataplane, which is the default (Gateway API is off by default).
104104
- **Admin-against-themselves** — Layer 3 (`cozystack-gateway-attached-namespaces-policy`). Rejects a `kubectl edit packages.cozystack.io` that would slip a `tenant-*` entry into the platform Package's `gateway.attachedNamespaces`. Layer 6 catches the same misconfiguration at helm render time.
105105

106-
1. **Namespace whitelist on listeners.** Every listener carries an `allowedRoutes.namespaces.from: Selector`, and there are two of them. The plain-HTTP port-80 listener selects on the built-in `kubernetes.io/metadata.name` label, which kube-apiserver writes and a namespace cannot forge. The HTTPS-terminate and TLS-passthrough listeners select on `namespace.cozystack.io/gateway`, which is an ordinary namespace label written by the `apps/tenant` chart and by the controller, so it carries no such guarantee on its own: what keeps a namespace from applying it and attaching is that `Namespace` is cluster-scoped and no tenant role grants a verb on it, and what such a namespace could publish if it did is bounded by the route-hostname policies of Layer 2. HTTPS / TLS-passthrough listeners accept routes from the publishing tenant's namespace plus `gateway.attachedNamespaces` in the platform chart, which the controller labels for exactly that purpose (the default list includes the `cozy-*` namespaces for platform services and `default` for the Kubernetes API TLSRoute). A namespace carrying neither the name match nor that label cannot attach any `HTTPRoute` or `TLSRoute` to those listeners. Under `certMode: edge` there are no HTTPS listeners to describe: the Gateway serves port 80 only, every listener carries the same broad selector, and the narrow port-80 rule stated further down this item does not apply — see [edge termination](#opt-in--edge-termination-no-certificates). Which class a tenant may put itself on is itself bounded: `tenant.spec.gatewayClass` is a tenant-writable field, so the gateway chart accepts only the platform default plus the classes listed in `gateway.tenantSelectableClasses`, and fails the tenant's own release otherwise. In every other mode: the plain-HTTP listener (port 80) carries a strictly narrower selector — only the tenant namespace itself and `cozy-cert-manager`. Both the controller-owned http→https redirect HTTPRoute and cert-manager's HTTP-01 challenge route live in the tenant namespace, the latter because cert-manager creates the Challenge, and with it the solver route, in the Certificate's own namespace; the `cozy-cert-manager` entry is not exercised by that path — so app HTTPRoutes attaching by hostname cannot bind to port 80 and serve plaintext. Every port-443 listener, HTTPS-terminate and TLS-passthrough alike, carries the same `allowedRoutes.kinds` of `HTTPRoute` and `TLSRoute` (cilium#45559 collapses port-443 listeners whose kinds diverge), which keeps GRPCRoute / TCPRoute / UDPRoute from attaching there. GRPCRoute reaches the port-80 listener instead, where the route-hostname VAP covers it; TCPRoute and UDPRoute carry no hostname.
106+
1. **Namespace whitelist on listeners.** Every listener carries an `allowedRoutes.namespaces.from: Selector`, and there are two of them. The plain-HTTP port-80 listener selects on the built-in `kubernetes.io/metadata.name` label, which kube-apiserver writes and a namespace cannot forge. The HTTPS-terminate and TLS-passthrough listeners select on `namespace.cozystack.io/gateway`, which is an ordinary namespace label written by the `apps/tenant` chart and by the controller, so it carries no such guarantee on its own: what keeps a namespace from applying it and attaching is that `Namespace` is cluster-scoped and no tenant role grants a verb on it, and what such a namespace could publish if it did is bounded by the route-hostname policies of Layer 2. HTTPS / TLS-passthrough listeners accept routes from the publishing tenant's namespace plus `gateway.attachedNamespaces` in the platform chart, which the controller labels for exactly that purpose (the default list includes the `cozy-*` namespaces for platform services and `default` for the Kubernetes API TLSRoute). A namespace carrying neither the name match nor that label cannot attach any `HTTPRoute` or `TLSRoute` to those listeners. Under `certMode: edge` there are no HTTPS listeners to describe: the Gateway serves port 80 only, every listener carries the same broad selector, and the narrow port-80 rule stated further down this item does not apply — see [edge termination](#opt-in--edge-termination-no-certificates). Which class a tenant may put itself on is itself bounded: `tenant.spec.gatewayClass` is a tenant-writable field, so the gateway chart accepts only the platform default plus the classes listed in `gateway.tenantSelectableClasses`, and fails the tenant's own release otherwise. In every other mode: the plain-HTTP listener (port 80) carries a strictly narrower selector — only the tenant namespace itself and `cozy-cert-manager`. Both the controller-owned http→https redirect HTTPRoute and cert-manager's HTTP-01 challenge route live in the tenant namespace, the latter because cert-manager creates the Challenge, and with it the solver route, in the Certificate's own namespace; the `cozy-cert-manager` entry is not exercised by that path. That selector keeps routes from the other `cozy-*` namespaces and from inheriting child tenants off port 80; it does not keep off the publishing tenant's own app routes, which live in that same namespace and are admitted by it. Every port-443 listener, HTTPS-terminate and TLS-passthrough alike, carries the same `allowedRoutes.kinds` of `HTTPRoute` and `TLSRoute` (cilium#45559 collapses port-443 listeners whose kinds diverge), which keeps GRPCRoute / TCPRoute / UDPRoute from attaching there. GRPCRoute reaches the port-80 listener instead, where the route-hostname VAP covers it; TCPRoute and UDPRoute carry no hostname.
107107
2. **`cozystack-gateway-hostname-policy`** — `ValidatingAdmissionPolicy` on `gateway.networking.k8s.io/v1 Gateway` CREATE/UPDATE. Reads `namespaceObject.metadata.labels["namespace.cozystack.io/host"]` and rejects any listener hostname that is not equal to that value or a subdomain of it. `matchConditions` gate the VAP to cozystack-managed namespaces only — Gateways in unrelated namespaces (e.g. `kube-system`) are not touched.
108108
3. **`cozystack-gateway-attached-namespaces-policy`** — VAP on `cozystack.io/v1alpha1 Package` CREATE/UPDATE. Rejects any `tenant-*` entry in `spec.components.platform.values.gateway.attachedNamespaces`. Catches direct `kubectl edit packages.cozystack.io` that would bypass the helm render-time guard in layer 6.
109109
4. **`cozystack-tenant-host-policy`** — VAP on `apps.cozystack.io/v1alpha1 Tenant` CREATE/UPDATE. Rejects setting or changing `spec.host` unless the caller's groups contain `system:masters`, `system:serviceaccounts:cozy-system`, `system:serviceaccounts:cozy-cert-manager`, `system:serviceaccounts:cozy-fluxcd` or `system:serviceaccounts:kube-system`. Closes the path where a tenant user sets `spec.host=dashboard.example.org` on their own tenant to have the tenant chart write a hijacked label into the namespace.

‎packages/system/cozystack-basics/templates/route-hostname-policy.yaml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,14 @@
5454
comparable against the tenant apex at all — CEL has no cross-object
5555
lookup, so a VAP cannot resolve the parent Gateway to learn what would
5656
otherwise be inherited.
57+
58+
For two of the three kinds nothing else enforces presence: the
59+
httproutes and grpcroutes schemas leave spec.hostnames optional at
60+
every version the vendored bundle serves, so this policy is the only
61+
rejection a hostname-less route of those kinds meets. tlsroutes is
62+
mixed — v1 and v1alpha3 mark the field required with minItems: 1,
63+
v1alpha2 marks neither — so there too the policy is what closes the
64+
gap while v1alpha2 is served.
5765
*/}}
5866
{{- /*
5967
Both operands of the hostname comparison are normalized with

0 commit comments

Comments
 (0)