Skip to content

Commit e14554f

Browse files
committed
docs(gateway): align gateway prose with the code
Equivalent matching resolves a request against the versions a rule names, so a rule reaches versions it omits only while one named version is still served. The template comment asserted that coverage unconditionally and the gateway README repeated it. The condition is load-bearing for TLSRoute, whose rule names v1alpha2 alone while the shipped bundle already serves tlsroutes at v1 as storage and marks both alpha versions deprecated. A bundle that drops v1alpha2 leaves the rule matching nothing: requests at v1 meet no policy, and failurePolicy: Fail does not fire, because an absence of match is not a failure. State the condition, the version state it rests on, and what has to change before taking such a bundle. The comment on the port-80 allowedRoutes filter credited the cert-manager namespace entry for letting ACME complete, which the paragraph above it retracts: the solver route lives in the tenant namespace, and that is what keeps ACME working. The same sentence claimed the filter keeps app routes off port 80, and the Layer 1 paragraph of the gateway README claimed it too. It does not. A tenant that owns its Gateway is its own gateway namespace, so its app routes are admitted by this very filter; what it excludes is cozy-* and inheriting children. Both copies now say that. Nothing said why naming the apex on the redirect costs no coverage, which is the question a reader has once the route stops matching every host. It holds because of the other half of this change: the port-80 listener admits routes only from the tenant namespace and cert-manager's, and the tightened route policy requires every route there to declare hostnames inside the apex. An inheriting child does not widen that set either, and the comment now says so at the point the hostnames are chosen: the gateway hostname policy compares every listener against the host label of the Gateway's own namespace, so a child apex outside it never gets its listener, the Gateway write carrying it being refused at admission whole, in reconcileGateway and before this route is rendered, while a child apex under this one passes that check and the wildcard already covers it. The empty-apex guard in renderHTTPRedirect, and the test pinning it, both read as though the guard is what spares an operator an unreadable schema error. It is, but the condition stated for it named the number of attached routes, which does not enter into it. In HTTP-01 the listener set is built from route hostnames alone, so reconcileGateway never reads the apex ahead of the guard however many routes attach. DNS-01 and existingSecret render an apex listener, and any tlsPassthroughServices entry builds <service>.<apex>, so in those the operator meets the equivalent error against the Gateway first. State that condition once, at the guard, and let the test defer to it rather than carry a second copy that can drift; the test stays green either way because the fake client does not validate against the CRD schema. The e2e suite explained omitting the TLSRoute empty-list probe by the v1 schema rejecting an empty list. Both halves of that are true and the conclusion does not follow: the probe runs at v1alpha2, which marks spec.hostnames neither required nor minItems, so an empty list reaches admission there exactly as an absent field does. What makes the omission safe is that both policies render one byte-identical CEL string, so the HTTPRoute pair proves that term already. Assisted-By: Claude <[email protected]> Signed-off-by: Aleksei Sviridkin <[email protected]>
1 parent e31af89 commit e14554f

5 files changed

Lines changed: 81 additions & 25 deletions

File tree

‎hack/e2e-chainsaw/gateway/chainsaw-test.yaml‎

Lines changed: 11 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -509,12 +509,17 @@ spec:
509509
# pin that the terms are present in the expression — this step is what proves
510510
# the apiserver acts on them.
511511
#
512-
# HTTPRoute carries both cases; TLSRoute contributes only the absent one,
513-
# because at v1 the CRD schema requires spec.hostnames and rejects an empty
514-
# list on minItems, leaving the deprecated v1alpha2 the only version where a
515-
# hostname-less TLSRoute reaches admission at all. Scripted rather than an
516-
# apply/expect pair so the grep can prove the VAP did the denying: a bare
517-
# "the apply failed" assertion would be satisfied by a typo in the manifest.
512+
# HTTPRoute carries both cases. The TLSRoute probe is applied at v1alpha2
513+
# because v1 and v1alpha3 mark spec.hostnames required with minItems: 1,
514+
# leaving v1alpha2 the only version where a hostname-less TLSRoute reaches
515+
# admission at all. It carries the absent case only, and that is a coverage
516+
# choice rather than a reachability fact: v1alpha2 sets neither constraint,
517+
# so an explicitly empty list reaches admission there just as an absent
518+
# field does. Both policies render one byte-identical CEL string, so the
519+
# HTTPRoute pair above already proves the empty-list term end to end.
520+
# Scripted rather than an apply/expect pair so the grep can prove the VAP
521+
# did the denying: a bare "the apply failed" assertion would be satisfied
522+
# by a typo in the manifest.
518523
- name: hostname-less-route-denied
519524
try:
520525
- script:

‎internal/controller/tenantgateway/reconciler_test.go‎

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1141,15 +1141,24 @@ func TestReconcile_OtherControllerOwnedRouteStillClaimsHostnames(t *testing.T) {
11411141
// hostnames ["", "*."], and both fail the HTTPRoute CRD's minLength and
11421142
// pattern, so the create is rejected and the reconcile dies on an
11431143
// apiserver validation error that says nothing about which field is at
1144-
// fault. Before the redirect carried hostnames an empty apex reached
1145-
// Ready, so this is a narrowing this branch is responsible for closing.
1144+
// fault. Under HTTP-01 an empty apex reached Ready before the redirect
1145+
// carried hostnames, so that much is a narrowing this branch closes.
1146+
// DNS-01 and existingSecret never reached Ready on an empty apex, at
1147+
// merge base or now: renderGateway builds its listener hostnames off
1148+
// the same field and fails ahead of the redirect either way.
11461149
//
11471150
// MinLength=1 on the field is the real guard and it stops this at
11481151
// admission. This check is the second layer, for the window where the
11491152
// CRD in the cluster is older than the controller binary: the two are
11501153
// rolled out separately, so the controller cannot assume the field
11511154
// validation it needs is already present. It must fail on its own terms
11521155
// and name the field.
1156+
//
1157+
// Which layer fires first depends on the cert mode; renderHTTPRedirect
1158+
// states the condition at the guard itself. This case stays green
1159+
// either way because the fake client does not validate against the CRD
1160+
// schema, so read it as pinning the error text, not as evidence of
1161+
// which layer fires.
11531162
func TestReconcile_EmptyApexFailsWithNamedError(t *testing.T) {
11541163
s := newScheme(t)
11551164
tgw := &gatewayv1alpha1.TenantGateway{

‎internal/controller/tenantgateway/renderers.go‎

Lines changed: 33 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -138,14 +138,14 @@ func buildAllowedRoutes(tgw *gatewayv1alpha1.TenantGateway) *gatewayv1.AllowedRo
138138
// itself runs in (see acmeChallengeNamespace).
139139
//
140140
// Why: app HTTPRoutes (harbor, keycloak, dashboard, bucket) attach
141-
// by hostname with no sectionName, so without this narrower filter
142-
// they would also bind to the HTTP listener — Gateway API
143-
// tie-breaks merged routes by creationTimestamp, so an app route
144-
// created before the controller's redirect would silently serve
145-
// plaintext on port 80 and leak credentials. Restricting the HTTP
146-
// listener's allowedRoutes namespaces excludes the cozy-* / tenant-*
147-
// namespaces apps live in, while keeping cert-manager's challenge
148-
// namespace open so ACME still completes.
141+
// by hostname with no sectionName, so any of them the filter admits
142+
// reaches the HTTP listener and can serve plaintext there. The filter
143+
// keeps out the ones published from elsewhere; an app in the Gateway's
144+
// own namespace is admitted and is not covered by this. Restricting the HTTP
145+
// listener's allowedRoutes namespaces keeps out routes from the other
146+
// cozy-* namespaces and from inheriting child tenants. The Gateway's
147+
// own namespace stays open, because the redirect and the ACME solver
148+
// route both live there, and so does acmeChallengeNamespace.
149149
func buildHTTPListenerAllowedRoutes(tgw *gatewayv1alpha1.TenantGateway) *gatewayv1.AllowedRoutes {
150150
values := []string{tgw.Namespace}
151151
if acmeChallengeNamespace != tgw.Namespace {
@@ -470,6 +470,14 @@ func (r *Reconciler) renderHTTPRedirect(tgw *gatewayv1alpha1.TenantGateway) (*ga
470470
// HTTPRoute schema. That surfaces as an apiserver validation error
471471
// naming the route rather than the field, which is a poor thing to
472472
// hand an operator.
473+
//
474+
// This wins the race in HTTP-01 without tlsPassthroughServices: the
475+
// listener set is built from route hostnames alone there, so however
476+
// many routes are attached, reconcileGateway never reads the apex
477+
// ahead of this point. DNS-01 and existingSecret render an apex
478+
// listener, and any tlsPassthroughServices entry builds
479+
// "<service>.<apex>", so in those the operator meets the equivalent
480+
// schema error against the Gateway first.
473481
if tgw.Spec.Apex == "" {
474482
return nil, fmt.Errorf("spec.apex is empty on TenantGateway %s/%s: the http-to-https redirect route derives its hostnames from the apex and cannot be rendered without one", tgw.Namespace, tgw.Name)
475483
}
@@ -498,7 +506,23 @@ func (r *Reconciler) renderHTTPRedirect(tgw *gatewayv1alpha1.TenantGateway) (*ga
498506
// "*.<apex>" covers every subdomain at every depth, but it
499507
// does not cover the bare "<apex>", hence the plain apex
500508
// first.
501-
509+
//
510+
// Naming them costs no coverage, and the other half of
511+
// this change is why: the port-80 listener admits routes
512+
// only from this namespace and cert-manager's, and the
513+
// tightened cozystack-route-hostname-policy requires every
514+
// route here to declare hostnames inside the apex. So any
515+
// host published from this namespace is already one of
516+
// these two.
517+
//
518+
// An inheriting child cannot widen that set either.
519+
// cozystack-gateway-hostname-policy compares every listener
520+
// against the host label of the Gateway's own namespace, so
521+
// a child apex outside it never gets its listener: the
522+
// Gateway write carrying it is refused at admission whole,
523+
// in reconcileGateway and before this route is rendered. A
524+
// child apex under this one passes that check, and the
525+
// wildcard above already covers it.
502526
Hostnames: []gatewayv1.Hostname{
503527
gatewayv1.Hostname(tgw.Spec.Apex),
504528
gatewayv1.Hostname("*." + tgw.Spec.Apex),

‎packages/extra/gateway/README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -84,13 +84,13 @@ The protections below split into three groups by who they defend against:
8484
- **Defense-in-depth** — Layers 1, 2, 5, 6, 7, 8. These do not protect against tenant-user input (tenants don't hold the relevant RBAC). They guard against bugs in cozystack-controller / Flux, supply-chain compromise of an app chart that emits Gateway API or Ingress resources, and confused-deputy mistakes by a cluster admin. Fail-closed via `failurePolicy: Fail` + `validationActions: [Deny]`. Layers 1-7 cover the opt-in Gateway API dataplane; Layer 8 covers the legacy Ingress dataplane, which is the default (Gateway API is off by default).
8585
- **Admin-against-themselves** — Layer 3 (`cozystack-gateway-attached-namespaces-policy`). Rejects a `kubectl edit packages.cozystack.io` that would slip a `tenant-*` entry into the platform Package's `gateway.attachedNamespaces`. Layer 6 catches the same misconfiguration at helm render time.
8686

87-
1. **Namespace whitelist on listeners.** Every listener carries an `allowedRoutes.namespaces.from: Selector` matching the built-in `kubernetes.io/metadata.name` label (written by kube-apiserver, unspoofable). HTTPS / TLS-passthrough listeners accept routes from the publishing tenant's namespace plus `gateway.attachedNamespaces` in the platform chart (default includes the `cozy-*` namespaces for platform services and `default` for the Kubernetes API TLSRoute). A namespace outside the list literally cannot attach any `HTTPRoute` or `TLSRoute` to those listeners. The plain-HTTP listener (port 80) carries a strictly narrower selector — only the tenant namespace itself and `cozy-cert-manager`. Both the controller-owned http→https redirect HTTPRoute and cert-manager's HTTP-01 challenge route live in the tenant namespace, the latter because cert-manager creates the Challenge, and with it the solver route, in the Certificate's own namespace; the `cozy-cert-manager` entry is not exercised by that path — so app HTTPRoutes attaching by hostname cannot bind to port 80 and serve plaintext. HTTPS listeners additionally restrict `allowedRoutes.kinds` to `HTTPRoute` (and TLS-passthrough listeners to `TLSRoute`), preventing GRPCRoute / TCPRoute / UDPRoute from attaching outside the route-hostname VAP's coverage.
87+
1. **Namespace whitelist on listeners.** Every listener carries an `allowedRoutes.namespaces.from: Selector` matching the built-in `kubernetes.io/metadata.name` label (written by kube-apiserver, unspoofable). HTTPS / TLS-passthrough listeners accept routes from the publishing tenant's namespace plus `gateway.attachedNamespaces` in the platform chart (default includes the `cozy-*` namespaces for platform services and `default` for the Kubernetes API TLSRoute). A namespace outside the list literally cannot attach any `HTTPRoute` or `TLSRoute` to those listeners. The plain-HTTP listener (port 80) carries a strictly narrower selector — only the tenant namespace itself and `cozy-cert-manager`. Both the controller-owned http→https redirect HTTPRoute and cert-manager's HTTP-01 challenge route live in the tenant namespace, the latter because cert-manager creates the Challenge, and with it the solver route, in the Certificate's own namespace; the `cozy-cert-manager` entry is not exercised by that path. That selector keeps routes from the other `cozy-*` namespaces and from inheriting child tenants off port 80; it does not keep off the publishing tenant's own app routes, which live in that same namespace and are admitted by it. HTTPS listeners additionally restrict `allowedRoutes.kinds` to `HTTPRoute` (and TLS-passthrough listeners to `TLSRoute`), preventing GRPCRoute / TCPRoute / UDPRoute from attaching outside the route-hostname VAP's coverage.
8888
2. **`cozystack-gateway-hostname-policy`** — `ValidatingAdmissionPolicy` on `gateway.networking.k8s.io/v1 Gateway` CREATE/UPDATE. Reads `namespaceObject.metadata.labels["namespace.cozystack.io/host"]` and rejects any listener hostname that is not equal to that value or a subdomain of it. `matchConditions` gate the VAP to cozystack-managed namespaces only — Gateways in unrelated namespaces (e.g. `kube-system`) are not touched.
8989
3. **`cozystack-gateway-attached-namespaces-policy`** — VAP on `cozystack.io/v1alpha1 Package` CREATE/UPDATE. Rejects any `tenant-*` entry in `spec.components.platform.values.gateway.attachedNamespaces`. Catches direct `kubectl edit packages.cozystack.io` that would bypass the helm render-time guard in layer 6.
9090
4. **`cozystack-tenant-host-policy`** — VAP on `apps.cozystack.io/v1alpha1 Tenant` CREATE/UPDATE. Rejects setting or changing `spec.host` unless the caller's groups contain `system:masters`, `system:serviceaccounts:cozy-system`, `system:serviceaccounts:cozy-cert-manager`, `system:serviceaccounts:cozy-fluxcd` or `system:serviceaccounts:kube-system`. Closes the path where a tenant user sets `spec.host=dashboard.example.org` on their own tenant to have the tenant chart write a hijacked label into the namespace.
9191
5. **`cozystack-namespace-host-label-policy`** — VAP on core `v1 Namespace` CREATE/UPDATE. Rejects any set or change of the `namespace.cozystack.io/host` label, except by the same trusted-caller whitelist as layer 4. This closes both first-time label writes on CREATE and first-time adds on UPDATE — only cozystack/Flux service accounts (which apply the tenant chart) can stamp the label.
9292
6. **Render-time `fail` in cozystack-basics.** The cozystack-basics chart fails the helm render if `_cluster.gateway-attached-namespaces` contains any `tenant-*` entry. Triggers on the helm-install path before the cluster ever sees the values — complements layer 3 which triggers at `kubectl apply` time.
93-
7. **`cozystack-route-hostname-policy`** — VAP on `gateway.networking.k8s.io` HTTPRoute and TLSRoute CREATE/UPDATE. The rules name `v1`/`v1beta1` for HTTPRoute and `v1alpha2` for TLSRoute, which covers every served version of both: `matchConstraints.matchPolicy` defaults to `Equivalent`, so the apiserver converts a request made at another version into the listed one before the policy sees it. Scoped to `tenant-*` namespaces (cozy-* are cluster-admin-managed and trusted to publish under any apex). Requires `spec.hostnames` to be present and non-empty, and rejects any entry that is not equal to the namespace's `namespace.cozystack.io/host` label or a subdomain of it. A route that names no hostname is not inert: it inherits the hostnames of every listener it attaches to, and `parentRefs` may name a Gateway in another namespace, so admitting it unchecked would let it cover whatever that listener covers. A VAP cannot resolve the parent Gateway to learn which hostnames would be inherited, so the policy requires the route to name its own. The controller-owned http-to-https redirect names the tenant apex and its wildcard for this reason. Defense-in-depth against an app chart bug or supply-chain compromise that emits Gateway API resources outside the tenant's apex — tenants in Cozystack do not hold `gateway.networking.k8s.io/*` RBAC by design, so this is not a tenant-user defense. The within-apex cross-namespace case (a tenant chart claiming a hostname that is published by a `cozy-*` app) is handled by the controller at reconciliation time: when two routes from different namespaces claim the same hostname, the `cozy-*` namespace wins and the loser receives a `HostnameConflict` condition under the controller's name in `Status.Parents`.
93+
7. **`cozystack-route-hostname-policy`** — VAP on `gateway.networking.k8s.io` HTTPRoute and TLSRoute CREATE/UPDATE. The rules name `v1`/`v1beta1` for HTTPRoute and `v1alpha2` for TLSRoute, which reaches the versions they do not name only while a named version is still served: `matchConstraints.matchPolicy` defaults to `Equivalent`, so the apiserver converts a request made at another version into a listed one before the policy sees it. That condition is load-bearing for TLSRoute, whose rule names only `v1alpha2` while the shipped bundle already serves `v1` as storage and marks both alpha versions deprecated, so the rule needs `v1` before a bundle drops `v1alpha2`. Scoped to `tenant-*` namespaces (cozy-* are cluster-admin-managed and trusted to publish under any apex). Requires `spec.hostnames` to be present and non-empty, and rejects any entry that is not equal to the namespace's `namespace.cozystack.io/host` label or a subdomain of it. A route that names no hostname is not inert: it inherits the hostnames of every listener it attaches to, and `parentRefs` may name a Gateway in another namespace, so admitting it unchecked would let it cover whatever that listener covers. A VAP cannot resolve the parent Gateway to learn which hostnames would be inherited, so the policy requires the route to name its own. The controller-owned http-to-https redirect names the tenant apex and its wildcard for this reason. Defense-in-depth against an app chart bug or supply-chain compromise that emits Gateway API resources outside the tenant's apex — tenants in Cozystack do not hold `gateway.networking.k8s.io/*` RBAC by design, so this is not a tenant-user defense. The within-apex cross-namespace case (a tenant chart claiming a hostname that is published by a `cozy-*` app) is handled by the controller at reconciliation time: when two routes from different namespaces claim the same hostname, the `cozy-*` namespace wins and the loser receives a `HostnameConflict` condition under the controller's name in `Status.Parents`.
9494
8. **`cozystack-ingress-hostname-policy`** — VAP on core `networking.k8s.io/v1 Ingress` CREATE/UPDATE. Gateway API is opt-in and **off by default**, so in the default configuration tenant applications publish through a legacy Ingress on the shared ingress-nginx; layers 1-7 constrain tenant hostnames on the opt-in Gateway path, and this VAP adds a hostname constraint on the default Ingress path. Scoped to `tenant-*` namespaces (cozy-* are cluster-admin-managed and trusted). A hostname on `spec.rules[].host` or `spec.tls[].hosts[]` is allowed when it is within the namespace's own `namespace.cozystack.io/host` apex, OR when it lies entirely outside the platform root apex (`_cluster.root-host`) — the second case lets a tenant route its own external custom domain (for example the `kubernetes` app's Proxied `addons.ingressNginx.hosts`, which routes a user-supplied domain to a nested cluster). A hostname that falls under the platform root apex but outside the namespace's own apex is rejected; a rule with no host (an unbounded catch-all) and `spec.defaultBackend` (a catch-all for unmatched traffic) are also rejected. Fail-closed: a `tenant-*` namespace missing its host label is denied, and the policy renders only when `_cluster.root-host` is set. This bounds which apexes a tenant Ingress may claim under the platform domain; it does not attempt to resolve every possible hostname collision.
9595

9696
For `tenant-root` the allowed host suffix is `publishing.host`; for any `tenant-<name>` that inherits from its parent the suffix is `<name>.<parent apex>`. A child tenant with an independent apex (`customer1.io` instead of a subdomain) is handled correctly because the VAP reads the per-namespace label rather than assuming a subdomain hierarchy.

‎packages/system/cozystack-basics/templates/route-hostname-policy.yaml‎

Lines changed: 24 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -17,12 +17,30 @@
1717
- VAP + Binding for HTTPRoute, rule naming v1 and v1beta1
1818
- VAP + Binding for TLSRoute, rule naming v1alpha2
1919

20-
A rule covers every served version of its resource, not only the ones
21-
it names, because matchConstraints.matchPolicy defaults to Equivalent
22-
and the apiserver converts a request into a listed version before the
23-
policy sees it. Do not set matchPolicy: Exact here. It reads like a
24-
tightening and is the opposite: the policy would stop seeing every
25-
version the rule does not name.
20+
A rule reaches versions it does not name, but only while one version
21+
it does name is still served: matchConstraints.matchPolicy defaults
22+
to Equivalent, and the apiserver converts a request into a listed
23+
version before the policy sees it. Do not set matchPolicy: Exact
24+
here. It reads like a tightening and is the opposite: the policy
25+
would stop seeing every version the rule does not name.
26+
27+
That condition is load-bearing for TLSRoute, whose rule names only
28+
v1alpha2. The bundle in packages/system/gateway-api-crds serves
29+
tlsroutes at v1 (storage), v1alpha2 and v1alpha3, and marks both
30+
alpha versions deprecated, so the single version this rule names is
31+
the one scheduled to go away. A bundle that drops v1alpha2 leaves the
32+
rule matching nothing: requests at v1 meet no policy at all, and
33+
failurePolicy: Fail does not fire, because there is no failure —
34+
only an absence of match. Give the TLSRoute rule v1 before taking a
35+
bundle that drops v1alpha2.
36+
37+
What that would cost is narrower than it first reads, and narrow in
38+
the direction that matters. The v1 and v1alpha3 schemas mark
39+
spec.hostnames required with minItems: 1, so an empty or absent list
40+
is still rejected without this policy — on v1alpha2, which has
41+
neither, this policy is the only thing rejecting it. What no schema
42+
replaces is the apex comparison: without a matching rule, a TLSRoute
43+
naming someone else's apex is admitted.
2644
*/}}
2745
{{- /*
2846
CEL safety note: has() only works on field access expressions, not

0 commit comments

Comments
 (0)