Skip to content

Commit 02a956f

Browse files
Add Flower HTTPRoute support to Helm chart (apache#69945)
* Add Flower HTTPRoute support to Helm chart * Update chart/tests/helm_tests/other/test_httproute_flower.py Co-authored-by: Przemysław Mirowski <[email protected]> * Clarify Flower HTTPRoute configuration guidance * Update chart/docs/production-guide.rst Co-authored-by: Przemysław Mirowski <[email protected]> * Simplify HTTPRoute configuration guidance --------- Co-authored-by: Przemysław Mirowski <[email protected]>
1 parent ec8da5c commit 02a956f

7 files changed

Lines changed: 694 additions & 20 deletions

File tree

‎chart/docs/production-guide.rst‎

Lines changed: 27 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -393,14 +393,16 @@ For more information on ``Ingress``, see the
393393
Gateway API (HTTPRoute)
394394
^^^^^^^^^^^^^^^^^^^^^^^
395395

396-
As an alternative to ``Ingress``, the chart can create a
397-
`Kubernetes Gateway API <https://gateway-api.sigs.k8s.io/>`_ ``HTTPRoute`` for the API server.
396+
As an alternative to ``Ingress``, the chart can create
397+
`Kubernetes Gateway API <https://gateway-api.sigs.k8s.io/>`_ ``HTTPRoute`` resources for the API server and Flower.
398398
This requires the Gateway API CRDs to be installed in the cluster and a ``Gateway`` to already exist —
399-
the chart only creates the ``HTTPRoute`` and attaches it to the Gateway via ``parentRefs``.
399+
the chart only creates the ``HTTPRoute`` resources and attaches them to the Gateway via ``parentRefs``.
400400

401401
.. code-block:: yaml
402402
:caption: values.yaml
403403
404+
executor: CeleryExecutor
405+
404406
apiServer:
405407
httpRoute:
406408
enabled: true
@@ -411,16 +413,31 @@ the chart only creates the ``HTTPRoute`` and attaches it to the Gateway via ``pa
411413
hostnames:
412414
- airflow.example.com
413415
414-
For fine-grained routing, supply ``apiServer.httpRoute.rules`` directly — the entry mirrors the
415-
upstream ``HTTPRouteRule`` schema and overrides the default rule generated from ``path`` + ``pathType``.
416+
flower:
417+
enabled: true
418+
httpRoute:
419+
enabled: true
420+
parentRefs:
421+
- name: main-gateway
422+
namespace: gateway-system
423+
sectionName: https
424+
hostnames:
425+
- flower.example.com
426+
427+
Flower HTTPRoute resources are only created when Flower itself is created, so ``flower.enabled`` must be
428+
``true`` and the executor must include ``CeleryExecutor``.
429+
430+
For fine-grained routing, supply ``apiServer.httpRoute.rules`` or ``flower.httpRoute.rules`` directly —
431+
the entry mirrors the upstream ``HTTPRouteRule`` schema and overrides the default rule generated from
432+
the corresponding ``apiServer.httpRoute.path`` and ``apiServer.httpRoute.pathType``, or
433+
``flower.httpRoute.path`` and ``flower.httpRoute.pathType`` values.
416434

417435
.. note::
418436

419-
``HTTPRoute`` is an alternative to the API server ``Ingress``, so enable only one of
420-
``ingress.apiServer`` or ``apiServer.httpRoute`` — enabling both at the same time fails template
421-
rendering. When ``apiServer.httpRoute.enabled`` is ``true``, the chart also verifies (via Helm
422-
``Capabilities``) that the Gateway API CRDs are installed and fails with a clear message if they
423-
are not.
437+
``HTTPRoute`` is an alternative to ``Ingress`` for the same component, so enable only one routing
438+
mechanism for each component. Enabling both for the same component fails template rendering.
439+
When an ``HTTPRoute`` is enabled, the chart also verifies (via Helm ``Capabilities``) that the Gateway
440+
API CRDs are installed and fails with a clear message if they are not.
424441

425442
LoadBalancer Service
426443
^^^^^^^^^^^^^^^^^^^^
Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
{{/*
2+
Licensed to the Apache Software Foundation (ASF) under one
3+
or more contributor license agreements. See the NOTICE file
4+
distributed with this work for additional information
5+
regarding copyright ownership. The ASF licenses this file
6+
to you under the Apache License, Version 2.0 (the
7+
"License"); you may not use this file except in compliance
8+
with the License. You may obtain a copy of the License at
9+
10+
http://www.apache.org/licenses/LICENSE-2.0
11+
12+
Unless required by applicable law or agreed to in writing,
13+
software distributed under the License is distributed on an
14+
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
KIND, either express or implied. See the License for the
16+
specific language governing permissions and limitations
17+
under the License.
18+
*/}}
19+
20+
###########################
21+
## Airflow Flower HTTPRoute
22+
###########################
23+
{{- if and .Values.flower.enabled .Values.flower.httpRoute.enabled (contains "CeleryExecutor" .Values.executor) }}
24+
{{- if .Values.ingress.flower.enabled }}
25+
{{- fail "`flower.httpRoute.enabled` and `ingress.flower.enabled` are both set to `true`. HTTPRoute (Gateway API) is an alternative to the Flower Ingress; enable only one of them." }}
26+
{{- end }}
27+
{{- if not (.Capabilities.APIVersions.Has "gateway.networking.k8s.io/v1") }}
28+
{{- fail "`flower.httpRoute.enabled` is `true` but the Gateway API HTTPRoute CRD (`gateway.networking.k8s.io/v1`) is not installed in the cluster. Install the Gateway API CRDs (https://gateway-api.sigs.k8s.io/guides/#installing-gateway-api) before enabling `flower.httpRoute`." }}
29+
{{- end }}
30+
{{- if not .Values.flower.httpRoute.parentRefs }}
31+
{{- fail "`flower.httpRoute.enabled` is `true` but `flower.httpRoute.parentRefs` is empty. An HTTPRoute must reference at least one parent Gateway; set `flower.httpRoute.parentRefs`." }}
32+
{{- end }}
33+
{{- $fullname := include "airflow.fullname" . }}
34+
apiVersion: gateway.networking.k8s.io/v1
35+
kind: HTTPRoute
36+
metadata:
37+
name: {{ $fullname }}-flower-httproute
38+
labels:
39+
tier: airflow
40+
component: flower-httproute
41+
release: {{ .Release.Name }}
42+
chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
43+
heritage: {{ .Release.Service }}
44+
{{- if or .Values.labels .Values.flower.labels .Values.flower.httpRoute.labels }}
45+
{{- mustMerge .Values.flower.httpRoute.labels .Values.flower.labels .Values.labels | toYaml | nindent 4 }}
46+
{{- end }}
47+
{{- with .Values.flower.httpRoute.annotations }}
48+
annotations: {{- toYaml . | nindent 4 }}
49+
{{- end }}
50+
spec:
51+
parentRefs: {{- toYaml .Values.flower.httpRoute.parentRefs | nindent 4 }}
52+
{{- with .Values.flower.httpRoute.hostnames }}
53+
hostnames:
54+
{{- range . }}
55+
- {{ tpl . $ | quote }}
56+
{{- end }}
57+
{{- end }}
58+
rules:
59+
{{- if .Values.flower.httpRoute.rules }}
60+
{{- toYaml .Values.flower.httpRoute.rules | nindent 4 }}
61+
{{- else }}
62+
- matches:
63+
- path:
64+
type: {{ .Values.flower.httpRoute.pathType }}
65+
value: {{ .Values.flower.httpRoute.path | quote }}
66+
backendRefs:
67+
- name: {{ $fullname }}-flower
68+
port: {{ .Values.ports.flowerUI }}
69+
{{- end }}
70+
{{- end }}

‎chart/templates/flower/flower-ingress.yaml‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,9 @@
2222
#########################
2323
{{- if .Values.flower.enabled }}
2424
{{- if and .Values.ingress.flower.enabled (contains "CeleryExecutor" .Values.executor) }}
25+
{{- if .Values.flower.httpRoute.enabled }}
26+
{{- fail "`ingress.flower.enabled` and `flower.httpRoute.enabled` are both set to `true`. The Flower Ingress is an alternative to HTTPRoute (Gateway API); enable only one of them." }}
27+
{{- end }}
2528
{{- $fullname := include "airflow.fullname" . }}
2629
apiVersion: networking.k8s.io/v1
2730
kind: Ingress

‎chart/tests/helm_tests/airflow_aux/test_basic_helm_chart.py‎

Lines changed: 39 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -207,7 +207,36 @@ def test_network_policies_are_valid(self, executor):
207207
"CeleryExecutor,harvest_exec:KubernetesExecutor",
208208
],
209209
)
210-
def test_labels_are_valid(self, executor):
210+
@pytest.mark.parametrize(
211+
("flower_routing_values", "flower_routing_resource", "api_versions"),
212+
[
213+
pytest.param(
214+
{
215+
"ingress": {"flower": {"enabled": True}, "apiServer": {"enabled": True}},
216+
"flower": {"enabled": True},
217+
},
218+
("flower-ingress", "Ingress", "flower-ingress"),
219+
[],
220+
id="flower-ingress",
221+
),
222+
pytest.param(
223+
{
224+
"ingress": {"apiServer": {"enabled": True}},
225+
"flower": {
226+
"enabled": True,
227+
"httpRoute": {
228+
"enabled": True,
229+
"parentRefs": [{"name": "main-gateway"}],
230+
},
231+
},
232+
},
233+
("flower-httproute", "HTTPRoute", "flower-httproute"),
234+
["gateway.networking.k8s.io/v1"],
235+
id="flower-httproute",
236+
),
237+
],
238+
)
239+
def test_labels_are_valid(self, executor, flower_routing_values, flower_routing_resource, api_versions):
211240
"""Test labels are correctly applied on all objects created by this chart."""
212241
release_name = "test-basic"
213242

@@ -227,25 +256,26 @@ def test_labels_are_valid(self, executor):
227256
},
228257
"pgbouncer": {"enabled": True},
229258
"redis": {"enabled": True},
230-
"ingress": {"flower": {"enabled": True}, "apiServer": {"enabled": True}},
231259
"networkPolicies": {"enabled": True},
232260
"cleanup": {"enabled": True},
233261
"databaseCleanup": {"enabled": True},
234-
"flower": {"enabled": True},
235262
"logs": {"persistence": {"enabled": True}},
236263
"dags": {"persistence": {"enabled": True}},
237264
"postgresql": {"enabled": False}, # We won't check the objects created by the postgres chart
238265
"priorityClasses": [
239266
{"name": "class1", "value": 10000},
240267
],
241268
}
269+
values.update(flower_routing_values)
242270

243-
k8s_objects = render_chart(name=release_name, values=values)
271+
k8s_objects = render_chart(name=release_name, values=values, api_versions=api_versions)
244272
kind_k8s_obj_labels_tuples = {
245273
(k8s_object["metadata"]["name"], k8s_object["kind"]): k8s_object["metadata"]["labels"]
246274
for k8s_object in k8s_objects
247275
}
248276

277+
flower_routing_name, flower_routing_kind, flower_routing_component = flower_routing_resource
278+
249279
kind_names_tuples = [
250280
(f"{release_name}-airflow-cleanup", "ServiceAccount", "airflow-cleanup-pods"),
251281
(f"{release_name}-airflow-database-cleanup", "ServiceAccount", "database-cleanup"),
@@ -274,7 +304,11 @@ def test_labels_are_valid(self, executor):
274304
(f"{release_name}-flower", "Deployment", "flower"),
275305
(f"{release_name}-flower", "Service", "flower"),
276306
(f"{release_name}-flower-policy", "NetworkPolicy", "airflow-flower-policy"),
277-
(f"{release_name}-flower-ingress", "Ingress", "flower-ingress"),
307+
(
308+
f"{release_name}-{flower_routing_name}",
309+
flower_routing_kind,
310+
flower_routing_component,
311+
),
278312
(f"{release_name}-pgbouncer", "Deployment", "pgbouncer"),
279313
(f"{release_name}-pgbouncer", "Service", "pgbouncer"),
280314
(f"{release_name}-pgbouncer-config", "Secret", "pgbouncer"),

0 commit comments

Comments
 (0)