Skip to content

Commit 3191d49

Browse files
zirainzirain
authored
e2e: use grafana alloy instead of fluent-bit (#4525)
* use grafana alloy instead of fluent-bit Signed-off-by: zirain <[email protected]> * make alloy disabled by default Signed-off-by: zirain <[email protected]> * enable alloy in e2e Signed-off-by: zirain <[email protected]> --------- Signed-off-by: zirain <[email protected]>
1 parent 2d68e6a commit 3191d49

File tree

10 files changed

+405
-149
lines changed

10 files changed

+405
-149
lines changed

‎charts/gateway-addons-helm/Chart.lock‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,9 @@ dependencies:
88
- name: fluent-bit
99
repository: https://fluent.github.io/helm-charts
1010
version: 0.30.4
11+
- name: alloy
12+
repository: https://grafana.github.io/helm-charts
13+
version: 0.9.2
1114
- name: loki
1215
repository: https://grafana.github.io/helm-charts
1316
version: 4.8.0
@@ -17,5 +20,5 @@ dependencies:
1720
- name: opentelemetry-collector
1821
repository: https://open-telemetry.github.io/opentelemetry-helm-charts
1922
version: 0.108.0
20-
digest: sha256:ea6663bb1358123b96b69d2c5b0b8c20650a43dc39b24c482f0560201fd2cc3a
21-
generated: "2024-10-19T12:59:47.251089661+02:00"
23+
digest: sha256:bc634c59972bfd4a01e0f4310a4949095752e659a9b5cb1d9c0fbe9a86f37011
24+
generated: "2024-10-25T10:55:26.755739+08:00"

‎charts/gateway-addons-helm/Chart.yaml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,10 @@ dependencies:
3737
repository: https://fluent.github.io/helm-charts
3838
version: 0.30.4
3939
condition: fluent-bit.enabled
40+
- name: alloy
41+
repository: https://grafana.github.io/helm-charts
42+
version: 0.9.2
43+
condition: alloy.enabled
4044
- name: loki
4145
version: 4.8.0
4246
repository: https://grafana.github.io/helm-charts

‎charts/gateway-addons-helm/README.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ An Add-ons Helm chart for Envoy Gateway
2222
| Repository | Name | Version |
2323
|------------|------|---------|
2424
| https://fluent.github.io/helm-charts | fluent-bit | 0.30.4 |
25+
| https://grafana.github.io/helm-charts | alloy | 0.9.2 |
2526
| https://grafana.github.io/helm-charts | grafana | 8.0.0 |
2627
| https://grafana.github.io/helm-charts | loki | 4.8.0 |
2728
| https://grafana.github.io/helm-charts | tempo | 1.3.1 |
@@ -55,6 +56,9 @@ To uninstall the chart:
5556

5657
| Key | Type | Default | Description |
5758
|-----|------|---------|-------------|
59+
| alloy.alloy.configMap.content | string | `"// Write your Alloy config here:\nlogging {\n level = \"info\"\n format = \"logfmt\"\n}\nloki.write \"alloy\" {\n endpoint {\n url = \"http://loki.monitoring.svc:3100/loki/api/v1/push\"\n }\n}\n// discovery.kubernetes allows you to find scrape targets from Kubernetes resources.\n// It watches cluster state and ensures targets are continually synced with what is currently running in your cluster.\ndiscovery.kubernetes \"pod\" {\n role = \"pod\"\n}\n\n// discovery.relabel rewrites the label set of the input targets by applying one or more relabeling rules.\n// If no rules are defined, then the input targets are exported as-is.\ndiscovery.relabel \"pod_logs\" {\n targets = discovery.kubernetes.pod.targets\n\n // Label creation - \"namespace\" field from \"__meta_kubernetes_namespace\"\n rule {\n source_labels = [\"__meta_kubernetes_namespace\"]\n action = \"replace\"\n target_label = \"namespace\"\n }\n\n // Label creation - \"pod\" field from \"__meta_kubernetes_pod_name\"\n rule {\n source_labels = [\"__meta_kubernetes_pod_name\"]\n action = \"replace\"\n target_label = \"pod\"\n }\n\n // Label creation - \"container\" field from \"__meta_kubernetes_pod_container_name\"\n rule {\n source_labels = [\"__meta_kubernetes_pod_container_name\"]\n action = \"replace\"\n target_label = \"container\"\n }\n\n // Label creation - \"app\" field from \"__meta_kubernetes_pod_label_app_kubernetes_io_name\"\n rule {\n source_labels = [\"__meta_kubernetes_pod_label_app_kubernetes_io_name\"]\n action = \"replace\"\n target_label = \"app\"\n }\n\n // Label creation - \"job\" field from \"__meta_kubernetes_namespace\" and \"__meta_kubernetes_pod_container_name\"\n // Concatenate values __meta_kubernetes_namespace/__meta_kubernetes_pod_container_name\n rule {\n source_labels = [\"__meta_kubernetes_namespace\", \"__meta_kubernetes_pod_container_name\"]\n action = \"replace\"\n target_label = \"job\"\n separator = \"/\"\n replacement = \"$1\"\n }\n\n // Label creation - \"container\" field from \"__meta_kubernetes_pod_uid\" and \"__meta_kubernetes_pod_container_name\"\n // Concatenate values __meta_kubernetes_pod_uid/__meta_kubernetes_pod_container_name.log\n rule {\n source_labels = [\"__meta_kubernetes_pod_uid\", \"__meta_kubernetes_pod_container_name\"]\n action = \"replace\"\n target_label = \"__path__\"\n separator = \"/\"\n replacement = \"/var/log/pods/*$1/*.log\"\n }\n\n // Label creation - \"container_runtime\" field from \"__meta_kubernetes_pod_container_id\"\n rule {\n source_labels = [\"__meta_kubernetes_pod_container_id\"]\n action = \"replace\"\n target_label = \"container_runtime\"\n regex = \"^(\\\\S+):\\\\/\\\\/.+$\"\n replacement = \"$1\"\n }\n}\n\n// loki.source.kubernetes tails logs from Kubernetes containers using the Kubernetes API.\nloki.source.kubernetes \"pod_logs\" {\n targets = discovery.relabel.pod_logs.output\n forward_to = [loki.process.pod_logs.receiver]\n}\n// loki.process receives log entries from other Loki components, applies one or more processing stages,\n// and forwards the results to the list of receivers in the component’s arguments.\nloki.process \"pod_logs\" {\n stage.static_labels {\n values = {\n cluster = \"envoy-gateway\",\n }\n }\n\n forward_to = [loki.write.alloy.receiver]\n}"` | |
60+
| alloy.enabled | bool | `false` | |
61+
| alloy.fullnameOverride | string | `"alloy"` | |
5862
| fluent-bit.config.filters | string | `"[FILTER]\n Name kubernetes\n Match kube.*\n Merge_Log On\n Keep_Log Off\n K8S-Logging.Parser On\n K8S-Logging.Exclude On\n\n[FILTER]\n Name grep\n Match kube.*\n Regex $kubernetes['container_name'] ^envoy$\n\n[FILTER]\n Name parser\n Match kube.*\n Key_Name log\n Parser envoy\n Reserve_Data True\n"` | |
5963
| fluent-bit.config.inputs | string | `"[INPUT]\n Name tail\n Path /var/log/containers/*.log\n multiline.parser docker, cri\n Tag kube.*\n Mem_Buf_Limit 5MB\n Skip_Long_Lines On\n"` | |
6064
| fluent-bit.config.outputs | string | `"[OUTPUT]\n Name loki\n Match kube.*\n Host loki.monitoring.svc.cluster.local\n Port 3100\n Labels job=fluentbit, app=$kubernetes['labels']['app'], k8s_namespace_name=$kubernetes['namespace_name'], k8s_pod_name=$kubernetes['pod_name'], k8s_container_name=$kubernetes['container_name']\n"` | |

‎charts/gateway-addons-helm/values.yaml‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,7 @@ prometheus:
6060

6161

6262
# Values for Fluent-bit dependency
63+
# TODO: remove fluent-bit dependency
6364
fluent-bit:
6465
enabled: true
6566
image:
@@ -167,6 +168,109 @@ loki:
167168
gateway:
168169
enabled: false
169170

171+
# Values for Alloy dependency
172+
alloy:
173+
enabled: false
174+
fullnameOverride: alloy
175+
alloy:
176+
configMap:
177+
content: |-
178+
// Write your Alloy config here:
179+
logging {
180+
level = "info"
181+
format = "logfmt"
182+
}
183+
loki.write "alloy" {
184+
endpoint {
185+
url = "http://loki.monitoring.svc:3100/loki/api/v1/push"
186+
}
187+
}
188+
// discovery.kubernetes allows you to find scrape targets from Kubernetes resources.
189+
// It watches cluster state and ensures targets are continually synced with what is currently running in your cluster.
190+
discovery.kubernetes "pod" {
191+
role = "pod"
192+
}
193+
194+
// discovery.relabel rewrites the label set of the input targets by applying one or more relabeling rules.
195+
// If no rules are defined, then the input targets are exported as-is.
196+
discovery.relabel "pod_logs" {
197+
targets = discovery.kubernetes.pod.targets
198+
199+
// Label creation - "namespace" field from "__meta_kubernetes_namespace"
200+
rule {
201+
source_labels = ["__meta_kubernetes_namespace"]
202+
action = "replace"
203+
target_label = "namespace"
204+
}
205+
206+
// Label creation - "pod" field from "__meta_kubernetes_pod_name"
207+
rule {
208+
source_labels = ["__meta_kubernetes_pod_name"]
209+
action = "replace"
210+
target_label = "pod"
211+
}
212+
213+
// Label creation - "container" field from "__meta_kubernetes_pod_container_name"
214+
rule {
215+
source_labels = ["__meta_kubernetes_pod_container_name"]
216+
action = "replace"
217+
target_label = "container"
218+
}
219+
220+
// Label creation - "app" field from "__meta_kubernetes_pod_label_app_kubernetes_io_name"
221+
rule {
222+
source_labels = ["__meta_kubernetes_pod_label_app_kubernetes_io_name"]
223+
action = "replace"
224+
target_label = "app"
225+
}
226+
227+
// Label creation - "job" field from "__meta_kubernetes_namespace" and "__meta_kubernetes_pod_container_name"
228+
// Concatenate values __meta_kubernetes_namespace/__meta_kubernetes_pod_container_name
229+
rule {
230+
source_labels = ["__meta_kubernetes_namespace", "__meta_kubernetes_pod_container_name"]
231+
action = "replace"
232+
target_label = "job"
233+
separator = "/"
234+
replacement = "$1"
235+
}
236+
237+
// Label creation - "container" field from "__meta_kubernetes_pod_uid" and "__meta_kubernetes_pod_container_name"
238+
// Concatenate values __meta_kubernetes_pod_uid/__meta_kubernetes_pod_container_name.log
239+
rule {
240+
source_labels = ["__meta_kubernetes_pod_uid", "__meta_kubernetes_pod_container_name"]
241+
action = "replace"
242+
target_label = "__path__"
243+
separator = "/"
244+
replacement = "/var/log/pods/*$1/*.log"
245+
}
246+
247+
// Label creation - "container_runtime" field from "__meta_kubernetes_pod_container_id"
248+
rule {
249+
source_labels = ["__meta_kubernetes_pod_container_id"]
250+
action = "replace"
251+
target_label = "container_runtime"
252+
regex = "^(\\S+):\\/\\/.+$"
253+
replacement = "$1"
254+
}
255+
}
256+
257+
// loki.source.kubernetes tails logs from Kubernetes containers using the Kubernetes API.
258+
loki.source.kubernetes "pod_logs" {
259+
targets = discovery.relabel.pod_logs.output
260+
forward_to = [loki.process.pod_logs.receiver]
261+
}
262+
// loki.process receives log entries from other Loki components, applies one or more processing stages,
263+
// and forwards the results to the list of receivers in the component’s arguments.
264+
loki.process "pod_logs" {
265+
stage.static_labels {
266+
values = {
267+
cluster = "envoy-gateway",
268+
}
269+
}
270+
271+
forward_to = [loki.write.alloy.receiver]
272+
}
273+
170274

171275
# Values for Tempo dependency
172276
tempo:

‎site/content/en/latest/install/gateway-addons-helm-api.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ An Add-ons Helm chart for Envoy Gateway
2424
| Repository | Name | Version |
2525
|------------|------|---------|
2626
| https://fluent.github.io/helm-charts | fluent-bit | 0.30.4 |
27+
| https://grafana.github.io/helm-charts | alloy | 0.9.2 |
2728
| https://grafana.github.io/helm-charts | grafana | 8.0.0 |
2829
| https://grafana.github.io/helm-charts | loki | 4.8.0 |
2930
| https://grafana.github.io/helm-charts | tempo | 1.3.1 |
@@ -34,6 +35,9 @@ An Add-ons Helm chart for Envoy Gateway
3435

3536
| Key | Type | Default | Description |
3637
|-----|------|---------|-------------|
38+
| alloy.alloy.configMap.content | string | `"// Write your Alloy config here:\nlogging {\n level = \"info\"\n format = \"logfmt\"\n}\nloki.write \"alloy\" {\n endpoint {\n url = \"http://loki.monitoring.svc:3100/loki/api/v1/push\"\n }\n}\n// discovery.kubernetes allows you to find scrape targets from Kubernetes resources.\n// It watches cluster state and ensures targets are continually synced with what is currently running in your cluster.\ndiscovery.kubernetes \"pod\" {\n role = \"pod\"\n}\n\n// discovery.relabel rewrites the label set of the input targets by applying one or more relabeling rules.\n// If no rules are defined, then the input targets are exported as-is.\ndiscovery.relabel \"pod_logs\" {\n targets = discovery.kubernetes.pod.targets\n\n // Label creation - \"namespace\" field from \"__meta_kubernetes_namespace\"\n rule {\n source_labels = [\"__meta_kubernetes_namespace\"]\n action = \"replace\"\n target_label = \"namespace\"\n }\n\n // Label creation - \"pod\" field from \"__meta_kubernetes_pod_name\"\n rule {\n source_labels = [\"__meta_kubernetes_pod_name\"]\n action = \"replace\"\n target_label = \"pod\"\n }\n\n // Label creation - \"container\" field from \"__meta_kubernetes_pod_container_name\"\n rule {\n source_labels = [\"__meta_kubernetes_pod_container_name\"]\n action = \"replace\"\n target_label = \"container\"\n }\n\n // Label creation - \"app\" field from \"__meta_kubernetes_pod_label_app_kubernetes_io_name\"\n rule {\n source_labels = [\"__meta_kubernetes_pod_label_app_kubernetes_io_name\"]\n action = \"replace\"\n target_label = \"app\"\n }\n\n // Label creation - \"job\" field from \"__meta_kubernetes_namespace\" and \"__meta_kubernetes_pod_container_name\"\n // Concatenate values __meta_kubernetes_namespace/__meta_kubernetes_pod_container_name\n rule {\n source_labels = [\"__meta_kubernetes_namespace\", \"__meta_kubernetes_pod_container_name\"]\n action = \"replace\"\n target_label = \"job\"\n separator = \"/\"\n replacement = \"$1\"\n }\n\n // Label creation - \"container\" field from \"__meta_kubernetes_pod_uid\" and \"__meta_kubernetes_pod_container_name\"\n // Concatenate values __meta_kubernetes_pod_uid/__meta_kubernetes_pod_container_name.log\n rule {\n source_labels = [\"__meta_kubernetes_pod_uid\", \"__meta_kubernetes_pod_container_name\"]\n action = \"replace\"\n target_label = \"__path__\"\n separator = \"/\"\n replacement = \"/var/log/pods/*$1/*.log\"\n }\n\n // Label creation - \"container_runtime\" field from \"__meta_kubernetes_pod_container_id\"\n rule {\n source_labels = [\"__meta_kubernetes_pod_container_id\"]\n action = \"replace\"\n target_label = \"container_runtime\"\n regex = \"^(\\\\S+):\\\\/\\\\/.+$\"\n replacement = \"$1\"\n }\n}\n\n// loki.source.kubernetes tails logs from Kubernetes containers using the Kubernetes API.\nloki.source.kubernetes \"pod_logs\" {\n targets = discovery.relabel.pod_logs.output\n forward_to = [loki.process.pod_logs.receiver]\n}\n// loki.process receives log entries from other Loki components, applies one or more processing stages,\n// and forwards the results to the list of receivers in the component’s arguments.\nloki.process \"pod_logs\" {\n stage.static_labels {\n values = {\n cluster = \"envoy-gateway\",\n }\n }\n\n forward_to = [loki.write.alloy.receiver]\n}"` | |
39+
| alloy.enabled | bool | `false` | |
40+
| alloy.fullnameOverride | string | `"alloy"` | |
3741
| fluent-bit.config.filters | string | `"[FILTER]\n Name kubernetes\n Match kube.*\n Merge_Log On\n Keep_Log Off\n K8S-Logging.Parser On\n K8S-Logging.Exclude On\n\n[FILTER]\n Name grep\n Match kube.*\n Regex $kubernetes['container_name'] ^envoy$\n\n[FILTER]\n Name parser\n Match kube.*\n Key_Name log\n Parser envoy\n Reserve_Data True\n"` | |
3842
| fluent-bit.config.inputs | string | `"[INPUT]\n Name tail\n Path /var/log/containers/*.log\n multiline.parser docker, cri\n Tag kube.*\n Mem_Buf_Limit 5MB\n Skip_Long_Lines On\n"` | |
3943
| fluent-bit.config.outputs | string | `"[OUTPUT]\n Name loki\n Match kube.*\n Host loki.monitoring.svc.cluster.local\n Port 3100\n Labels job=fluentbit, app=$kubernetes['labels']['app'], k8s_namespace_name=$kubernetes['namespace_name'], k8s_pod_name=$kubernetes['pod_name'], k8s_container_name=$kubernetes['container_name']\n"` | |

0 commit comments

Comments
 (0)