Skip to content

Commit 7fb3aa6

Browse files
authored
Expose the controller's /metrics endpoint through a dedicated Service (#1803)
Operators deploying through this chart can now point Prometheus, VictoriaMetrics, or any other authenticated scraper at the controller's `/metrics` endpoint without hand-rolling a Service, ClusterRoles, and a binding. The controller binary already understands `--metrics-bind-address` and `--metrics-secure` via flags or env vars (the env-var loader at `go/core/pkg/app/app.go` translates every flag automatically), but the chart had no way to surface the endpoint short of stuffing values into `controller.env` and writing the missing Service and RBAC by hand. Closes #1369. The shape mirrors the `kmcp` sub-chart that already lives in this same umbrella: a separate `<fullname>-controller-metrics` Service exposing `https:8443`, two ClusterRoles (`metrics-auth-role` granting the controller's ServiceAccount the right to call `TokenReview` and `SubjectAccessReview` against the API server; `metrics-reader` granting `get` on the `/metrics` non-resource URL), and a ClusterRoleBinding wiring only the auth role to the controller. The reader role is deliberately left unbound — picking a subject (Prometheus, VictoriaMetrics, an OpenTelemetry collector) belongs to the cluster operator, not the chart. This is also what `kubebuilder` and `operator-sdk` scaffold by default, for the same reason. The auth role and its binding are gated additionally on `controller.metrics.secureServing` because they're no-ops when the authn/authz filter isn't installed. The deployment's existing `controller.env` pass-through still wins over the chart-supplied defaults (the conditional env vars are emitted before the user-supplied list), so anyone who needs `METRICS_BIND_ADDRESS=0` for debugging or `--metrics-cert-path` wired through `volumes`/`volumeMounts` retains the escape hatch. Designed exclusions: `ServiceMonitor` is deferred — it pulls in a Prometheus-Operator coupling the rest of the chart has avoided, and it's straightforward to layer on top once the underlying Service exists, gated on `controller.metrics.serviceMonitor.enabled`. Cert-manager-issued metrics certificates are also out of scope; controller-runtime's self-signed default works for in-cluster scrapers that skip TLS verification or trust the kubelet CA, and the existing cert-path flags remain reachable via `controller.env` for those who want to wire their own. Default is opt-in (`controller.metrics.enabled: false`), so existing installations are unchanged. --------- Signed-off-by: Daniel Orbach <[email protected]>
1 parent 45b98d7 commit 7fb3aa6

10 files changed

Lines changed: 533 additions & 1 deletion

‎helm/kagent/templates/_helpers.tpl‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -143,6 +143,33 @@ Check if leader election should be enabled (more than 1 replica)
143143
{{- gt (.Values.controller.replicas | int) 1 -}}
144144
{{- end -}}
145145

146+
{{/*
147+
Extract the TCP port from controller.metrics.bindAddress.
148+
149+
Anchors the digit run to the end of the string so every Go-style
150+
address form the controller binary accepts is handled correctly: bare
151+
":port", host-qualified "host:port", and bracketed IPv6 "[::1]:port"
152+
all yield the trailing port. Returns "0" or "" when the binary's
153+
disable sentinel is in use; callers must consult
154+
`kagent.controller.metricsEnabled` before rendering manifests.
155+
*/}}
156+
{{- define "kagent.controller.metricsPort" -}}
157+
{{- regexFind "[0-9]+$" (.Values.controller.metrics.bindAddress | toString) -}}
158+
{{- end -}}
159+
160+
{{/*
161+
Returns "1" when the controller metrics resources (Service, RBAC,
162+
container port, env vars) should render, empty otherwise. Honours both
163+
disable signals: `controller.metrics.enabled=false` and the binary's
164+
own `--metrics-bind-address=0` sentinel reached through `bindAddress`.
165+
The two are equivalent so the field name keeps faith with the binary's
166+
documented contract (see go/core/pkg/app/app.go).
167+
*/}}
168+
{{- define "kagent.controller.metricsEnabled" -}}
169+
{{- $port := include "kagent.controller.metricsPort" . -}}
170+
{{- if and .Values.controller.metrics.enabled $port (ne $port "0") -}}1{{- end -}}
171+
{{- end -}}
172+
146173
{{/*
147174
PostgreSQL service name for the bundled postgres instance
148175
*/}}

‎helm/kagent/templates/controller-deployment.yaml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,12 @@ spec:
8484
{{- else }}
8585
{{ fail "No database connection configured. Set database.postgres.url, database.postgres.urlFile, or enable database.postgres.bundled." }}
8686
{{- end }}
87+
{{- if include "kagent.controller.metricsEnabled" . }}
88+
- name: METRICS_BIND_ADDRESS
89+
value: {{ .Values.controller.metrics.bindAddress | quote }}
90+
- name: METRICS_SECURE
91+
value: {{ .Values.controller.metrics.secureServing | quote }}
92+
{{- end }}
8793
{{- with .Values.controller.env }}
8894
{{- toYaml . | nindent 12 }}
8995
{{- end }}
@@ -133,6 +139,11 @@ spec:
133139
- name: http
134140
containerPort: {{ .Values.controller.service.ports.targetPort }}
135141
protocol: TCP
142+
{{- if .Values.controller.metrics.enabled }}
143+
- name: metrics
144+
containerPort: {{ include "kagent.controller.metricsPort" . | int }}
145+
protocol: TCP
146+
{{- end }}
136147
resources:
137148
{{- toYaml .Values.controller.resources | nindent 12 }}
138149
{{- with (.Values.controller.securityContext | default .Values.securityContext) }}
Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
{{- if include "kagent.controller.metricsEnabled" . }}
2+
apiVersion: v1
3+
kind: Service
4+
metadata:
5+
name: {{ include "kagent.fullname" . }}-controller-metrics
6+
namespace: {{ include "kagent.namespace" . }}
7+
labels:
8+
{{- include "kagent.controller.labels" . | nindent 4 }}
9+
spec:
10+
type: {{ .Values.controller.metrics.service.type }}
11+
ports:
12+
- name: {{ ternary "https" "http-metrics" .Values.controller.metrics.secureServing }}
13+
port: {{ .Values.controller.metrics.service.port }}
14+
targetPort: {{ include "kagent.controller.metricsPort" . | int }}
15+
protocol: TCP
16+
selector:
17+
{{- include "kagent.controller.selectorLabels" . | nindent 4 }}
18+
{{- end }}
Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
{{- if and (include "kagent.controller.metricsEnabled" .) .Values.controller.metrics.secureServing }}
2+
apiVersion: rbac.authorization.k8s.io/v1
3+
kind: ClusterRole
4+
metadata:
5+
name: {{ include "kagent.fullname" . }}-metrics-auth-role
6+
labels:
7+
{{- include "kagent.controller.labels" . | nindent 4 }}
8+
rules:
9+
- apiGroups:
10+
- authentication.k8s.io
11+
resources:
12+
- tokenreviews
13+
verbs:
14+
- create
15+
- apiGroups:
16+
- authorization.k8s.io
17+
resources:
18+
- subjectaccessreviews
19+
verbs:
20+
- create
21+
{{- end }}
Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
{{- if and (include "kagent.controller.metricsEnabled" .) .Values.controller.metrics.secureServing }}
2+
apiVersion: rbac.authorization.k8s.io/v1
3+
kind: ClusterRoleBinding
4+
metadata:
5+
name: {{ include "kagent.fullname" . }}-metrics-auth-rolebinding
6+
labels:
7+
{{- include "kagent.controller.labels" . | nindent 4 }}
8+
roleRef:
9+
apiGroup: rbac.authorization.k8s.io
10+
kind: ClusterRole
11+
name: {{ include "kagent.fullname" . }}-metrics-auth-role
12+
subjects:
13+
- kind: ServiceAccount
14+
name: {{ include "kagent.fullname" . }}-controller
15+
namespace: {{ include "kagent.namespace" . }}
16+
{{- end }}
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
{{- if include "kagent.controller.metricsEnabled" . }}
2+
apiVersion: rbac.authorization.k8s.io/v1
3+
kind: ClusterRole
4+
metadata:
5+
name: {{ include "kagent.fullname" . }}-metrics-reader
6+
labels:
7+
{{- include "kagent.controller.labels" . | nindent 4 }}
8+
rules:
9+
- nonResourceURLs:
10+
- "/metrics"
11+
verbs:
12+
- get
13+
{{- end }}

‎helm/kagent/tests/controller-deployment_test.yaml‎

Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -584,3 +584,152 @@ tests:
584584
value: 60
585585
- notExists:
586586
path: spec.template.spec.containers[0].readinessProbe.httpGet
587+
588+
- it: should not expose metrics container port by default
589+
template: controller-deployment.yaml
590+
asserts:
591+
- lengthEqual:
592+
path: spec.template.spec.containers[0].ports
593+
count: 1
594+
- notContains:
595+
path: spec.template.spec.containers[0].ports
596+
content:
597+
name: metrics
598+
599+
- it: should not set metrics env vars by default
600+
template: controller-deployment.yaml
601+
asserts:
602+
- notContains:
603+
path: spec.template.spec.containers[0].env
604+
content:
605+
name: METRICS_BIND_ADDRESS
606+
any: true
607+
- notContains:
608+
path: spec.template.spec.containers[0].env
609+
content:
610+
name: METRICS_SECURE
611+
any: true
612+
613+
- it: should expose metrics container port when enabled
614+
template: controller-deployment.yaml
615+
set:
616+
controller.metrics.enabled: true
617+
asserts:
618+
- contains:
619+
path: spec.template.spec.containers[0].ports
620+
content:
621+
name: metrics
622+
containerPort: 8443
623+
protocol: TCP
624+
625+
- it: should set metrics env vars when enabled
626+
template: controller-deployment.yaml
627+
set:
628+
controller.metrics.enabled: true
629+
asserts:
630+
- contains:
631+
path: spec.template.spec.containers[0].env
632+
content:
633+
name: METRICS_BIND_ADDRESS
634+
value: ":8443"
635+
- contains:
636+
path: spec.template.spec.containers[0].env
637+
content:
638+
name: METRICS_SECURE
639+
value: "true"
640+
641+
- it: should reflect insecure serving in env
642+
template: controller-deployment.yaml
643+
set:
644+
controller.metrics.enabled: true
645+
controller.metrics.secureServing: false
646+
asserts:
647+
- contains:
648+
path: spec.template.spec.containers[0].env
649+
content:
650+
name: METRICS_SECURE
651+
value: "false"
652+
653+
- it: should derive metrics container port from bindAddress
654+
template: controller-deployment.yaml
655+
set:
656+
controller.metrics.enabled: true
657+
controller.metrics.bindAddress: ":9443"
658+
asserts:
659+
- contains:
660+
path: spec.template.spec.containers[0].env
661+
content:
662+
name: METRICS_BIND_ADDRESS
663+
value: ":9443"
664+
- contains:
665+
path: spec.template.spec.containers[0].ports
666+
content:
667+
name: metrics
668+
containerPort: 9443
669+
protocol: TCP
670+
671+
- it: should derive metrics container port from a host-qualified bindAddress
672+
template: controller-deployment.yaml
673+
set:
674+
controller.metrics.enabled: true
675+
controller.metrics.bindAddress: "127.0.0.1:9443"
676+
asserts:
677+
- contains:
678+
path: spec.template.spec.containers[0].ports
679+
content:
680+
name: metrics
681+
containerPort: 9443
682+
protocol: TCP
683+
684+
- it: should derive metrics container port from a bracketed IPv6 bindAddress
685+
template: controller-deployment.yaml
686+
set:
687+
controller.metrics.enabled: true
688+
controller.metrics.bindAddress: "[::1]:9443"
689+
asserts:
690+
- contains:
691+
path: spec.template.spec.containers[0].ports
692+
content:
693+
name: metrics
694+
containerPort: 9443
695+
protocol: TCP
696+
697+
- it: should not gain metrics wiring when bindAddress disables metrics
698+
template: controller-deployment.yaml
699+
set:
700+
controller.metrics.enabled: true
701+
controller.metrics.bindAddress: "0"
702+
asserts:
703+
- notContains:
704+
path: spec.template.spec.containers[0].env
705+
content:
706+
name: METRICS_BIND_ADDRESS
707+
any: true
708+
- notContains:
709+
path: spec.template.spec.containers[0].env
710+
content:
711+
name: METRICS_SECURE
712+
any: true
713+
- notContains:
714+
path: spec.template.spec.containers[0].ports
715+
content:
716+
name: metrics
717+
718+
- it: should let controller.env override the chart-supplied metrics env
719+
template: controller-deployment.yaml
720+
set:
721+
controller.metrics.enabled: true
722+
controller.env:
723+
- name: METRICS_BIND_ADDRESS
724+
value: "0"
725+
asserts:
726+
- contains:
727+
path: spec.template.spec.containers[0].env
728+
content:
729+
name: METRICS_BIND_ADDRESS
730+
value: ":8443"
731+
- contains:
732+
path: spec.template.spec.containers[0].env
733+
content:
734+
name: METRICS_BIND_ADDRESS
735+
value: "0"

0 commit comments

Comments
 (0)