Repository navigation
fix(chart): emit prometheus.io/scheme=https across all annotation paths - #3055
Conversation
There was a problem hiding this comment.
Code Review
This pull request updates the MetalLB Helm chart by adding the prometheus.io/scheme: "https" annotation to the controller and speaker pod templates to support HTTPS metrics scraping. The reviewer pointed out that the changes are incomplete, as podmonitor.yaml and servicemonitor.yaml also require updates to their schemes and TLS configurations to prevent Prometheus scraping failures.
|
@lexfrei please fill the release notes and kind as per the template you overrode |
|
and thanks for the PR! |
|
@fedepaol thanks for the heads-up! I missed the template on the first pass; the body now follows the
Rebasing each branch onto current |
01409cd to
c77fa0a
Compare
Since v0.16.0 kube-rbac-proxy is gone and both controller and speaker serve
`/metrics` directly over HTTPS on port 9120 (`tls.controllerMetricsTLSSecret`
/ `tls.speakerMetricsTLSSecret` provide certs; a self-signed cert is
auto-generated if neither is set). Three different scrape-discovery
surfaces in the chart still pointed at HTTP, all of which now fail with
TLS handshake errors on the v0.16 endpoint:
1. Pod-level `prometheus.io/scrape` annotations on the controller
Deployment and speaker DaemonSet. Annotation-based scrapers default to
HTTP without an explicit `prometheus.io/scheme`.
2. `PodMonitor.spec.podMetricsEndpoints` for both controller and speaker
in `templates/podmonitor.yaml`. PodMonitor endpoints default to HTTP
when `scheme:` is unset.
3. The annotation-only `Service` objects bundled alongside the
ServiceMonitors in `templates/servicemonitor.yaml` (speaker-monitor-
service and controller-monitor-service). These services exist to be
discovered via `prometheus.io/scrape` annotations by scrapers that do
not use the Operator's ServiceMonitor CRD; they were missing the
matching `prometheus.io/scheme` annotation.
Emit `prometheus.io/scheme: "https"` / `scheme: https` in each of those
three paths.
Why unconditional `https`:
The chart no longer exposes a knob to disable TLS on metrics
(`tls.controllerMetricsTLSSecret` / `tls.speakerMetricsTLSSecret` are
optional only in the sense that a self-signed cert is auto-generated
when unset). The chart's own ServiceMonitor `spec.endpoints[].scheme` is
already hardcoded to `"https"`, confirming the project's stance that the
endpoint is HTTPS-only. The annotation and PodMonitor paths were simply
out of sync.
Speaker FRR-mode guard (pod-level annotations only):
The new `prometheus.io/scheme` annotation on the speaker pod template
sits inside the existing `{{- if not .Values.speaker.frr.enabled }}`
block. Legacy FRR mode runs multiple metrics endpoints (speaker on
9120, frr sidecar on 9121) and a single annotation pair cannot cover
both — the chart already suppresses `prometheus.io/port` in that case
and emits only `prometheus.io/scrape: "true"`. Suppressing the new
`scheme` annotation under the same guard keeps the annotation set
internally consistent; users on legacy FRR mode are expected to use
ServiceMonitor.
Verified locally (helm v3.18, v3.21):
- Default mode (frr-k8s): controller and speaker pods both emit
`scrape: true` + `scheme: https` + `port: 9120` annotations.
- Legacy FRR mode (`speaker.frr.enabled=true`, `frrk8s.enabled=false`):
speaker emits only `scrape: true`; controller emits the full triple.
- `prometheus.podMonitor.enabled=true`: both PodMonitors render with
`podMetricsEndpoints[].scheme: https`.
- `prometheus.serviceMonitor.enabled=true`: both monitor-services render
with `prometheus.io/scrape: "true"` AND `prometheus.io/scheme: "https"`.
- Without `prometheus.scrapeAnnotations`: zero `prometheus.io/*`
annotations on the pods.
- `helm lint` clean.
ServiceMonitor users are unaffected by the pod-annotation change (they
declare `scheme: https` explicitly in the ServiceMonitor spec). The
PodMonitor and Service-annotation changes round out the missing pieces
in the same area.
Signed-off-by: Aleksei Sviridkin <[email protected]>
c77fa0a to
8bdda33
Compare
Is this a BUG FIX or a FEATURE ?:
/kind bug
What this PR does / why we need it:
Since v0.16.0 kube-rbac-proxy is gone and both controller and speaker serve
/metricsdirectly over HTTPS on port 9120. Three different scrape-discovery surfaces in the chart still pointed at HTTP, all of which now fail with TLS handshake errors on the v0.16 endpoint:prometheus.io/scrapeannotations on the controller Deployment and speaker DaemonSet. Annotation-based scrapers default to HTTP without an explicitprometheus.io/scheme.PodMonitor.spec.podMetricsEndpointsfor both controller and speaker intemplates/podmonitor.yaml. PodMonitor endpoints default to HTTP whenscheme:is unset.Serviceobjects bundled alongside the ServiceMonitors intemplates/servicemonitor.yaml(speaker-monitor-serviceandcontroller-monitor-service). These services exist to be discovered viaprometheus.io/scrapeannotations by scrapers that do not use the Operator's ServiceMonitor CRD; they were missing the matchingprometheus.io/schemeannotation.Emit
prometheus.io/scheme: "https"/scheme: httpsin each of those three paths.The chart no longer exposes a knob to disable TLS on metrics (
tls.controllerMetricsTLSSecret/tls.speakerMetricsTLSSecretare optional only in the sense that a self-signed cert is auto-generated when unset). The chart's own ServiceMonitorspec.endpoints[].schemeis already hardcoded to"https", confirming the project's stance that the endpoint is HTTPS-only. The annotation and PodMonitor paths were simply out of sync.The new
prometheus.io/schemeannotation on the speaker pod template sits inside the existing{{- if not .Values.speaker.frr.enabled }}block, matching the guard already used forprometheus.io/port(legacy FRR mode runs multiple metrics endpoints and a single annotation pair cannot cover both — users on FRR mode are expected to use ServiceMonitor anyway).Special notes for your reviewer:
Verified locally (helm v3.18, v3.21):
scrape: true+scheme: https+port: 9120annotations.speaker.frr.enabled=true,frrk8s.enabled=false): speaker emits onlyscrape: true; controller emits the full triple.prometheus.podMonitor.enabled=true: both PodMonitors render withpodMetricsEndpoints[].scheme: https.prometheus.serviceMonitor.enabled=true: both monitor-services render withprometheus.io/scrape: "true"ANDprometheus.io/scheme: "https".prometheus.scrapeAnnotations: zeroprometheus.io/*annotations on the pods.helm lintclean.ServiceMonitor users are unaffected by the pod-annotation change (they declare
scheme: httpsexplicitly in the ServiceMonitor spec). The PodMonitor and Service-annotation changes round out the missing pieces in the same area.Release note: