Skip to content

helm: templating prometheus url into cronjob heartbeat  #11342

@david972

Description

@david972

What is the issue?

As the prometheus url is hard coded we have to edit it after the deployment if we use external instance or linkerd-viz installed in non-default namespace.

How can it be reproduced?

helm install linkerd-control-plane -n linkerd \
  --set-file identityTrustAnchorsPEM=ca.crt \
  --set-file identity.issuer.tls.crtPEM=issuer.crt \
  --set-file identity.issuer.tls.keyPEM=issuer.key \
  linkerd/linkerd-control-plane

Logs, error output, etc

kubectl get cronjob  

output:

apiVersion: batch/v1
kind: CronJob
metadata:
  ...
  name: linkerd-heartbeat
  namespace: linkerd
...
        spec:
          containers:
          - args:
            - heartbeat
            - -controller-namespace=linkerd
            - -log-level=info
            - -log-format=plain
            - -prometheus-url=http://prometheus.linkerd-viz.svc.cluster.local:9090

output of linkerd check -o short

inkerd-identity
----------------
‼ trust anchors are valid for at least 60 days
    Anchors expiring soon:
	* 498695134056946804576461668678438017343534005643 root.linkerd.cluster.local will expire on 2023-09-30T07:30:25Z
    see https://linkerd.io/2.13/checks/#l5d-identity-trustAnchors-not-expiring-soon for hints
‼ issuer cert is valid for at least 60 days
    issuer certificate will expire on 2023-09-07T20:43:28Z
    see https://linkerd.io/2.13/checks/#l5d-identity-issuer-cert-not-expiring-soon for hints

linkerd-version
---------------
‼ cli is up-to-date
    is running version 2.13.5 but the latest stable version is 2.14.0
    see https://linkerd.io/2.13/checks/#l5d-version-cli for hints

control-plane-version
---------------------
‼ control plane is up-to-date
    is running version 2.13.4 but the latest stable version is 2.14.0
    see https://linkerd.io/2.13/checks/#l5d-version-control for hints
‼ control plane and cli versions match
    control plane running stable-2.13.4 but cli running stable-2.13.5
    see https://linkerd.io/2.13/checks/#l5d-version-control for hints

linkerd-control-plane-proxy
---------------------------
‼ control plane proxies are up-to-date
    some proxies are not running the current version:
	* linkerd-destination-65c9bd846b-m6ctj (stable-2.13.4)
	* linkerd-identity-5c9cf7796c-gspsq (stable-2.13.4)
	* linkerd-proxy-injector-768454944f-k6trm (stable-2.13.4)
	* metrics-api-7cf574988f-jrr5q (stable-2.13.4)
	* tap-65f8b84587-9h9nh (stable-2.13.4)
	* tap-injector-8557f9fdbc-8dnnf (stable-2.13.4)
	* web-695f5ddbc4-9mkdd (stable-2.13.4)
    see https://linkerd.io/2.13/checks/#l5d-cp-proxy-version for hints
‼ control plane proxies and cli versions match
    linkerd-destination-65c9bd846b-m6ctj running stable-2.13.4 but cli running stable-2.13.5
    see https://linkerd.io/2.13/checks/#l5d-cp-proxy-cli-version for hints

linkerd-ha-checks
-----------------
‼ pod injection disabled on kube-system
    kube-system namespace needs to have the label config.linkerd.io/admission-webhooks: disabled if injector webhook failure policy is Fail
    see https://linkerd.io/2.13/checks/#l5d-injection-disabled for hints

linkerd-jaeger
--------------
‼ jaeger extension pods are injected
    could not find proxy container for collector-f4db58985-ks7cp pod
    see https://linkerd.io/2.13/checks/#l5d-jaeger-pods-injection for hints
‼ jaeger injector pods are running
    container "linkerd-proxy" in pod "collector-f4db58985-ks7cp" is not ready
    see https://linkerd.io/2.13/checks/#l5d-jaeger-pods-running for hints
‼ jaeger extension proxies are healthy
    no "linkerd-proxy" containers found in the "linkerd" namespace
    see https://linkerd.io/2.13/checks/#l5d-jaeger-proxy-healthy for hints

linkerd-viz
-----------
‼ linkerd-viz pods are injected
    could not find proxy container for trust-manager-55c864b669-9f4x6 pod
    see https://linkerd.io/2.13/checks/#l5d-viz-pods-injection for hints
‼ viz extension proxies are up-to-date
    some proxies are not running the current version:
	* linkerd-destination-65c9bd846b-m6ctj (stable-2.13.4)
	* linkerd-identity-5c9cf7796c-gspsq (stable-2.13.4)
	* linkerd-proxy-injector-768454944f-k6trm (stable-2.13.4)
	* metrics-api-7cf574988f-jrr5q (stable-2.13.4)
	* tap-65f8b84587-9h9nh (stable-2.13.4)
	* tap-injector-8557f9fdbc-8dnnf (stable-2.13.4)
	* web-695f5ddbc4-9mkdd (stable-2.13.4)
    see https://linkerd.io/2.13/checks/#l5d-viz-proxy-cp-version for hints
‼ viz extension proxies and cli versions match
    linkerd-destination-65c9bd846b-m6ctj running stable-2.13.4 but cli running stable-2.13.5
    see https://linkerd.io/2.13/checks/#l5d-viz-proxy-cli-version for hints
‼ prometheus is installed and configured correctly
    missing ClusterRoles: linkerd-linkerd-prometheus
    see https://linkerd.io/2.13/checks/#l5d-viz-prometheus for hints

Status check results are √

Environment

  • Kubernetes Version: v1.26.7-eks-2d98532
  • Cluster Environment: EKS
  • Host OS: Amazon Linux 2
  • Linkerd version: 2.13.4

Possible solution

Adding prometheusUrl key in the control-plane chart like viz chart can be fix the issue

Additional context

No response

Would you like to work on fixing this bug?

yes

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions