Skip to content

Linkerd check displays incorrect image versions. #12058

@jobinjosem1

Description

@jobinjosem1

What is the issue?

We are deploying linkerd by using image with digest in the helm charts. However, when running the linkerd check, the output warns that "some proxies are not running the current version," despite us running the latest version. Upon further investigation, I found that the linkerd check splits the image name by ":" and only compares the last part, leading to the warning due to the inclusion of the digest. Now that linkerd also supports controllerimageversion in the helm charts, we need to address and rectify this issue. Could someone please look into this?

Included below snippet from the helm values that are relevant to this issue.
spec:
values:
debugContainer:
image:
name: testacr.azurecr.io/linkerd/debug
version: stable-2.14.9@sha256:1beee1914418a582a29a36a36fee3067f6414790fbf6c490a3d3b16159249661
controllerImage: testacr.azurecr.io/linkerd/controller
controllerImageVersion: stable-2.14.9@sha256:d4be1bf315970b79d15234ced46296a1ba9bf14cf2277026ab15222d90773081
policyController:
image:
name: testacr.azurecr.io/linkerd/policy-controller
version: stable-2.14.9@sha256:b0fddb180e7034191e5f6c2d1d87df1c2dd287246239c5d79da8518da271e1df
proxy:
image:
name: testacr.azurecr.io/linkerd/proxy
version: stable-2.14.9@sha256:df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb
proxyInit:
image:
name: testacr.azurecr.io/linkerd/proxy-init
version: v2.2.3@sha256:1075bc22a4a8f0852311dc84c9db0552f1245d07fe4fdebd4bc6cf4566bcbc76

How can it be reproduced?

Modify the values.yaml and add the digest to the image as given tn the sample values.yaml

Logs, error output, etc

kubernetes-api

√ can initialize the client
√ can query the Kubernetes API

kubernetes-version

√ is running the minimum Kubernetes API version

linkerd-existence

√ 'linkerd-config' config map exists
√ heartbeat ServiceAccount exist
√ control plane replica sets are ready
√ no unschedulable pods
√ control plane pods are ready
√ cluster networks contains all pods
√ cluster networks contains all services

linkerd-config

√ control plane Namespace exists
√ control plane ClusterRoles exist
√ control plane ClusterRoleBindings exist
√ control plane ServiceAccounts exist
√ control plane CustomResourceDefinitions exist
√ control plane MutatingWebhookConfigurations exist
√ control plane ValidatingWebhookConfigurations exist
√ proxy-init container runs as root user if docker container runtime is used

linkerd-identity

√ certificate config is valid
√ trust anchors are using supported crypto algorithm
√ trust anchors are within their validity period
√ trust anchors are valid for at least 60 days
√ issuer cert is using supported crypto algorithm
√ issuer cert is within its validity period
√ issuer cert is valid for at least 60 days
√ issuer cert is issued by the trust anchor

linkerd-webhooks-and-apisvc-tls

√ proxy-injector webhook has valid cert
√ proxy-injector cert is valid for at least 60 days
√ sp-validator webhook has valid cert
√ sp-validator cert is valid for at least 60 days
√ policy-validator webhook has valid cert
√ policy-validator cert is valid for at least 60 days

linkerd-version

√ can determine the latest version
√ cli is up-to-date

control-plane-version

√ can retrieve the control plane version
√ control plane is up-to-date
√ control plane and cli versions match

linkerd-control-plane-proxy

√ control plane proxies are healthy
‼ control plane proxies are up-to-date
some proxies are not running the current version:
* linkerd-destination-584db84f4-6d9jp (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
* linkerd-destination-584db84f4-tsrcm (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
* linkerd-identity-74f544b9f5-cvcfx (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
* linkerd-identity-74f544b9f5-htz4z (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
* linkerd-proxy-injector-768b4c8fff-q5dcv (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
* linkerd-proxy-injector-768b4c8fff-scn7r (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
see https://linkerd.io/2.14/checks/#l5d-cp-proxy-version for hints
‼ control plane proxies and cli versions match
linkerd-destination-584db84f4-6d9jp running df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb but cli running stable-2.14.9
see https://linkerd.io/2.14/checks/#l5d-cp-proxy-cli-version for hints

linkerd-viz

√ linkerd-viz Namespace exists
√ can initialize the client
√ linkerd-viz ClusterRoles exist
√ linkerd-viz ClusterRoleBindings exist
√ tap API server has valid cert
√ tap API server cert is valid for at least 60 days
√ tap API service is running
√ linkerd-viz pods are injected
√ viz extension pods are running
√ viz extension proxies are healthy
‼ viz extension proxies are up-to-date
some proxies are not running the current version:
* metrics-api-575b57ccd8-p6vmd (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
* metrics-api-575b57ccd8-rzjp8 (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
* prometheus-58c6479c45-9pbr6 (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
* tap-c4d6bb7c4-vts5x (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
* tap-injector-6fdf557689-zcq6q (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
* web-7dfdbc5987-59xlv (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
* web-7dfdbc5987-rtkgw (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
see https://linkerd.io/2.14/checks/#l5d-viz-proxy-cp-version for hints
‼ viz extension proxies and cli versions match
metrics-api-575b57ccd8-p6vmd running df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb but cli running stable-2.14.9
see https://linkerd.io/2.14/checks/#l5d-viz-proxy-cli-version for hints
√ prometheus is installed and configured correctly
√ viz extension self-check

linkerd-smi

√ linkerd-smi extension Namespace exists
√ SMI extension service account exists
√ SMI extension pods are injected
√ SMI extension pods are running
√ SMI extension proxies are healthy

Status check results are √

output of linkerd check -o short

linkerd-control-plane-proxy

‼ control plane proxies are up-to-date
some proxies are not running the current version:
* linkerd-destination-584db84f4-6d9jp (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
* linkerd-destination-584db84f4-tsrcm (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
* linkerd-identity-74f544b9f5-cvcfx (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
* linkerd-identity-74f544b9f5-htz4z (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
* linkerd-proxy-injector-768b4c8fff-q5dcv (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
* linkerd-proxy-injector-768b4c8fff-scn7r (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
see https://linkerd.io/2.14/checks/#l5d-cp-proxy-version for hints
‼ control plane proxies and cli versions match
linkerd-destination-584db84f4-6d9jp running df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb but cli running stable-2.14.9
see https://linkerd.io/2.14/checks/#l5d-cp-proxy-cli-version for hints

linkerd-viz

‼ viz extension proxies are up-to-date
some proxies are not running the current version:
* metrics-api-575b57ccd8-p6vmd (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
* metrics-api-575b57ccd8-rzjp8 (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
* prometheus-58c6479c45-9pbr6 (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
* tap-c4d6bb7c4-vts5x (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
* tap-injector-6fdf557689-zcq6q (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
* web-7dfdbc5987-59xlv (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
* web-7dfdbc5987-rtkgw (df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb)
see https://linkerd.io/2.14/checks/#l5d-viz-proxy-cp-version for hints
‼ viz extension proxies and cli versions match
metrics-api-575b57ccd8-p6vmd running df43272004ad029cd48236b60d066722a4a923c3c8e4d6f49225873dc2cdf6cb but cli running stable-2.14.9
see https://linkerd.io/2.14/checks/#l5d-viz-proxy-cli-version for hints

Status check results are √

Environment

kubernetes: 1.27.7
Cluster: AKS

Possible solution

Looks like some changes are needed here. Instead of checking the last part, we may need skip the digest part and do the comparison.

func GetProxyVersion(pod corev1.Pod) string {

Additional context

No response

Would you like to work on fixing this bug?

None

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions