Skip to content

linkerd-multicluster remoteDiscoverySelector default selects all remote services #11309

@willhughes-au

Description

@willhughes-au

What is the issue?

remoteDiscoverySelector was introduced on the link-crd for linkerd-multicluster in stable-2.14.

Added a new remoteDiscoverySelector field to the multicluster Link CRD,
which enables a service mirroring mode where the control plane
performs discovery for the mirrored service from the remote cluster, rather
than creating Endpoints for the mirrored service in the source cluster

Unfortunately the default behavior is that it selects all remote services.

How can it be reproduced?

Setup two K8S clusters, clusterA and clusterB with Linkerd 2.13.6 installed and working with linkerd-multicluster.

On clusterB have at least two services. Add a label on one service with 'mirror.linkerd.io/mirror-to-clusterA'=true

On clusterA add a Link resource that looks like:

apiVersion: multicluster.linkerd.io/v1alpha1
kind: Link
metadata:
  name: clusterB
  namespace: linkerd-multicluster
spec:
  clusterCredentialsSecret: cluster-credentials-clusterB
  gatewayAddress: >-
    the-gateway-address-for-clusterB.example.com
  gatewayIdentity: >-
    the-gateway-identity-for-clusterB.cluster.local
  gatewayPort: '4143'
  probeSpec:
    path: /ready
    period: 3s
    port: '4191'
  selector:
    matchLabels:
      mirror.linkerd.io/mirror-to-clusterA: 'true'
  targetClusterDomain: cluster.local
  targetClusterLinkerdNamespace: linkerd
  targetClusterName: clusterB

Make sure the link is all working, and that the service from clusterB appears on clusterA.

Now upgrade clusterA to 2.14, including linkerd-multicluster.

Recreate the link.

Observe that all the services from clusterB are mirrored to clusterA.

To stop this behavior, you have to set the remoteDiscoverySelector to something that is not null or an empty set.

eg:

  remoteDiscoverySelector:
    matchExpressions: []
    matchLabels:
      mirror.linkerd.io/stop-with-the-remote-discovery: 'true'

Logs, error output, etc

I'm not sure what relevant logs there are.

The pod linkerd-service-mirror-clusterB on clusterA is filled with a whole lot of events about creating a new service mirror for

output of linkerd check -o short

$ linkerd check -o short --context clusterA
linkerd-multicluster
--------------------
× all mirror services have endpoints
    Some mirror services do not have endpoints:
    (list of services that were picked up automatically)
    see https://linkerd.io/2.14/checks/#l5d-multicluster-services-endpoints for hints

Status check results are ×

Environment

  • Kubernetes Version: 1.27
  • Cluster Environment: AWS EKS
  • Host OS: Amazon Linux 2, EKS Optimised version
  • Linkerd: stable-2.14

Possible solution

If remoteDiscoverySelector is not set on the Link resource, remote discovery should not be used.

Additional context

No response

Would you like to work on fixing this bug?

no

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions