Skip to content

Unbundle Prometheus and Grafana #3406

@andrew-waters

Description

@andrew-waters

Feature Request

What problem are you trying to solve?

linkerd2 deployments come with Grafana and Prometheus bundled into the deployment. This is great for single cluster setups and PoC's but it comes at the cost of resources when an existing cluster already has prometheus running. The documentation around exporting metrics is useful and for the specific use case of running thanos, the service monitor is the preferable route (the purpose of thanos is to avoid federation, which has legitimate issues).

Below is an approximate topology of a multi cluster setup running thanos (as close as possible to the linkerd2 supplied one).

linkerd (1)

As you can see it should in theory be possible to remove prometheus and grafana from the linkerd deployment and maintain metric collection.

However, removing these cause errors with linkerd check and linkerd dashboard (they appear to check for the existence.

How should the problem be solved?

I'm not entirely sure of the scope of changes that are required, but adding a flag to the commands that would otherwise fail may be a useful starting point for discussion:

linkerd check --exclude=prometheus,grafana
linkerd install --exclude=prometheus,grafana
linkerd dashboard

Note that linkerd dashboard fails when prometheus isn't available which suggests there may be some core logic that is shared validation logic between the cli.

Any alternatives you've considered?

The only alternatives are to:

  • ignore this issue and force users to install prometheus and grafana
  • avoid using the CLI

How would users interact with this feature?

Referenced above in bash scripts.


Edit: updated diagram to have a white background for legibility

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions