external-prometheus-deep TestDirectEdges is flakey in CI #6511
Description
The external-prometheus-deep Integration test frequently fails (in GitHub Actions) with the following error:
2021-07-16T22:15:51.6876915Z ##[error]TestDirectEdges - timed-out checking edges (50s)
2021-07-16T22:15:51.6888009Z --- FAIL: TestDirectEdges (73.31s)
2021-07-16T22:15:51.6888603Z edges_test.go:161: Expected output:
2021-07-16T22:15:51.6889017Z \[
2021-07-16T22:15:51.6889302Z \{
2021-07-16T22:15:51.6889673Z "src": "prometheus",
2021-07-16T22:15:51.6890325Z "src_namespace": "external\-prometheus",
2021-07-16T22:15:51.6890981Z "dst": "slow-cooker",
2021-07-16T22:15:51.6891746Z "dst_namespace": "linkerd-direct-edges-test",
2021-07-16T22:15:51.6892644Z "client_id": "prometheus.external\-prometheus",
2021-07-16T22:15:51.6893673Z "server_id": "default.linkerd-direct-edges-test",
2021-07-16T22:15:51.6898279Z "no_tls_reason": ""
2021-07-16T22:15:51.6898620Z \},
2021-07-16T22:15:51.6898927Z \{
2021-07-16T22:15:51.6899301Z "src": "prometheus",
2021-07-16T22:15:51.6900110Z "src_namespace": "external\-prometheus",
2021-07-16T22:15:51.6900617Z "dst": "terminus",
2021-07-16T22:15:51.6901354Z "dst_namespace": "linkerd-direct-edges-test",
2021-07-16T22:15:51.6902276Z "client_id": "prometheus.external\-prometheus",
2021-07-16T22:15:51.6903283Z "server_id": "default.linkerd-direct-edges-test",
2021-07-16T22:15:51.6903952Z "no_tls_reason": ""
2021-07-16T22:15:51.6904299Z \},
2021-07-16T22:15:51.6904601Z \{
2021-07-16T22:15:51.6905076Z "src": "slow-cooker",
2021-07-16T22:15:51.6905837Z "src_namespace": "linkerd-direct-edges-test",
2021-07-16T22:15:51.6906453Z "dst": "terminus",
2021-07-16T22:15:51.6907191Z "dst_namespace": "linkerd-direct-edges-test",
2021-07-16T22:15:51.6908230Z "client_id": "default.linkerd-direct-edges-test",
2021-07-16T22:15:51.6909329Z "server_id": "default.linkerd-direct-edges-test",
2021-07-16T22:15:51.6910009Z "no_tls_reason": ""
2021-07-16T22:15:51.6910354Z \}
2021-07-16T22:15:51.6910638Z \]
2021-07-16T22:15:51.6910939Z
2021-07-16T22:15:51.6911256Z actual:
2021-07-16T22:15:51.6911558Z [
2021-07-16T22:15:51.6911866Z {
2021-07-16T22:15:51.6912340Z "src": "slow-cooker",
2021-07-16T22:15:51.6913102Z "src_namespace": "linkerd-direct-edges-test",
2021-07-16T22:15:51.6913704Z "dst": "terminus",
2021-07-16T22:15:51.6914428Z "dst_namespace": "linkerd-direct-edges-test",
2021-07-16T22:15:51.6915469Z "client_id": "default.linkerd-direct-edges-test",
2021-07-16T22:15:51.6916598Z "server_id": "default.linkerd-direct-edges-test",
2021-07-16T22:15:51.6917265Z "no_tls_reason": ""
2021-07-16T22:15:51.6917604Z }
2021-07-16T22:15:51.6917900Z ]
2021-07-16T22:15:51.6918184Z
2021-07-16T22:15:51.6918490Z FAIL
2021-07-16T22:15:51.6919002Z FAIL github.com/linkerd/linkerd2/test/integration/edges 73.390s
2021-07-16T22:15:51.6919519Z FAIL
We should look into this and figure out if the test is flakey or if this indicates some unpredictability in the edges implementation.