The proxy buffers requests in several places, especially when routing requests. These buffers are composed with layers like Router -> Buffer -> Route; though, the buffer layer is actually two distinct layers, separated by a channel. The "inside" (Route-side) of the stack is owned by the executor and the "outside" (Router-side) of the stack may be cloned and queue requests into the task.

Consider the case where the following happens:

1. Route is unavailable and returns NotReady.
2. Requests are routed and buffered.
3. The Router drops this route.

At this point, the inner buffer worker has a channel that has requests on it and is unable to dispatch them to the internal service. If it finished consuming the buffered requests, it would be informed that the senders were released and it would be able to shutdown. However, the whole buffer is basically leaked into the executor if the internal service never consumes the rest of the requests!

linkerd/linkerd2-proxy#246 treats one of the symptoms of this bug -- timing out response futures if the request is not dequeued at least prevents clients from seeing requests hang forever; But it doesn't address the underlying leak of the inner stack.

This can be especially egregious if the inner stack is driving service discovery resolutions, etc.