Description

What happens:
Connection is tried to brokers not reported in Metadata response, leading to timeouts that can vary depending if broker DNS entry has been removed (quicker), or DNS is available but there's no route to host (slower), or broker is reachable but port isn't bound (quicker).
Failed connection to these removed brokers can continue until client is restarted.

Conditions:
When scaling down a cluster, or just during a cluster roll, in case there aren't available connections already, the least used broker is selected and, in that case, one of the brokers removed from the cluster temporarily or permanently (client cannot know) can be used.

How to reproduce

TBW

Checklist

Please provide the following information:

• librdkafka version (release number or git tag): any
• Apache Kafka version: any
• librdkafka client configuration: any
• Operating system: any
• Provide logs (with debug=.. as necessary) from librdkafka
• Provide broker log excerpts
• Critical issue: it can be critical depending on the duration of the timeout, otherwise client recovers but it causes increased latency