c-ares resolver blocks channel indefinitely if partial or no response received from DNS #17436
Description
What version of gRPC and what language are you using?
1.16
What operating system (Linux, Windows, …) and version?
Linux - Ubuntu 16.04
What runtime / compiler are you using (e.g. python version or version of gcc)
gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.10)
What did you do?
export GRPC_DNS_RESOLVER=ares
change greeter_client.cc:
GreeterClient greeter(grpc::CreateChannel(
"dns://172.217.3.174:53/example.com", grpc::InsecureChannelCredentials()));where 172.217.3.174 is partially or fully unreachable IP address.
What did you expect to see?
DNS resolution attempt would time out and call would fail.
What did you see instead?
Resolution was tried once and never tried again, and the resolution never timed out blocking call indefinitely.
You can see from the trace output that ares never tried again:
D1206 15:32:24.060189349 90518 dns_resolver_ares.cc:175] AresDnsResolver::NextLocked() is called.
D1206 15:32:24.060193149 90518 dns_resolver_ares.cc:415] Start resolving.
D1206 15:32:24.060335349 90518 grpc_ares_ev_driver.cc:128] grpc_ares_ev_driver_create_locked
I1206 15:32:24.060505148 90518 ev_posix.cc:245] (fd-trace) fd_create(5, c-ares fd: 5, 0)
D1206 15:32:24.060519448 90518 grpc_ares_ev_driver.cc:255] new fd: c-ares fd: 5
D1206 15:32:24.060524848 90518 grpc_ares_ev_driver.cc:85] Ref ev_driver 11563248
D1206 15:32:24.060528948 90518 grpc_ares_ev_driver.cc:272] notify read on: c-ares fd: 5
E1206 15:32:24.060534548 90518 lockfree_event.cc:98] LockfreeEvent::NotifyOn: 0xb0a7b8 curr=(nil) closure=0xb0a668
I1206 15:32:24.060540048 90518 client_channel.cc:2857] chand=0xb07d70 calld=0xb08fc0: deferring pick pending resolver result
Then no forward progress.
Same behavior is observed if no response received for only one of the queries (A, AAAA, SRV, TXT).