gRPC Python doesn't handle utf-8 error messages. #16102
Description
When gRPC python connects to a server that sends back an error with a details string that contains utf-8 characters, the Python client will choke on it:
Invalid encoding on b'\xc3'
Traceback (most recent call last):
File "src/python/grpcio/grpc/_cython/_cygrpc/grpc_string.pyx.pxi", line 51, in grpc._cython.cygrpc._decode
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc3 in position 0: unexpected end of data
Traceback (most recent call last):
File "greeter_client.py", line 32, in <module>
run()
File "greeter_client.py", line 27, in run
response = stub.SayHello(helloworld_pb2.HelloRequest(name='you'))
File "/home/pixel/sources/python3/venv/lib/python3.5/site-packages/grpc/_channel.py", line 514, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/home/pixel/sources/python3/venv/lib/python3.5/site-packages/grpc/_channel.py", line 448, in _end_unary_response_blocking
raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with:
status = StatusCode.UNKNOWN
details = "Ã"
debug_error_string = "{"created":"@1532368813.278306147","description":"Error received from peer","file":"src/core/lib/surface/call.cc","file_line":1083,"grpc_message":"\u00c3","grpc_status":2}"
>
This above is done by modifying the hello world nodejs server the following way:
function sayHello(call, callback) {
var error = {
code: grpc.status.UNKNOWN,
details: 'é',
};
callback(error, {});
}
Yes, this is odd, yes, the file snippet is a utf-8 file, and yes, 'é' is supposed to be an utf-8 string in JavaScript, so I'm not sure why the python client is complaining that the utf-8 decoder can't work on this string, nor why it is using "Ã" as an output. More digging is needed, I guess. Other languages seems to get this one properly.
Additionally, the Python server itself can't send utf-8 encoded strings. Modifying the hello world example the following way:
def SayHello(self, request, context):
context.set_code(grpc.StatusCode.UNKNOWN)
context.set_details(u'é')
return helloworld_pb2.HelloReply()
yields the following error when this handler is being called:
Exception calling application: 'ascii' codec can't encode character '\xe9' in position 0: ordinal not in range(128)
Traceback (most recent call last):
File "/home/pixel/sources/python3/venv/lib/python3.5/site-packages/grpc/_server.py", line 389, in _call_behavior
return behavior(argument, context), True
File "greeter_server.py", line 31, in SayHello
context.set_details(u'é')
File "/home/pixel/sources/python3/venv/lib/python3.5/site-packages/grpc/_server.py", line 300, in set_details
self._state.details = _common.encode(details)
File "/home/pixel/sources/python3/venv/lib/python3.5/site-packages/grpc/_common.py", line 68, in encode
return s.encode('ascii')
UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' in position 0: ordinal not in range(128)