{"id":19960,"date":"2018-01-08T12:15:13","date_gmt":"2018-01-08T10:15:13","guid":{"rendered":"https:\/\/www.webcodegeeks.com\/?p=19960"},"modified":"2018-01-03T11:35:16","modified_gmt":"2018-01-03T09:35:16","slug":"exploring-security-metrics-error-handling-grpc-python","status":"publish","type":"post","link":"https:\/\/www.webcodegeeks.com\/python\/exploring-security-metrics-error-handling-grpc-python\/","title":{"rendered":"Exploring Security, Metrics, and Error-handling with gRPC in Python"},"content":{"rendered":"

In my post \u201cUsing gRPC in Python,\u201d<\/a> we wrote a basic gRPC server implementing a users<\/code> service. We are going to expand on it and explore more gRPC concepts, such as secure client-server communication via self-signed SSL certificates, implementing gRPC middleware (or interceptors<\/code>), and error handling.<\/p>\n

We will be using Python 3.6 for our demos in this article. The git repository<\/a> has all the code listings we will be discussing in this article in the subdirectory demo2<\/code>.<\/p>\n

Securing Client-server Communication<\/h2>\n

We initialized the server in the first article as follows:<\/p>\n

..\r\nserver.add_insecure_port('127.0.0.1:50051')\r\n..<\/pre>\n
The add_insecure_port()<\/code> method initializes an insecure<\/code> server and client-server communication is unencrypted. It is however recommended that a gRPC client-server communication takes place over a secure channel via SSL\/TLS<\/a>.<\/p>\n
First, we will create a self-signed certificate using openssl<\/code>:<\/p>\n
$ # Generate a private key\r\n\r\n$ openssl genrsa -out server.key 2048\r\nGenerating RSA private key, 2048 bit long modulus\r\n.........+++\r\n.......................................................................+++\r\ne is 65537 (0x10001)\r\n\r\n$ openssl req -new -x509 -sha256 -key server.key -out server.crt -days 3650\r\nYou are about to be asked to enter information that will be incorporated\r\ninto your certificate request.\r\nWhat you are about to enter is what is called a Distinguished Name or a DN.\r\nThere are quite a few fields but you can leave some blank\r\nFor some fields there will be a default value,\r\nIf you enter '.', the field will be left blank.\r\n-----\r\nCountry Name (2 letter code) [AU]:AU\r\nState or Province Name (full name) [Some-State]:NSW\r\nLocality Name (eg, city) []:Sydney\r\nOrganization Name (eg, company) [Internet Widgits Pty Ltd]:gRPC Demo\r\nOrganizational Unit Name (eg, section) []:gRPC\r\nCommon Name (e.g. server FQDN or YOUR name) []:localhost\r\nEmail Address []:a@a.com\r\n(<\/pre>\n
At this stage, we will have two files: server.key<\/code> and server.crt<\/code>. On the server side, we will need both these files; on the client side, we will only need the server.crt<\/code> file.<\/p>\n
Using the above certificate, we will now modify our server code as follows:<\/p>\n
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))\r\nusers_service.add_UsersServicer_to_server(UsersService(), server)\r\n# read in key and certificate\r\nwith open(os.path.join(os.path.split(__file__)[0], 'server.key')) as f:\r\n    private_key = f.read().encode()\r\nwith open(os.path.join(os.path.split(__file__)[0], 'server.crt')) as f:\r\n    certificate_chain = f.read().encode()\r\n# create server credentials\r\nserver_creds = grpc.ssl_server_credentials(((private_key, certificate_chain,),))\r\nserver.add_secure_port('localhost:50051', server_creds)<\/pre>\n
Comparing it to the earlier version of our server, add_secure_port<\/code> has replaced add_insecure_port()<\/code>. We have kept things simple here and have copied the server.key<\/code> and server.crt<\/code> file to the same directory as the server.py<\/code> file. For all purposes beyond demonstration, these locations should be passed as configuration values (environment variables, for example).<\/p>\n
On the client side, we will do a similar modification. Earlier, we created a channel as follows:<\/p>\n
channel = grpc.insecure_channel('localhost:50051')<\/pre>\n
Now to communicate with our server over TLS, we will do the following:<\/p>\n
with open('server.crt') as f:\r\n    trusted_certs = f.read().encode()\r\n# create credentials\r\ncredentials = grpc.ssl_channel_credentials(root_certificates=trusted_certs)\r\nchannel = grpc.secure_channel('localhost:50051', credentials)\r\n...<\/pre>\n
Implementing Server-side gRPC Interceptors<\/h2>\n
Interceptors are custom code that run on servers and clients during a request\/response lifecycle. If you are familiar with the concept of middleware in HTTP, interceptors are middleware for gRPC. Support for interceptors in Python is still under discussion, but here we will play with the current proposed implementation<\/a>.<\/p>\n
The demo2\/grpc-services\/grpc_interceptors<\/code> sub-directory contains a copy of the proposed implementation (commit hash<\/a>).<\/p>\n
Using the above implementation, we will be writing our first server interceptor.<\/p>\n
Exporting server metrics<\/h3>\n
The metric interceptor<\/code> we will implement will push request latency in seconds (for every request) to statsd<\/code>, which we will then convert to prometheus<\/code> metrics via a statsd bridge<\/code>. I recommend my earlier post<\/a> to understand more about my choice of the approach.<\/p>\n
To write an interceptor, we first subclass the appropriate classes from the grpc_interceptors<\/code> package:<\/p>\n
...\r\nfrom grpc_interceptors import UnaryUnaryServerInterceptor, UnaryStreamServerInterceptor\r\nclass MetricInterceptor(UnaryUnaryServerInterceptor, UnaryStreamServerInterceptor):\r\n\r\n    def __init__(self):\r\n        print(\"Initializing metric interceptor\")\r\n\r\n    @send_metrics\r\n    def intercept_unary_unary_handler(self, handler, method, request, servicer_context):\r\n        return handler(request, servicer_context)\r\n\r\n    @send_metrics\r\n    def intercept_unary_stream_handler(self, handler, method, request, servicer_context):\r\n        result = handler(request, servicer_context)\r\n        for response in result:\r\n            yield response\r\n...<\/pre>\n
Our class, MetricInterceptor<\/code>, derives from UnaryUnaryServerInterceptor<\/code> and UnaryStreamServerInterceptor<\/code> and then overrides the intercept_unary_unary_handler<\/code> and intercept_unary_stream_handler<\/code> methods respectively. This is because our service implements only two kinds of RPC methods:<\/p>\n