Describe the enhancement requested

We use Arrow flight over GRPC in C++ to request large volumes of data from another service. This data is streamed to us using potentially large individual messages. Each request uses its own grpc::channel.
To guarantee the stability of our service, we need to be able to track the memory that is allocated for a request. When reading a message, we see that the process memory intermittently increases by the message’s size. Since we have messages of multiple megabytes, we need to be able to track these internal allocations.
It is important for us that this memory tracking happens before the allocation, such that, we can decide to either allow or forbid the allocation based on our memory budget.

Our problems could be solved with some kind of custom allocator support or if we could get the size of the next incoming message before it is completely read into memory.

Using GRPC’s NextMessageSize() does not fit our needs as it always returns the set MaxReceiveMessageSize of the underlying GRPC channel. However, we would need to know the actual upcoming message size because our maximum message size is a lot larger than what we expect to receive in the “normal” cases. Always expecting the maximum message size would lead to large amounts of unused memory being held in reserve by our system.

We also investigated whether this problem can be solved on the GRPC level but did not find a satisfying solution. We opened an issue for this as well.

Component(s)

C++, FlightRPC