As pointed out by @zdevito in #23569, instead of serializing the entire Python UDF RPC into a byte vector, it should stay consistent in the message layout with Builtin Ops where tensors are stored in a separate tensor table. This design avoids serializing everything all at once, and makes it possible to introduce streaming serialization in the future.

cc @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera