[perf] Replace MessageToDict with optimized custom dict builder - 4x faster JSON serialization #6013

Current Behavior

The feature server's /get-online-features endpoint uses google.protobuf.json_format.MessageToDict to convert protobuf responses to JSON. MessageToDict is a generic converter designed for any protobuf type, but for the known GetOnlineFeaturesResponse structure, a type-specific converter is 4x faster.

Steps to reproduce

• Deploy Feast feature server to Kubernetes
• Call /get-online-features with 100 entities and 50 features
• Profile the endpoint and isolate MessageToDict time
• Observe ~12ms spent in MessageToDict (98% of serialization time)
• Compare with proto binary serialization which takes only 0.1ms
• Implement custom dict builder and observe 2.9ms (4x improvement)

Specifications

• Feast version: 0.60.0
• Python version: 3.11
• Tested on: OpenShift/Kubernetes with Redis, PostgreSQL, DynamoDB

Benchmark Results

Tested with 100 entities, 50 features in-cluster:

Current (MessageToDict + orjson):  12.0ms
- MessageToDict only:            11.8ms  (98% of time!)
- orjson (dict→JSON):             0.2ms

Custom dict builder + orjson:       2.9ms
-  Custom builder:                 2.7ms
-  orjson (dict→JSON):             0.2ms

Proto binary (for comparison):      0.1ms

Why MessageToDict is Slow

• Reflection-based field discovery for ANY protobuf message
• Dynamic type checking for each field
• Generic handling of all value types (oneofs, repeated, maps)
• CamelCase/snake_case conversion logic
• For GetOnlineFeaturesResponse, we know the exact schema, so all this reflection is unnecessary.

Possible Solution

Add sdk/python/feast/feature_server_utils.py with a custom dict builder