MLFlow 2.6.0 introduced inference params support mlflow/mlflow#9068.
May I suggest a PR with your guidance to:
- Propagate this support to the mlflow runtime for /invocation endpoint only ?
Not for */infer endpoint of V2 Inference Protocole at the moment, as the parameter field of JSON payloads for inference requests doesn't seem to have been standardized yet.
Currently no parameters are defined. As required a future proposal may define one or more standard parameters to allow portable functionality across different inference servers. A server can implement server-specific parameters to provide non-standard capabilities.
See https://kserve.github.io/website/0.10/modelserving/data_plane/v2_protocol/#parameters
So we could have parameter colision, for example, if a model uses a parameter named "content_type", which is used in MLServer to decode data in the payload...
I don't know if I've missed something, but maybe we can open another issue later to solve this problem ?
- And fix this related warning that appears when we call /invocation :
/usr/local/lib/python3.10/site-packages/mlserver_mlflow/runtime.py:128: FutureWarning: mlflow.pyfunc.scoring_server.infer_and_parse_json_input is deprecated since 2.6.0. This method will be removed in a future release. Use infer_and_parse_data instead.
data = infer_and_parse_json_input(raw_body, self._input_schema)
MLFlow 2.6.0 introduced inference params support mlflow/mlflow#9068.
May I suggest a PR with your guidance to:
Not for */infer endpoint of V2 Inference Protocole at the moment, as the parameter field of JSON payloads for inference requests doesn't seem to have been standardized yet.
See https://kserve.github.io/website/0.10/modelserving/data_plane/v2_protocol/#parameters
So we could have parameter colision, for example, if a model uses a parameter named "content_type", which is used in MLServer to decode data in the payload...
I don't know if I've missed something, but maybe we can open another issue later to solve this problem ?
/usr/local/lib/python3.10/site-packages/mlserver_mlflow/runtime.py:128: FutureWarning:
mlflow.pyfunc.scoring_server.infer_and_parse_json_inputis deprecated since 2.6.0. This method will be removed in a future release. Useinfer_and_parse_datainstead.data = infer_and_parse_json_input(raw_body, self._input_schema)