Hi everyone! I am stumbling into some issues with using mlserver 1.6 and Opentelemetry.
When I call:
poetry run opentelemetry-instrument --metrics_exporter none --traces_exporter otlp mlserver start some-project/
With the following envars set:
OTEL_SERVICE_NAME: "mlserver.serve"
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT: "0.0.0.0:4317"
I get:
File "/Users/***/Library/Caches/pypoetry/virtualenvs/some-project-XRSGcY6i-py3.10/lib/python3.10/site-packages/mlserver/cli/main.py", line 48, in start
await server.start(models_settings)
File "/Users/***/Library/Caches/pypoetry/virtualenvs/some-project-XRSGcY6i-py3.10/lib/python3.10/site-packages/mlserver/server.py", line 137, in start
await servers_task
File "/Users***/Library/Caches/pypoetry/virtualenvs/some-project-XRSGcY6i-py3.10/lib/python3.10/site-packages/mlserver/grpc/server.py", line 112, in start
self._create_server()
File "/Users/***/Library/Caches/pypoetry/virtualenvs/some-project-XRSGcY6i-py3.10/lib/python3.10/site-packages/mlserver/grpc/server.py", line 71, in _create_server
self._server = aio.server(
File "/Users/***/Library/Caches/pypoetry/virtualenvs/some-project-XRSGcY6i-py3.10/lib/python3.10/site-packages/opentelemetry/instrumentation/grpc/__init__.py", line 390, in server
kwargs["interceptors"].insert(
AttributeError: 'tuple' object has no attribute 'insert'
It seems like:
https://github.com/open-telemetry/opentelemetry-python-contrib/blob/main/instrumen[…]ntation-grpc/src/opentelemetry/instrumentation/grpc/init.py
Doesn't expect the tuple that is passed from:
https://github.com/SeldonIO/MLServer/blob/master/mlserver/grpc/server.py#L73
Looks like a bug or do you see that there might be something else missing from my side?
Hi everyone! I am stumbling into some issues with using mlserver 1.6 and Opentelemetry.
When I call:
poetry run opentelemetry-instrument --metrics_exporter none --traces_exporter otlp mlserver start some-project/With the following envars set:
I get:
It seems like:
https://github.com/open-telemetry/opentelemetry-python-contrib/blob/main/instrumen[…]ntation-grpc/src/opentelemetry/instrumentation/grpc/init.py
Doesn't expect the tuple that is passed from:
https://github.com/SeldonIO/MLServer/blob/master/mlserver/grpc/server.py#L73
Looks like a bug or do you see that there might be something else missing from my side?