Describe your environment
python3.8
opentelemetry-instrumentation-elasticsearch==0.41b0
Steps to reproduce
The elastic library converts msearch queries into a series of stringified json objects, which the otlp instrumentation library then complains about since it tries to decode them and flatten the objects. Here's an example code snippet that causes the behavior -
# regular ES library, happens to be version 7.17.9
from elasticsearch import Elasticsearch
# OTLP libraries, instrumentation=0.41b0, sdk=1.20.0
from opentelemetry.instrumentation.elasticsearch import ElasticsearchInstrumentor
from opentelemetry.exporter.otlp.proto.grpc.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.resources import Resource
from opentelemetry.sdk.trace.export import ConsoleSpanExporter
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.trace import set_tracer_provider
# setup tracer and span exporting
tracer_provider = TracerProvider()
set_tracer_provider(tracer_provider)
span_exporter = OTLPSpanExporter()
tracer_provider.add_span_processor(BatchSpanProcessor(span_exporter))
# generic ES constructor
es = Elasticsearch(['https://my_es_server:9200'])
# create multiple queries, doesn't really matter what's in here as long
# as they're syntactically valid. It has to be more than 1 query due to the way the ES
# library concatenates the stringified versions of the queries
queries = []
queries.append({"index": "index1"})
queries.append({"size": 0,
"query": {
"bool": {
"filter": [
{
"range": {
"start": {
"gte": 123,
"lte": 456,
"format": "epoch_millis"
}
}
},
{
"query_string": {
"analyze_wildcard": True,
"query": "(foo: bar)"
}
}
]
}
}
})
# enable ES instrumentation
ElasticsearchInstrumentor().instrument()
# try to msearch multiple queries. Kaboom
res = es.msearch(body=queries)
What is the expected behavior?
I expect the elastic queries to execute and the responses returned, along with traces emitted.
What is the actual behavior?
The code ends up throwing an exception.
Traceback (most recent call last):
File "example.py", line 50, in <module>
res = es.msearch(body=queries)
File "./lib/python3.8/site-packages/elasticsearch/client/utils.py", line 347, in _wrapped
return func(*args, params=params, headers=headers, **kwargs)
File "./lib/python3.8/site-packages/elasticsearch/client/__init__.py", line 1226, in msearch
return self.transport.perform_request(
File "./lib/python3.8/site-packages/opentelemetry/instrumentation/elasticsearch/__init__.py", line 242, in wrapper
attributes[SpanAttributes.DB_STATEMENT] = sanitize_body(
File "./lib/python3.8/site-packages/opentelemetry/instrumentation/elasticsearch/utils.py", line 56, in sanitize_body
body = json.loads(body)
File "/usr/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.8/json/decoder.py", line 340, in decode
raise JSONDecodeError("Extra data", s, end)
Additional context
Here's the relevant bit where the ES library is converting multiple queries into a bulk request. The latest version in main handles this quite differently since the transport library was split out, but it looks like it still serializes the queries so msearch is likely broken there as well.
https://github.com/elastic/elasticsearch-py/blob/7.17/elasticsearch/client/utils.py#L354
Describe your environment
python3.8
opentelemetry-instrumentation-elasticsearch==0.41b0
Steps to reproduce
The elastic library converts msearch queries into a series of stringified json objects, which the otlp instrumentation library then complains about since it tries to decode them and flatten the objects. Here's an example code snippet that causes the behavior -
What is the expected behavior?
I expect the elastic queries to execute and the responses returned, along with traces emitted.
What is the actual behavior?
The code ends up throwing an exception.
Additional context
Here's the relevant bit where the ES library is converting multiple queries into a bulk request. The latest version in main handles this quite differently since the transport library was split out, but it looks like it still serializes the queries so msearch is likely broken there as well.
https://github.com/elastic/elasticsearch-py/blob/7.17/elasticsearch/client/utils.py#L354