Opensearch transport query sanitization#15634
Conversation
| | System property | Type | Default | Description | | ||
| |-----------------------------------------------------------------|---------| ------- |------------------------------------------------------| | ||
| | `otel.instrumentation.opensearch.experimental-span-attributes` | Boolean | `false` | Enable the capture of experimental span attributes. | | ||
| | `otel.instrumentation.opensearch.capture-search-query` | Boolean | `false` | Enable the capture of sanitized search query bodies. | |
There was a problem hiding this comment.
I wonder if it's worth mentioning that this will likely impact performance in some way due to the serialization?
| | `otel.instrumentation.opensearch.capture-search-query` | Boolean | `false` | Enable the capture of sanitized search query bodies. | | |
| | `otel.instrumentation.opensearch.capture-search-query` | Boolean | `false` | Enable the capture of sanitized search query bodies. **Note**: Enabling this feature adds overhead for JSON serialization and parsing on search requests. | |
There was a problem hiding this comment.
@jaydeluca I think so. This feature can add overhead to search requests. The overhead is expected to be slight, but could be unexpected on systems that use a lot of search requests. It's better to clarify this.
- remove duplicated code - fix wrong comment
|
The build failed due to a link validation check on armeria.dev, which is returning 404 not found(line/armeria#6560) |
|
armeria.dev is back online. We apologize for the inconvenience. 🙇 |
| if (request instanceof GenericSerializable) { | ||
| // GenericSerializable writes directly to output stream, cannot sanitize | ||
| // This path is typically not used for search queries | ||
| ByteArrayOutputStream baos = new ByteArrayOutputStream(); | ||
| ((GenericSerializable) request).serialize(baos); | ||
| String body = baos.toString(UTF_8); | ||
| return body.isEmpty() ? null : body; | ||
| } |
There was a problem hiding this comment.
none of the tests seem to use this part
There was a problem hiding this comment.
@laurit since the type of request is checked to be either SearchRequest or MsearchRequest before calling extractSanitized, so the request can't be an instance of GenericSerializable. it was not a dead code before, but it became the dead code after some modification. it should be removed.
| | System property | Type | Default | Description | | ||
| | -------------------------------------------------------------- | ------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | | ||
| | `otel.instrumentation.opensearch.experimental-span-attributes` | Boolean | `false` | Enable the capture of experimental span attributes. | | ||
| | `otel.instrumentation.opensearch.capture-search-query` | Boolean | `false` | Enable the capture of sanitized search query bodies. **Note**: Enabling this feature adds overhead for JSON serialization and parsing on search requests. | |
There was a problem hiding this comment.
Since capturing the query is now simpler than in the initial version I'm wondering whether we should add the performance note at all. Perhaps we should enable this by default?
cc @trask
There was a problem hiding this comment.
seems reasonable, it's definitely useful data
Since extracting query body can impact application performance, I made it optional with
otel.instrumentation.elasticsearch.capture-search-query. Not sure if adding this config is okay though.close #15466