Add endpoint attribute to otlp exporter#14998
Conversation
f78e5b8 to
abaa2ca
Compare
|
If the scope of this PR is to broad, I'm happy to split this into one PR adding the |
endpoint attribute to otlp exporterendpoint attribute to otlp exporter
| exporterhelper.WithQueue(oCfg.QueueConfig), | ||
| exporterhelper.WithStart(oce.start), | ||
| exporterhelper.WithShutdown(oce.shutdown), | ||
| exporterhelper.WithAttrs(attribute.String("endpoint", oCfg.ClientConfig.Endpoint)), |
There was a problem hiding this comment.
Is this the semantic convention for it?
There was a problem hiding this comment.
I considered both server/destination.address but since they don't include the path, they're not a good fit.
url.full would work but IMHO this doesn't clearly communicate that the value is the remote endpoint
There was a problem hiding this comment.
I think we should define something in the semantic conventions before including it here.
There was a problem hiding this comment.
I can do that, which broader scope would be appropriate here? I was thinking maybe otel.component.endpoint (next to otel.component.name/otel.component.type)
Alternatively, extending the server/destination attributes with .path is reasonable as well.
abaa2ca to
859e181
Compare
859e181 to
d6dfcb8
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #14998 +/- ##
==========================================
+ Coverage 91.29% 91.31% +0.02%
==========================================
Files 699 699
Lines 44940 44979 +39
==========================================
+ Hits 41028 41073 +45
+ Misses 2765 2761 -4
+ Partials 1147 1145 -2 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
ee9f6d8 to
8abb683
Compare
Merging this PR will not alter performance
|
8abb683 to
a645664
Compare
916d6c7 to
7e3682c
Compare
|
@dmathieu I've rebased on the latest main & the merge freeze should be over, can you allow the CI to run & merge if everything is good from your side? |
85e56f2 to
0eb460c
Compare
|
@theSuess Could you describe in a little more depth what's currently missing? We instrument the clients for the OTLP exporters with the Go SDK gRPC/HTTP instrumentations. I just verified I see the If possible, I think those instrumentations would be the best place to put anything that should be there, but I may be missing something. |
|
@evan-bradley the Another downside of this set of metrics is that they don't split by exporter so it's not possible to figure out which exporter is sending to which endpoint without parsing the config |
|
Got it, thanks for the explanation. I think I got a bit spun around by the attributes being added, I see why this makes sense. |
evan-bradley
left a comment
There was a problem hiding this comment.
Makes sense to me. Checked the attributes against the HTTP/RPC semconvs and the Slack conversation and they make sense to me.
(There could be an argument for including http.request.method to align with the http.request.* metrics, but it will generally be POST, so maybe there's no benefit)
0eb460c to
89c8104
Compare
evan-bradley
left a comment
There was a problem hiding this comment.
Nit comment about the changelog entry, but looks good to me. Thanks.
89c8104 to
8689a0c
Compare
…ocessortest to v0.151.0 (#65) This PR contains the following updates: | Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) | |---|---|---|---| | [go.opentelemetry.io/collector/processor/processortest](https://github.com/open-telemetry/opentelemetry-collector) | `v0.150.0` → `v0.151.0` |  |  | --- ### Release Notes <details> <summary>open-telemetry/opentelemetry-collector (go.opentelemetry.io/collector/processor/processortest)</summary> ### [`v0.151.0`](https://github.com/open-telemetry/opentelemetry-collector/blob/HEAD/CHANGELOG.md#v1570v01510) [Compare Source](open-telemetry/opentelemetry-collector@v0.150.0...v0.151.0) ##### 🛑 Breaking changes 🛑 - `cmd/builder`: In the generated Collector source, the `replace` statements in the Go module will now use relative paths by default. ([#​15097](open-telemetry/opentelemetry-collector#15097)) We expect that this will not break existing use-cases where the generated collector is only used in an interim manner for builds. It enables the possibility of tracking the generated Collector code as a longer living artifact, allowing it to be run on any machine (whereas absolute paths will be different depending on the machine the Collector source is generated on.) We have added `dist::use_absolute_replace_paths` to go back to the absolute path behaviour in the case where there is an unforeseen use-case that requires absolute paths. - `pkg/confighttp`: Stabilize framedSnappy feature gate. ([#​15096](open-telemetry/opentelemetry-collector#15096)) ##### 💡 Enhancements 💡 - `all`: Add declarative schema support for service telemetry resource configuration. ([#​14411](open-telemetry/opentelemetry-collector#14411)) The `service::telemetry::resource` configuration now accepts the declarative schema with explicit name/value pairs: ```yaml service: telemetry: resource: schema_url: https://opentelemetry.io/schemas/1.38.0 attributes: - name: service.name value: my-collector - name: host.name value: collector-host ``` The legacy inline attribute map format is still supported for backward compatibility: ```yaml service: telemetry: resource: service.name: my-collector host.name: collector-host ``` Note: `resource.detectors` is accepted for forward compatibility but is not yet applied by the collector. - `exporter/otlp_grpc`: Added the `server.address` and `url.path` attributes to metrics generated by the otlp exporter. ([#​14998](open-telemetry/opentelemetry-collector#14998)) - `exporter/otlp_http`: Added the `server.address` and `url.path` attributes to metrics generated by the otlp\_http exporter. ([#​14998](open-telemetry/opentelemetry-collector#14998)) - `pkg/config/configgrpc`: Add `UserAgent` field to `ClientConfig` to allow overriding the default gRPC user-agent string. ([#​14686](open-telemetry/opentelemetry-collector#14686)) The otlp gRPC exporter was unconditionally setting the User-Agent via grpc.WithUserAgent() at dial time, which takes precedence over per-call metadata, causing any user-configured User-Agent to be silently discarded. A dedicated `UserAgent` field has been added to `ClientConfig` which, when set, is used in the dial option directly instead of the default BuildInfo-derived string. - `pkg/config/configgrpc`: Accept gRPC resolver scheme URIs in client endpoint (e.g. passthrough:///host:port) to allow control over name resolution ([#​14990](open-telemetry/opentelemetry-collector#14990)) After the migration to grpc.NewClient, some gRPC client components such as the OTLP exporter experienced connection issues in dual-stack DNS environments. This can now be fixed by using the passthrough:/// gRPC resolver scheme in the endpoint field. - `pkg/config/confignet`: Add support for Windows Named Pipe (npipe) transport ([#​15085](open-telemetry/opentelemetry-collector#15085)) - `pkg/service`: Emit a warning when using the old v0.2.0 declarative config format ([#​15088](open-telemetry/opentelemetry-collector#15088)) ##### 🧰 Bug fixes 🧰 - `pkg/otelcol`: Print components exactly once in the `otelcol components` command ([#​14682](open-telemetry/opentelemetry-collector#14682)) This resolves an issue where aliased components were skipped. - `pkg/otelcol`: Synchronize Collector Run and Shutdown lifecycles so that Shutdown blocks until Run completes all cleanup. ([#​4947](open-telemetry/opentelemetry-collector#4947)) Shutdown now blocks until Run finishes cleanup, matching http.Server semantics. If Shutdown is called before Run, the next Run call returns nil after cleaning up the config provider. - `pkg/pdata`: Use spec-compliant string representation for NaN, Infinity, and -Infinity in Value.AsString(). ([#​14487](open-telemetry/opentelemetry-collector#14487)) - `pkg/pprofile`: Fix data corruption of resource and scope attributes after marshal-unmarshal-merge round-trip. ([#​15084](open-telemetry/opentelemetry-collector#15084)) - `pkg/service`: Non-string resource attributes in telemetry configuration now return an error instead of panicking ([#​15171](open-telemetry/opentelemetry-collector#15171)) - `pkg/xscraperhelper`: fix the merge of profiles in the profiling scraper helpers ([#​14790](open-telemetry/opentelemetry-collector#14790)) - `receiver/otlp`: Fix profiles receiver reporting its samples as spans ([#​15089](open-telemetry/opentelemetry-collector#15089)) <!-- previous-version --> </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My41LjQiLCJ1cGRhdGVkSW5WZXIiOiI0My41LjQiLCJ0YXJnZXRCcmFuY2giOiJtYWluIiwibGFiZWxzIjpbXX0=--> Reviewed-on: https://gitea.t000-n.de/t.behrendt/tracebasedlogsampler/pulls/65 Reviewed-by: t.behrendt <[email protected]> Co-authored-by: Renovate Bot <[email protected]> Co-committed-by: Renovate Bot <[email protected]>
…1.58.0 (#60) This PR contains the following updates: | Package | Change | [Age](https://docs.renovatebot.com/merge-confidence/) | [Confidence](https://docs.renovatebot.com/merge-confidence/) | |---|---|---|---| | [go.opentelemetry.io/collector/confmap](https://github.com/open-telemetry/opentelemetry-collector) | `v1.56.0` → `v1.58.0` |  |  | --- ### Release Notes <details> <summary>open-telemetry/opentelemetry-collector (go.opentelemetry.io/collector/confmap)</summary> ### [`v1.58.0`](https://github.com/open-telemetry/opentelemetry-collector/blob/HEAD/CHANGELOG.md#v1580v01520) ##### 💡 Enhancements 💡 - `pkg/exporterhelper`: Add `otelcol_exporter_in_flight_requests` metric to track the number of export requests currently in-flight per exporter. ([#​15009](open-telemetry/opentelemetry-collector#15009)) This UpDownCounter increments in startOp and decrements in endOp, allowing operators to monitor concurrent export activity and detect when an exporter is saturating its worker pool. ##### 🧰 Bug fixes 🧰 - `pkg/confighttp`: Close the original request body after reading block-format `Content-Encoding: snappy` requests. ([#​15262](open-telemetry/opentelemetry-collector#15262)) - `pkg/confighttp`: Recover from panics in decompression libraries, return HTTP 400 instead of 500. ([#​13228](open-telemetry/opentelemetry-collector#13228)) - `pkg/confighttp`: Enforce `max_request_body_size` on `Content-Encoding: snappy` requests before the decoded buffer is allocated. ([#​15252](open-telemetry/opentelemetry-collector#15252)) - `pkg/otelcol`: Stop emitting verbose gRPC transport messages at WARN during normal client disconnect. ([#​5169](open-telemetry/opentelemetry-collector#5169)) grpc-go gates chatty per-RPC notices (e.g. "HandleStreams failed to read frame: connection reset by peer") behind `LoggerV2.V(2)`. zapgrpc.Logger.V conflates grpclog verbosity with zap severity, so V(2) returns true whenever WARN is enabled and these messages emit at WARN. Wrap the installed grpclog.LoggerV2 with a corrected V() that compares against a fixed verbosity threshold, matching grpclog's intended semantics. See [uber-go/zap#1544](uber-go/zap#1544). - `pkg/pdata`: `pcommon.Value.AsString` no longer HTML-escapes `<`, `>`, and `&` inside `ValueTypeMap` and `ValueTypeSlice` values, matching the behavior already used for `ValueTypeStr`. ([#​14662](open-telemetry/opentelemetry-collector#14662)) - `pkg/service`: Fix Prometheus config defaults mismatch when host is explicitly set in telemetry configuration. ([#​13867](open-telemetry/opentelemetry-collector#13867)) When users explicitly configured the telemetry metrics section (e.g. to change the host), the Prometheus exporter boolean fields (WithoutScopeInfo, WithoutUnits, WithoutTypeSuffix) defaulted to nil/false instead of true, causing metric name format changes compared to the implicit default configuration. This fix applies the correct defaults during config unmarshaling. - `pkg/service`: Return noop tracer provider when no trace processors are defined ([#​15135](open-telemetry/opentelemetry-collector#15135)) <!-- previous-version --> ### [`v1.57.0`](https://github.com/open-telemetry/opentelemetry-collector/blob/HEAD/CHANGELOG.md#v1570v01510) ##### 🛑 Breaking changes 🛑 - `cmd/builder`: In the generated Collector source, the `replace` statements in the Go module will now use relative paths by default. ([#​15097](open-telemetry/opentelemetry-collector#15097)) We expect that this will not break existing use-cases where the generated collector is only used in an interim manner for builds. It enables the possibility of tracking the generated Collector code as a longer living artifact, allowing it to be run on any machine (whereas absolute paths will be different depending on the machine the Collector source is generated on.) We have added `dist::use_absolute_replace_paths` to go back to the absolute path behaviour in the case where there is an unforeseen use-case that requires absolute paths. - `pkg/confighttp`: Stabilize framedSnappy feature gate. ([#​15096](open-telemetry/opentelemetry-collector#15096)) ##### 💡 Enhancements 💡 - `all`: Add declarative schema support for service telemetry resource configuration. ([#​14411](open-telemetry/opentelemetry-collector#14411)) The `service::telemetry::resource` configuration now accepts the declarative schema with explicit name/value pairs: ```yaml service: telemetry: resource: schema_url: https://opentelemetry.io/schemas/1.38.0 attributes: - name: service.name value: my-collector - name: host.name value: collector-host ``` The legacy inline attribute map format is still supported for backward compatibility: ```yaml service: telemetry: resource: service.name: my-collector host.name: collector-host ``` Note: `resource.detectors` is accepted for forward compatibility but is not yet applied by the collector. - `exporter/otlp_grpc`: Added the `server.address` and `url.path` attributes to metrics generated by the otlp exporter. ([#​14998](open-telemetry/opentelemetry-collector#14998)) - `exporter/otlp_http`: Added the `server.address` and `url.path` attributes to metrics generated by the otlp\_http exporter. ([#​14998](open-telemetry/opentelemetry-collector#14998)) - `pkg/config/configgrpc`: Add `UserAgent` field to `ClientConfig` to allow overriding the default gRPC user-agent string. ([#​14686](open-telemetry/opentelemetry-collector#14686)) The otlp gRPC exporter was unconditionally setting the User-Agent via grpc.WithUserAgent() at dial time, which takes precedence over per-call metadata, causing any user-configured User-Agent to be silently discarded. A dedicated `UserAgent` field has been added to `ClientConfig` which, when set, is used in the dial option directly instead of the default BuildInfo-derived string. - `pkg/config/configgrpc`: Accept gRPC resolver scheme URIs in client endpoint (e.g. passthrough:///host:port) to allow control over name resolution ([#​14990](open-telemetry/opentelemetry-collector#14990)) After the migration to grpc.NewClient, some gRPC client components such as the OTLP exporter experienced connection issues in dual-stack DNS environments. This can now be fixed by using the passthrough:/// gRPC resolver scheme in the endpoint field. - `pkg/config/confignet`: Add support for Windows Named Pipe (npipe) transport ([#​15085](open-telemetry/opentelemetry-collector#15085)) - `pkg/service`: Emit a warning when using the old v0.2.0 declarative config format ([#​15088](open-telemetry/opentelemetry-collector#15088)) ##### 🧰 Bug fixes 🧰 - `pkg/otelcol`: Print components exactly once in the `otelcol components` command ([#​14682](open-telemetry/opentelemetry-collector#14682)) This resolves an issue where aliased components were skipped. - `pkg/otelcol`: Synchronize Collector Run and Shutdown lifecycles so that Shutdown blocks until Run completes all cleanup. ([#​4947](open-telemetry/opentelemetry-collector#4947)) Shutdown now blocks until Run finishes cleanup, matching http.Server semantics. If Shutdown is called before Run, the next Run call returns nil after cleaning up the config provider. - `pkg/pdata`: Use spec-compliant string representation for NaN, Infinity, and -Infinity in Value.AsString(). ([#​14487](open-telemetry/opentelemetry-collector#14487)) - `pkg/pprofile`: Fix data corruption of resource and scope attributes after marshal-unmarshal-merge round-trip. ([#​15084](open-telemetry/opentelemetry-collector#15084)) - `pkg/service`: Non-string resource attributes in telemetry configuration now return an error instead of panicking ([#​15171](open-telemetry/opentelemetry-collector#15171)) - `pkg/xscraperhelper`: fix the merge of profiles in the profiling scraper helpers ([#​14790](open-telemetry/opentelemetry-collector#14790)) - `receiver/otlp`: Fix profiles receiver reporting its samples as spans ([#​15089](open-telemetry/opentelemetry-collector#15089)) <!-- previous-version --> </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My41LjQiLCJ1cGRhdGVkSW5WZXIiOiI0My41LjQiLCJ0YXJnZXRCcmFuY2giOiJtYWluIiwibGFiZWxzIjpbXX0=--> Reviewed-on: https://gitea.t000-n.de/t.behrendt/tracebasedlogsampler/pulls/60 Reviewed-by: t.behrendt <[email protected]> Co-authored-by: Renovate Bot <[email protected]> Co-committed-by: Renovate Bot <[email protected]>
Description
This PR exposes the
endpointvalue of theotlpandotlphttpexporter components to the metrics generated by these exporters.This is useful for operators of the collector to observe data flow to different endpoints across large numbers of collector instances without having to cross-reference the specific configuration per instance.
The implementation extends the
exporterhelperwith a new optionWithAttrsthat allows developers of exporters to add extra labels to metrics produced. This allows for more flexibility for implementers of exporters to add attributes that might be relevant in the specific use case.Testing
endpointshows up correctly.Documentation
Assuming this change is accepted, I'll create a PR to update the internal telemetry page in the OpenTelemetry documentation