Skip to content

[FEATURE REQ] Distributed tracing for EventHubs: add enqueueTime to Process span links #9599

@lmolkova

Description

@lmolkova

Is your feature request related to a problem? Please describe.
One of the interesting questions for EventHubs tracing is how much time message spent in EventHubs (or when it was enqueued). It helps to detect worker scaling issues.

Today, we trace when message was sent and also when processing has started. This, however, only helps if the same tracing tool is used on both sides AND if users have permissions to access telemetry from both sides.

Also, assuming heavy batching scenarios, it's hard for analysis/visualization tool such as Azure Monitor to calculate and visualize aggregated information about 'time in queue'.

Describe the solution you'd like
EventData provides information about EnqueuedTime already and we should populate it on the telemetry to help customers easily understand how much time message spent in the hub.

Please add enqueuedTime attribute on each link when processing messages: unix epoch time with milliseconds precision representing when message was enqueued (x-opt-enqueued-time system property). Attribute value should have long type.

Describe alternatives you've considered

  1. Populate time in queue as a metric. this requires heavy investment into metrics but otherwise a valid long term approach. However metric does not help answer questions about particular message.

  2. Do not populate, let users log Enqueued time. this won't allow tracing tools to analyze/visualize this value in cases when telemetry from just consumer side is available. It will also complicate analysis for cases when telemetry from producer and consumer is available.

Additional context

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

  • Description Added
  • Expected solution specified

Metadata

Metadata

Assignees

Labels

ClientThis issue points to a problem in the data-plane of the library.Event Hubs

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions