Skip to content

Add OpenTelemetry annotations support#5593

Merged
PerfectSlayer merged 7 commits into
masterfrom
bbujon/otel-annotations
Aug 23, 2023
Merged

Add OpenTelemetry annotations support#5593
PerfectSlayer merged 7 commits into
masterfrom
bbujon/otel-annotations

Conversation

@PerfectSlayer

@PerfectSlayer PerfectSlayer commented Jul 18, 2023

Copy link
Copy Markdown
Contributor

What Does This Do

This PR brings initial support for OpenTelemetry @WithSpan, @SpanAttributes and @AddingSpanAttributes annotations.

  • @WithSpan annotation is similar to the @Trace annotation and create span for the annotated method,
  • @SpanAttritubes annotation can be attached to a method parameter to store its value as span tag
    (this requires the related method to be annotated either with @WithSpan attribute or @AddingSpanAttributes annotation),
  • @AddingSpanAttributes will allow to use @SpanAttributes without creating a new span, adding span tag for the current active span if any (doing nothing if no active span).

Check the official documentation for more details.

Motivation

This should improve our OpenTelemetry compatibility.

Additional Notes

@PerfectSlayer PerfectSlayer added type: feature Enhancements and improvements inst: opentelemetry OpenTelemetry instrumentation labels Jul 18, 2023
@pr-commenter

pr-commenter Bot commented Jul 18, 2023

Copy link
Copy Markdown

Benchmarks

Parameters

Baseline Candidate
commit 1.20.0-SNAPSHOT~44d47a398d 1.19.0-SNAPSHOT~b11bc2a6bc
config baseline candidate
See matching parameters
Baseline Candidate
module Agent Agent
parent None None

Summary

Found 3 performance improvements and 0 performance regressions! Performance is the same for 19 cases.

scenario Δ mean execution_time
scenario:Startup-base-Telemetry better
[-616.882µs; -478.244µs] or [-10.438%; -8.092%]
scenario:Startup-iast-Telemetry better
[-735.556µs; -362.339µs] or [-12.669%; -6.241%]
scenario:Startup-waf-Telemetry better
[-706.859µs; -640.213µs] or [-12.089%; -10.949%]

@PerfectSlayer
PerfectSlayer force-pushed the bbujon/otel-annotations branch from 7f229e7 to 871717b Compare July 19, 2023 07:42
@PerfectSlayer
PerfectSlayer marked this pull request as ready for review July 19, 2023 12:44
@PerfectSlayer
PerfectSlayer requested a review from a team as a code owner July 19, 2023 12:44
}
} else if (result instanceof CompletionStage<?>) {
CompletionStage<?> completionStage = (CompletionStage<?>) result;
completionStage.whenComplete(finishSpan(span));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't we be returning the new CompletionStage which also finishes the span?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed. It should otherwise the span will be closed twice (the first time too early, and the second time as expected).

The tests didn't catch the issue because they contain an issue about span duration test I fix in the next PR... #5737
I wanted to avoid update this old PR but I guess it's time to do it and andl backport the test fix (and your add the CompletionStage return).

@PerfectSlayer
PerfectSlayer force-pushed the bbujon/otel-annotations branch from 8ac8f53 to 8bf0cb2 Compare August 21, 2023 14:51
@AlexeiZenin

AlexeiZenin commented Nov 13, 2023

Copy link
Copy Markdown

So does @Trace (datadog.trace.api) work with Mono/Flux or do I still have to manually instrument the actual execution time vs time taken during assembly of the flux? Right now I added DD's @Trace annotation and it seems to only track the time taken to construct the flux not the time taken as it was executing during subscription.

It seems I would have to wrap all my code in ugly Mono.defers() to get the tracer context etc which leads to a lot of code changes just to get some custom spans injected. Any sample code if that is the case?

@PerfectSlayer

Copy link
Copy Markdown
Contributor Author

Hello @AlexeiZenin

It should be working but you would have to enable dd.trace.annotation.async first.
It was documented in the related PR #5802:

As a behavior change, this is not the default behavior despite it should be closer of the expected behavior.
The feature must be enabled by setting dd.trace.annotation.async to true.

If not, I would recommend to contact the support so we can properly track the missing feature.

@AlexeiZenin

AlexeiZenin commented Nov 14, 2023

Copy link
Copy Markdown

I set the DD_TRACE_ANNOTATION_ASYNC env var, seems it still takes microseconds in my spans. Wondering if it has to be set as a property on the command line.

@PerfectSlayer

Copy link
Copy Markdown
Contributor Author

Assuming you are running tracer version 1.21+, setting DD_TRACE_ANNOTATION_ASYNC=true in your environment variable should trigger the async support for trace annotation.

@bric3 bric3 added the tag: concurrency Virtual Threads, Coroutines, Async, RX, Executors label Feb 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

inst: opentelemetry OpenTelemetry instrumentation tag: concurrency Virtual Threads, Coroutines, Async, RX, Executors type: feature Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants