Skip to content

[OpenTracing] Fix context propagation between OpenTracing and Datadog#518

Merged
delner merged 2 commits into
feature/opentracingfrom
bugfix/opentracing_context_between_tracers
Aug 24, 2018
Merged

[OpenTracing] Fix context propagation between OpenTracing and Datadog#518
delner merged 2 commits into
feature/opentracingfrom
bugfix/opentracing_context_between_tracers

Conversation

@delner

@delner delner commented Aug 20, 2018

Copy link
Copy Markdown
Contributor

When using ddtrace OpenTracer in conjunction with an app already instrumented by ruby-rack-tracer, we were seeing some disjoint traces. Spans generated by ddtrace out of the box instrumentation was not ending up parented by the OpenTracing integration parent span (in this case, Rack), resulting in multiple traces that should have appeared as one.

To fix this, the pull request does two things:

  • For distributed tracing oriented OT integrations, they pass SpanContext objects as child_of. We need to make sure the Datadog::Context object that comes from this is set on the tracer, such that the following generated Datadog span inherits the proper trace info (trace ID, parent ID.) It was not actually using this context properly before.
  • For the first OpenTracing span created, regardless of being preceded or followed by Datadog spans, it was not resolving a Datadog context (because it was trying to derive it from a non-existing OT parent span), and thus passing nil to the start_span method which would create a brand new context that would not be re-used on subsequent spans. We now make sure its passing a Datadog::Context to start_span.

@delner delner added bug Involves a bug core Involves Datadog core libraries labels Aug 20, 2018
@delner delner self-assigned this Aug 20, 2018
@delner
delner requested review from palazzem and pawelchcki August 20, 2018 21:42
Comment thread lib/ddtrace/opentracer/tracer.rb Outdated

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This block is responsible for allowing OpenTracer spans to append to traces that were created by Datadog auto-instrumentation. e.g. datadog.span --> open_tracer.span.

@delner
delner force-pushed the bugfix/opentracing_context_between_tracers branch from 3d223ff to fab0f89 Compare August 22, 2018 18:58
Comment thread lib/ddtrace/opentracer/tracer.rb Outdated

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line is responsible for allowing Datadog auto-instrumentation spans to append to traces that were created by OpenTracer. e.g. open_tracer.span --> datadog.span .

Comment thread lib/ddtrace/opentracer/tracer.rb Outdated

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just a refactor from #start_span.

@delner
delner force-pushed the bugfix/opentracing_context_between_tracers branch from fab0f89 to 0612ae0 Compare August 22, 2018 19:06
@delner

delner commented Aug 23, 2018

Copy link
Copy Markdown
Contributor Author

Made some significant changes to how context is resolved and written to. This should be ready for review.

child_of = if scope_manager.active
scope_manager.active.span.context
else
SpanContextFactory.build(datadog_context: datadog_tracer.call_context)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@pawelchcki pawelchcki left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@delner delner changed the title [OpenTracing] Fix Tracer not using child_of SpanContext properly [OpenTracing] Fix context propagation between OpenTracing and Datadog Aug 23, 2018
@delner delner added this to the 0.16.0 milestone Aug 23, 2018
@delner delner mentioned this pull request Aug 23, 2018

@Kyle-Verhoog Kyle-Verhoog left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, I had a nearly identical approach with the Python tracer.

end
end

context 'preceded by a Datadog span' do

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

it { expect(child_datadog_span.trace_id).to eq(parent_datadog_span.trace_id) }
end

context 'followed by a Datadog span' do

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

end

context 'for a nested span' do
context 'when there is no active scope' do

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@delner
delner merged commit 287cc51 into feature/opentracing Aug 24, 2018
@delner
delner deleted the bugfix/opentracing_context_between_tracers branch August 24, 2018 13:29
delner added a commit that referenced this pull request Sep 13, 2018
…#518)

* Fixed: OpenTracer::Tracer not using child_of SpanContext properly.

* Changed: OpenTracer to append its spans to Datadog auto-instrumentation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Involves a bug core Involves Datadog core libraries

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants