Skip to content

[Tracer] Add API to extract SpanContext#2694

Merged
pierotibou merged 5 commits into
masterfrom
pierre/new-extraction-api
Apr 21, 2022
Merged

[Tracer] Add API to extract SpanContext#2694
pierotibou merged 5 commits into
masterfrom
pierre/new-extraction-api

Conversation

@pierotibou

@pierotibou pierotibou commented Apr 18, 2022

Copy link
Copy Markdown
Contributor

Summary of changes

Introduce an API to allow the extraction of the SpanContext when the Tracer couldn't do it.

Api:

ISpanContext? Extract<TCarrier>(TCarrier carrier, Func<TCarrier, string, IEnumerable<string?>> getter)

Calling code:

var spanContextExtractor = new SpanContextExtractor();
var parentContext = spanContextExtractor.Extract(headers, (headers, key) => GetHeaderValues(headers, key));
var spanCreationSettings = new SpanCreationSettings() { Parent = parentContext }; 
using var scope = Tracer.Instance.StartActive("operation", spanCreationSettings);

Where GetHeaderValues will depend on the underlying system used. Here are a few examples:

// Confluent.Kafka
IEnumerable<string> GetHeaderValues(Headers headers, string name)
{
    if (headers.TryGetLastBytes(name, out var bytes))
    {
        try
        {
            return new[] { Encoding.UTF8.GetString(bytes) };
        }
        catch (Exception)
        {
            // ignored
        }
    }

    return Enumerable.Empty<string>();
}

// RabbitMQ
IEnumerable<string> GetHeaderValues(IDictionary<string, object> headers, string name)
{
    if (headers.TryGetValue(name, out object value) && value is byte[] bytes)
    {
        return new[] { Encoding.UTF8.GetString(bytes) };
    }

    return Enumerable.Empty<string>();
}

Reason for change

In some cases, propagation cannot happen. For instance, Streamiz, a library implemented above the Confluent Kafka, enqueues messages before spans are created, and thus before context can be propagated.
This API gives the liberty of extracting such context and passing it to custom spans.

Implementation details

A new API called SpanContextExtractor calls our SpanContextPropagator given a carrier and a getter. It is basically a public API above an internal one.

I've chosen to make it non static and to add an interface as some of our customers ask to be given ways to mock our APIs. It is located in Datadog.Trace namespace for better usability for users.
We had discussed other ways to provide this API, but this one seemed the most simple and relevant.

Test coverage

No Utests as it would be testing what we already do.
Changed our RabbitMq tests to validate that distributed tracing is working in both cases

Other details

Fixes APMS-7021

@pierotibou
pierotibou requested a review from a team as a code owner April 18, 2022 21:10
@pierotibou pierotibou added the area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations) label Apr 19, 2022
@pierotibou
pierotibou force-pushed the pierre/new-extraction-api branch from 3801c4c to 0721193 Compare April 19, 2022 09:18

@andrewlock andrewlock left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, though not a fan of the name. How about SpanContextExtractor?

Also, although we don't need tests per-se, it would be good to do one or both of:

  • Update the sample app (tracer/samples/RabbitMQ.DistributedTracing) to use this API
  • Maybe update the kafka tests to use the API too?

Comment thread tracer/src/Datadog.Trace/ISpanContextGetter.cs Outdated
Comment thread tracer/src/Datadog.Trace/SpanContextGetter.cs Outdated
Comment thread tracer/src/Datadog.Trace/SpanContextGetter.cs Outdated
@andrewlock

This comment has been minimized.

@pierotibou
pierotibou force-pushed the pierre/new-extraction-api branch from 003fbac to 1b1c81f Compare April 19, 2022 13:05
@andrewlock

This comment has been minimized.

@andrewlock

This comment has been minimized.

Comment thread tracer/src/Datadog.Trace/ISpanContextExtractor.cs Outdated
Comment thread tracer/src/Datadog.Trace/ISpanContextExtractor.cs Outdated
Comment thread tracer/src/Datadog.Trace/SpanContextExtractor.cs Outdated
Comment thread tracer/src/Datadog.Trace/SpanContextExtractor.cs Outdated
@andrewlock

Copy link
Copy Markdown
Member

Benchmarks Report 🐌

Benchmarks for #2694 compared to master:

  • 2 benchmarks are faster, with geometric mean 1.157
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 941μs 4.62μs 20.2μs 0 0 0 3.16 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 735μs 4.25μs 35μs 0 0 0 2.57 KB
#2694 WriteAndFlushEnrichedTraces net472 954μs 5.19μs 31.5μs 0 0 0 3.16 KB
#2694 WriteAndFlushEnrichedTraces netcoreapp3.1 733μs 4.16μs 30.9μs 0 0 0 2.57 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 311ns 1.54ns 6.33ns 0.0661 0 0 417 B
master AllCycleSimpleBody netcoreapp3.1 426ns 2.06ns 8.23ns 0.00586 0 0 416 B
master AllCycleMoreComplexBody net472 310ns 1.45ns 7.25ns 0.0622 0 0 393 B
master AllCycleMoreComplexBody netcoreapp3.1 407ns 2.12ns 9.95ns 0.0055 0 0 392 B
master BodyExtractorSimpleBody net472 438ns 2.28ns 13.9ns 0.057 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 456ns 2.69ns 24.9ns 0.00361 0 0 272 B
master BodyExtractorMoreComplexBody net472 24.3μs 64.4ns 241ns 1.19 0.0118 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 21.2μs 109ns 499ns 0.0944 0 0 6.75 KB
#2694 AllCycleSimpleBody net472 320ns 1.75ns 9.88ns 0.0659 0 0 417 B
#2694 AllCycleSimpleBody netcoreapp3.1 412ns 2.1ns 9.15ns 0.00567 0 0 416 B
#2694 AllCycleMoreComplexBody net472 322ns 1.81ns 12ns 0.0623 0 0 393 B
#2694 AllCycleMoreComplexBody netcoreapp3.1 402ns 2.04ns 10ns 0.00532 0 0 392 B
#2694 BodyExtractorSimpleBody net472 431ns 2.16ns 9.18ns 0.0569 0 0 361 B
#2694 BodyExtractorSimpleBody netcoreapp3.1 429ns 2.02ns 8.07ns 0.00369 0 0 272 B
#2694 BodyExtractorMoreComplexBody net472 24.3μs 115ns 459ns 1.19 0.0124 0 7.62 KB
#2694 BodyExtractorMoreComplexBody netcoreapp3.1 22.3μs 115ns 526ns 0.0871 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 305μs 1.64μs 8.68μs 0.154 0 0 19.74 KB
#2694 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2694 SendRequest netcoreapp3.1 303μs 1.65μs 8.86μs 0.152 0 0 19.74 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.83μs 10.5ns 77.7ns 0.0938 0 0 594 B
master ExecuteNonQuery netcoreapp3.1 1.52μs 8.11ns 43.7ns 0.00912 0 0 632 B
#2694 ExecuteNonQuery net472 1.95μs 11.3ns 97.6ns 0.0939 0.000903 0 594 B
#2694 ExecuteNonQuery netcoreapp3.1 1.69μs 7.9ns 45.4ns 0.00888 0 0 632 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.69μs 15.2ns 113ns 0.125 0 0 802 B
master CallElasticsearch netcoreapp3.1 1.76μs 9.11ns 44.6ns 0.011 0 0 792 B
master CallElasticsearchAsync net472 2.86μs 13.2ns 62.1ns 0.149 0 0 939 B
master CallElasticsearchAsync netcoreapp3.1 1.88μs 9.83ns 50.1ns 0.0129 0 0 912 B
#2694 CallElasticsearch net472 2.89μs 16.2ns 104ns 0.124 0 0 803 B
#2694 CallElasticsearch netcoreapp3.1 1.72μs 9.69ns 62.1ns 0.0109 0 0 792 B
#2694 CallElasticsearchAsync net472 2.95μs 16.9ns 122ns 0.146 0 0 939 B
#2694 CallElasticsearchAsync netcoreapp3.1 1.83μs 9.63ns 48.2ns 0.0124 0 0 912 B
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 3.12μs 18.1ns 169ns 0.166 0 0 1.06 KB
master ExecuteAsync netcoreapp3.1 2.06μs 11.6ns 74ns 0.0141 0 0 1.03 KB
#2694 ExecuteAsync net472 3.13μs 16.5ns 87.2ns 0.168 0.00155 0 1.06 KB
#2694 ExecuteAsync netcoreapp3.1 2.04μs 9.48ns 35.5ns 0.0142 0 0 1.03 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 7.03μs 31.6ns 122ns 0.362 0 0 2.28 KB
master SendAsync netcoreapp3.1 4.92μs 25.8ns 129ns 0.0314 0 0 2.21 KB
#2694 SendAsync net472 7.38μs 42.1ns 385ns 0.354 0 0 2.28 KB
#2694 SendAsync netcoreapp3.1 4.88μs 24ns 107ns 0.0295 0 0 2.21 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 3.51μs 18ns 86.2ns 0.227 0 0 1.45 KB
master EnrichedLog netcoreapp3.1 3.14μs 17.7ns 114ns 0.0208 0 0 1.53 KB
#2694 EnrichedLog net472 3.39μs 18.6ns 108ns 0.227 0 0 1.45 KB
#2694 EnrichedLog netcoreapp3.1 3.13μs 14.7ns 68.8ns 0.0203 0 0 1.53 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 289μs 1.42μs 6.04μs 0.43 0.143 0 4.33 KB
master EnrichedLog netcoreapp3.1 232μs 1.12μs 5.5μs 0 0 0 4.21 KB
#2694 EnrichedLog net472 283μs 1.31μs 5.07μs 0.442 0.147 0 4.33 KB
#2694 EnrichedLog netcoreapp3.1 231μs 1.2μs 5.89μs 0 0 0 4.21 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 8.75μs 48.8ns 297ns 0.505 0 0 3.23 KB
master EnrichedLog netcoreapp3.1 6.82μs 38.3ns 249ns 0.0486 0 0 3.6 KB
#2694 EnrichedLog net472 8.9μs 50.8ns 377ns 0.503 0 0 3.23 KB
#2694 EnrichedLog netcoreapp3.1 6.67μs 37ns 225ns 0.0505 0 0 3.6 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.54μs 14.1ns 86.6ns 0.16 0.00285 0.00143 1.01 KB
master SendReceive netcoreapp3.1 2.3μs 11ns 48ns 0.0141 0 0 1.01 KB
#2694 SendReceive net472 2.47μs 12.7ns 59.6ns 0.16 0.00131 0 1.01 KB
#2694 SendReceive netcoreapp3.1 2.06μs 11.5ns 72.6ns 0.0147 0 0 1.01 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 6.67μs 38.5ns 315ns 0.291 0 0 1.87 KB
master EnrichedLog netcoreapp3.1 5.38μs 23.1ns 89.4ns 0.0209 0 0 1.49 KB
#2694 EnrichedLog net472 6.07μs 33.1ns 182ns 0.292 0 0 1.87 KB
#2694 EnrichedLog netcoreapp3.1 5.23μs 21.5ns 77.7ns 0.0191 0 0 1.49 KB
Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #2694

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1 1.161 1,296.46 1,116.51
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net472 1.152 1,256.46 1,090.36

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 947ns 4.61ns 18.4ns 0.0717 0 0 457 B
master StartFinishSpan netcoreapp3.1 884ns 4.26ns 16.5ns 0.00632 0 0 456 B
master StartFinishScope net472 1.26μs 7.19ns 54.8ns 0.0845 0 0 538 B
master StartFinishScope netcoreapp3.1 1.3μs 6.73ns 32.3ns 0.00786 0 0 576 B
#2694 StartFinishSpan net472 975ns 4.98ns 22.3ns 0.0715 0 0 457 B
#2694 StartFinishSpan netcoreapp3.1 874ns 4.01ns 17.5ns 0.0062 0 0 456 B
#2694 StartFinishScope net472 1.1μs 5.48ns 25.1ns 0.0845 0 0 538 B
#2694 StartFinishScope netcoreapp3.1 1.12μs 3.38ns 13.1ns 0.00769 0 0 576 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.37μs 5.36ns 20.1ns 0.0838 0 0 538 B
master RunOnMethodBegin netcoreapp3.1 1.25μs 7.29ns 65.2ns 0.00798 0 0 576 B
#2694 RunOnMethodBegin net472 1.27μs 5.62ns 28.1ns 0.0845 0 0 538 B
#2694 RunOnMethodBegin netcoreapp3.1 1.24μs 7.02ns 46ns 0.00789 0 0 576 B

@pierotibou
pierotibou force-pushed the pierre/new-extraction-api branch from bd68fa8 to dc57737 Compare April 20, 2022 22:08
@pierotibou
pierotibou force-pushed the pierre/new-extraction-api branch from 61fbf92 to 15eb8c5 Compare April 20, 2022 22:18
@andrewlock

Copy link
Copy Markdown
Member

Code Coverage Report 📊

⚠️ Merging #2694 into master will will decrease line coverage by 2%
⚠️ Merging #2694 into master will will decrease branch coverage by 2%
⛔ Merging #2694 into master will will increase complexity by 21

master #2694 Change
Lines 13581 / 18665 13304 / 18699
Lines % 73% 71% -2% ⚠️
Branches 7830 / 11147 7640 / 11165
Branches % 70% 68% -2% ⚠️
Complexity 12433 12454 21

View the full report for further details:

Datadog.Trace Breakdown ⚠️

master #2694 Change
Lines % 73% 71% -2% ⚠️
Branches % 70% 68% -2% ⚠️
Complexity 12433 12454 21

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.ClrProfiler.AutoInstrumentation.Kafka.Partition -100% -100% 0 ✔️
Datadog.Trace.ClrProfiler.AutoInstrumentation.RabbitMQ.ContextPropagation -100% -100% 0 ✔️
Datadog.Trace.ClrProfiler.AutoInstrumentation.Kafka.KafkaConsumerCloseIntegration -100% 0% ✔️ 0 ✔️
Datadog.Trace.ClrProfiler.AutoInstrumentation.Kafka.KafkaConsumerDisposeIntegration -100% 0% ✔️ 0 ✔️
Datadog.Trace.ClrProfiler.AutoInstrumentation.RabbitMQ.ExchangeDeclareIntegration -100% 0% ✔️ 0 ✔️
Datadog.Trace.ClrProfiler.AutoInstrumentation.RabbitMQ.QueueBindIntegration -100% 0% ✔️ 0 ✔️
Datadog.Trace.ClrProfiler.AutoInstrumentation.RabbitMQ.QueueDeclareIntegration -100% 0% ✔️ 0 ✔️
Datadog.Trace.ClrProfiler.AutoInstrumentation.Kafka.KafkaProduceAsyncIntegration -95% -81% 0 ✔️
Datadog.Trace.ClrProfiler.AutoInstrumentation.Kafka.KafkaConsumerConsumeIntegration -94% -88% 0 ✔️
Datadog.Trace.ClrProfiler.AutoInstrumentation.Kafka.KafkaProduceSyncIntegration -91% -50% 0 ✔️
...And 15 more

The following classes were added in #2694:

File Line coverage Branch coverage Complexity
Datadog.Trace.SpanContextExtractor 0% 100% 1

View the full reports for further details:

@pierotibou
pierotibou merged commit a5656c4 into master Apr 21, 2022
@pierotibou
pierotibou deleted the pierre/new-extraction-api branch April 21, 2022 05:22
@github-actions github-actions Bot added this to the vNext milestone Apr 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations) type:new-feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants