Skip to content

[Tracer] Add telemetry for the stats computation feature#3053

Merged
zacharycmontoya merged 1 commit into
masterfrom
zach/stats/telemetry
Aug 23, 2022
Merged

[Tracer] Add telemetry for the stats computation feature#3053
zacharycmontoya merged 1 commit into
masterfrom
zach/stats/telemetry

Conversation

@zacharycmontoya

@zacharycmontoya zacharycmontoya commented Aug 4, 2022

Copy link
Copy Markdown
Contributor

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

@zacharycmontoya zacharycmontoya added the area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations) label Aug 4, 2022
@zacharycmontoya zacharycmontoya self-assigned this Aug 4, 2022
@zacharycmontoya
zacharycmontoya requested a review from a team as a code owner August 4, 2022 18:35
@andrewlock

andrewlock commented Aug 4, 2022

Copy link
Copy Markdown
Member

Benchmarks Report 🐌

Benchmarks for #3053 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.280
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 722μs 326ns 1.26μs 0.359 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 478μs 344ns 1.33μs 0 0 0 2.58 KB
#3053 WriteAndFlushEnrichedTraces net472 717μs 425ns 1.59μs 0.357 0 0 3.18 KB
#3053 WriteAndFlushEnrichedTraces netcoreapp3.1 466μs 209ns 809ns 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #3053

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.AppSecBodyBenchmark.AllCycleSimpleBody‑net472 1.280 2,208.74 1,725.19

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 2.21μs 2.52ns 9.44ns 0.237 0 0 1.49 KB
master AllCycleSimpleBody netcoreapp3.1 1.8μs 2.11ns 8.16ns 0.0186 0 0 1.37 KB
master AllCycleMoreComplexBody net472 17.1μs 14.3ns 53.6ns 1.38 0.0171 0 8.75 KB
master AllCycleMoreComplexBody netcoreapp3.1 14.2μs 17.1ns 63.8ns 0.107 0 0 7.85 KB
master BodyExtractorSimpleBody net472 277ns 0.284ns 1.02ns 0.0573 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 227ns 0.136ns 0.509ns 0.00376 0 0 272 B
master BodyExtractorMoreComplexBody net472 15.1μs 6.97ns 26.1ns 1.21 0.0151 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12μs 16.6ns 62.2ns 0.0902 0 0 6.75 KB
#3053 AllCycleSimpleBody net472 1.73μs 1.44ns 5.59ns 0.237 0 0 1.49 KB
#3053 AllCycleSimpleBody netcoreapp3.1 1.83μs 6.74ns 23.3ns 0.0189 0 0 1.37 KB
#3053 AllCycleMoreComplexBody net472 16.8μs 14.5ns 56.1ns 1.38 0.025 0 8.75 KB
#3053 AllCycleMoreComplexBody netcoreapp3.1 14.4μs 27.4ns 106ns 0.107 0 0 7.85 KB
#3053 BodyExtractorSimpleBody net472 275ns 0.386ns 1.5ns 0.0574 0 0 361 B
#3053 BodyExtractorSimpleBody netcoreapp3.1 225ns 0.227ns 0.88ns 0.00372 0 0 272 B
#3053 BodyExtractorMoreComplexBody net472 14.9μs 10.6ns 39.8ns 1.2 0.0149 0 7.62 KB
#3053 BodyExtractorMoreComplexBody netcoreapp3.1 11.9μs 14.8ns 51.2ns 0.0891 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 177μs 225ns 872ns 0.266 0 0 20.33 KB
#3053 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3053 SendRequest netcoreapp3.1 178μs 81.1ns 304ns 0.267 0 0 20.33 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.57μs 0.513ns 1.92ns 0.126 0.000785 0 794 B
master ExecuteNonQuery netcoreapp3.1 1.22μs 0.346ns 1.29ns 0.011 0 0 824 B
#3053 ExecuteNonQuery net472 1.56μs 0.557ns 2.08ns 0.126 0.000772 0 794 B
#3053 ExecuteNonQuery netcoreapp3.1 1.24μs 0.651ns 2.44ns 0.0111 0 0 824 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.28μs 0.705ns 2.73ns 0.159 0 0 1 KB
master CallElasticsearch netcoreapp3.1 1.43μs 0.613ns 2.3ns 0.0136 0 0 984 B
master CallElasticsearchAsync net472 2.43μs 0.81ns 3.14ns 0.18 0 0 1.14 KB
master CallElasticsearchAsync netcoreapp3.1 1.49μs 0.281ns 1.09ns 0.0149 0 0 1.1 KB
#3053 CallElasticsearch net472 2.1μs 0.619ns 2.4ns 0.159 0 0 1 KB
#3053 CallElasticsearch netcoreapp3.1 1.38μs 0.855ns 3.31ns 0.013 0 0 984 B
#3053 CallElasticsearchAsync net472 2.36μs 0.83ns 3.21ns 0.18 0 0 1.14 KB
#3053 CallElasticsearchAsync netcoreapp3.1 1.45μs 0.374ns 1.35ns 0.0145 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.34μs 5.11ns 19.8ns 0.199 0 0 1.26 KB
master ExecuteAsync netcoreapp3.1 1.59μs 2.72ns 10.2ns 0.0164 0 0 1.22 KB
#3053 ExecuteAsync net472 2.48μs 1.49ns 5.76ns 0.2 0 0 1.26 KB
#3053 ExecuteAsync netcoreapp3.1 1.59μs 0.939ns 3.38ns 0.0167 0 0 1.22 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 4.88μs 8.01ns 31ns 0.393 0 0 2.48 KB
master SendAsync netcoreapp3.1 3.28μs 7.51ns 29.1ns 0.0324 0 0 2.36 KB
#3053 SendAsync net472 5.04μs 3.5ns 13.6ns 0.392 0 0 2.48 KB
#3053 SendAsync netcoreapp3.1 3.21μs 1.79ns 6.69ns 0.0305 0 0 2.36 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 2.89μs 0.935ns 3.5ns 0.263 0 0 1.66 KB
master EnrichedLog netcoreapp3.1 2.29μs 0.961ns 3.72ns 0.0239 0 0 1.73 KB
#3053 EnrichedLog net472 2.86μs 0.982ns 3.8ns 0.263 0 0 1.66 KB
#3053 EnrichedLog netcoreapp3.1 2.35μs 2.13ns 8.24ns 0.0234 0 0 1.73 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 146μs 93.9ns 364ns 0.66 0.22 0 4.5 KB
master EnrichedLog netcoreapp3.1 113μs 82.4ns 308ns 0.056 0 0 4.38 KB
#3053 EnrichedLog net472 150μs 58.8ns 220ns 0.67 0.223 0 4.5 KB
#3053 EnrichedLog netcoreapp3.1 114μs 236ns 912ns 0 0 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.32μs 12.7ns 49.4ns 0.545 0.00262 0 3.43 KB
master EnrichedLog netcoreapp3.1 4.22μs 6.58ns 25.5ns 0.0512 0 0 3.8 KB
#3053 EnrichedLog net472 5.36μs 3.35ns 12.5ns 0.544 0.00267 0 3.43 KB
#3053 EnrichedLog netcoreapp3.1 4.32μs 3.06ns 11ns 0.0517 0 0 3.8 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 1.98μs 1.18ns 4.43ns 0.194 0 0 1.22 KB
master SendReceive netcoreapp3.1 1.61μs 0.467ns 1.75ns 0.0162 0 0 1.21 KB
#3053 SendReceive net472 2.06μs 0.812ns 3.04ns 0.194 0 0 1.22 KB
#3053 SendReceive netcoreapp3.1 1.64μs 0.568ns 2.2ns 0.0164 0 0 1.21 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.78μs 1.93ns 7.46ns 0.329 0 0 2.08 KB
master EnrichedLog netcoreapp3.1 4.12μs 1.73ns 6.48ns 0.0227 0 0 1.69 KB
#3053 EnrichedLog net472 4.67μs 1.85ns 7.16ns 0.329 0 0 2.08 KB
#3053 EnrichedLog netcoreapp3.1 4.08μs 2.21ns 8.58ns 0.0223 0 0 1.69 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 860ns 0.59ns 2.28ns 0.105 0 0 658 B
master StartFinishSpan netcoreapp3.1 733ns 0.251ns 0.906ns 0.00875 0 0 648 B
master StartFinishScope net472 1.05μs 0.702ns 2.63ns 0.117 0 0 738 B
master StartFinishScope netcoreapp3.1 881ns 0.501ns 1.87ns 0.0101 0 0 768 B
#3053 StartFinishSpan net472 894ns 0.365ns 1.41ns 0.104 0 0 658 B
#3053 StartFinishSpan netcoreapp3.1 759ns 0.502ns 1.94ns 0.00864 0 0 648 B
#3053 StartFinishScope net472 1.04μs 0.194ns 0.727ns 0.117 0 0 738 B
#3053 StartFinishScope netcoreapp3.1 879ns 0.363ns 1.41ns 0.0101 0 0 768 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.16μs 0.7ns 2.71ns 0.117 0 0 738 B
master RunOnMethodBegin netcoreapp3.1 1.04μs 0.506ns 1.75ns 0.0104 0 0 768 B
#3053 RunOnMethodBegin net472 1.27μs 0.473ns 1.83ns 0.117 0 0 738 B
#3053 RunOnMethodBegin netcoreapp3.1 1.1μs 0.302ns 1.09ns 0.0104 0 0 768 B

@andrewlock

andrewlock commented Aug 5, 2022

Copy link
Copy Markdown
Member

Code Coverage Report 📊

✔️ Merging #3053 into master will not change line coverage
✔️ Merging #3053 into master will not change branch coverage
✔️ Merging #3053 into master will not change complexity

master #3053 Change
Lines 16792 / 22859 16801 / 22860
Lines % 73% 73% 0% ✔️
Branches 10000 / 14184 10010 / 14184
Branches % 71% 71% 0% ✔️
Complexity 15245 15245 0 ✔️

View the full report for further details:

Datadog.Trace Breakdown ✔️

master #3053 Change
Lines % 73% 73% 0% ✔️
Branches % 71% 71% 0% ✔️
Complexity 15245 15245 0 ✔️

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Agent.Transports.HttpStreamRequestFactory -12% ⛔ 0% ✔️ 0 ✔️
Datadog.Trace.Telemetry.Transports.JsonTelemetryTransport -8% ⛔ -8% ⛔ 0 ✔️
Datadog.Trace.Ci.CIVisibility 7% ✔️ 7% ✔️ 0 ✔️

View the full reports for further details:

@andrewlock andrewlock mentioned this pull request Aug 8, 2022
Merged
3 tasks
andrewlock
andrewlock approved these changes Aug 8, 2022
View reviewed changes
Comment thread tracer/src/Datadog.Trace/Telemetry/Collectors/ConfigurationTelemetryCollector.cs
var settings = _settings.Settings;

var data = new List<TelemetryValue>(_azureApServicesMetadata.IsRelevant ? 21 : 17)
var data = new List<TelemetryValue>(_azureApServicesMetadata.IsRelevant ? 27 : 23)

@andrewlock andrewlock Aug 8, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops, nice catch

Comment thread tracer/src/Datadog.Trace/Telemetry/Collectors/ConfigurationTelemetryCollector.cs
new(ConfigTelemetryData.ActivityListenerEnabled, value: settings.IsActivityListenerEnabled),
new(ConfigTelemetryData.ProfilerLoaded, value: _profiler?.Status.IsProfilerReady),
new(ConfigTelemetryData.CodeHotspotsEnabled, value: _profiler?.ContextTracker.IsEnabled),
new(ConfigTelemetryData.StatsComputationEnabled, value: settings.StatsComputationEnabled),

@andrewlock andrewlock Aug 8, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we record the send interval too?

@zacharycmontoya zacharycmontoya Aug 23, 2022

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the non-response! For now, I'd prefer not to because this value should rarely ever be set by the customer and we don't offer a toggle for it in the Tracer right now

@zacharycmontoya
zacharycmontoya merged commit 2b32bda into master Aug 23, 2022
@zacharycmontoya
zacharycmontoya deleted the zach/stats/telemetry branch August 23, 2022 18:38
@github-actions github-actions Bot added this to the vNext milestone Aug 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@andrewlock andrewlock andrewlock approved these changes
@pierotibou pierotibou pierotibou approved these changes

Assignees

@zacharycmontoya zacharycmontoya

Labels

area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations)

Projects

None yet

Milestone

2.15.0

Development

Successfully merging this pull request may close these issues.

3 participants

@zacharycmontoya @andrewlock @pierotibou