Skip to content

[CIVisibility] Adds support to disable the tracer profiler + Lazy loading the tracer settings#3108

Merged
tonyredondo merged 3 commits into
masterfrom
tony/disable-tracer-clr-profiler
Aug 26, 2022
Merged

[CIVisibility] Adds support to disable the tracer profiler + Lazy loading the tracer settings#3108
tonyredondo merged 3 commits into
masterfrom
tony/disable-tracer-clr-profiler

Conversation

@tonyredondo

@tonyredondo tonyredondo commented Aug 22, 2022

Copy link
Copy Markdown
Member

Summary of changes

This PR adds support to disable the tracer profiler on demand and change the tracer settings loading to be lazy (to avoid the PInvoke call to get the RuntimeId made by the DebuggerSettings)

Reason for change

Improve the performance in the CI Visibility scenarios where we want to instrument the testhost child process only.

According to the tests, we are able to cut 738ms in average for the Datadog.Trace.DuckTyping.Tests test suite:

image

@tonyredondo tonyredondo self-assigned this Aug 22, 2022
@tonyredondo
tonyredondo requested review from a team as code owners August 22, 2022 10:39
@tonyredondo tonyredondo added area:native-library Automatic instrumentation native C++ code (Datadog.Trace.ClrProfiler.Native) area:ci-visibility labels Aug 22, 2022
@andrewlock

This comment has been minimized.

Sign in to view
@andrewlock

This comment has been minimized.

Sign in to view
@lucaspimentel lucaspimentel added the type:performance Performance, speed, latency, resource usage (CPU, memory) label Aug 23, 2022
@andrewlock

This comment has been minimized.

Sign in to view
andrewlock
andrewlock approved these changes Aug 26, 2022
View reviewed changes

@andrewlock andrewlock left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It feels wrong that creating an instance of the TracerSettings does a PInvoke to me (because in a custom-instrumentation-only secenario, that will fail), but the workaround seems fine to me, so we should potentially address that separately

@andrewlock

andrewlock commented Aug 26, 2022

Copy link
Copy Markdown
Member

Benchmarks Report 🐌

Benchmarks for #3108 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 716μs 471ns 1.76μs 0.355 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 467μs 338ns 1.26μs 0 0 0 2.59 KB
#3108 WriteAndFlushEnrichedTraces net472 712μs 411ns 1.59μs 0.357 0 0 3.18 KB
#3108 WriteAndFlushEnrichedTraces netcoreapp3.1 460μs 226ns 877ns 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 187ns 0.146ns 0.566ns 0.0676 0 0 425 B
master AllCycleSimpleBody netcoreapp3.1 241ns 0.332ns 1.2ns 0.0058 0 0 424 B
master AllCycleMoreComplexBody net472 185ns 0.144ns 0.559ns 0.0637 0 0 401 B
master AllCycleMoreComplexBody netcoreapp3.1 235ns 0.371ns 1.44ns 0.0054 0 0 400 B
master BodyExtractorSimpleBody net472 253ns 0.2ns 0.775ns 0.0574 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 230ns 0.265ns 0.99ns 0.00373 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.5μs 19.7ns 73.7ns 1.21 0.0216 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.3μs 17.3ns 67ns 0.0863 0 0 6.75 KB
#3108 AllCycleSimpleBody net472 190ns 0.343ns 1.33ns 0.0675 0 0 425 B
#3108 AllCycleSimpleBody netcoreapp3.1 239ns 0.287ns 1.11ns 0.00574 0 0 424 B
#3108 AllCycleMoreComplexBody net472 186ns 0.399ns 1.54ns 0.0637 0 0 401 B
#3108 AllCycleMoreComplexBody netcoreapp3.1 241ns 0.333ns 1.29ns 0.0055 0 0 400 B
#3108 BodyExtractorSimpleBody net472 256ns 0.483ns 1.81ns 0.0574 0 0 361 B
#3108 BodyExtractorSimpleBody netcoreapp3.1 235ns 0.464ns 1.74ns 0.0036 0 0 272 B
#3108 BodyExtractorMoreComplexBody net472 15.3μs 28.2ns 105ns 1.21 0.0152 0 7.62 KB
#3108 BodyExtractorMoreComplexBody netcoreapp3.1 12.6μs 25.7ns 99.5ns 0.0881 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 182μs 201ns 780ns 0.273 0 0 20.57 KB
#3108 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3108 SendRequest netcoreapp3.1 179μs 38.3ns 138ns 0.269 0 0 20.57 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.82μs 0.707ns 2.65ns 0.15 0.000908 0 947 B
master ExecuteNonQuery netcoreapp3.1 1.38μs 0.452ns 1.75ns 0.0124 0 0 936 B
#3108 ExecuteNonQuery net472 1.86μs 0.714ns 2.77ns 0.15 0.000921 0 947 B
#3108 ExecuteNonQuery netcoreapp3.1 1.36μs 0.43ns 1.67ns 0.0124 0 0 936 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.52μs 0.648ns 2.51ns 0.183 0 0 1.16 KB
master CallElasticsearch netcoreapp3.1 1.54μs 0.525ns 2.04ns 0.0147 0 0 1.1 KB
master CallElasticsearchAsync net472 2.67μs 2.83ns 11ns 0.205 0 0 1.29 KB
master CallElasticsearchAsync netcoreapp3.1 1.59μs 0.611ns 2.29ns 0.0167 0 0 1.22 KB
#3108 CallElasticsearch net472 2.43μs 0.66ns 2.47ns 0.183 0 0 1.16 KB
#3108 CallElasticsearch netcoreapp3.1 1.51μs 0.719ns 2.79ns 0.0151 0 0 1.1 KB
#3108 CallElasticsearchAsync net472 2.65μs 2.05ns 7.69ns 0.205 0 0 1.29 KB
#3108 CallElasticsearchAsync netcoreapp3.1 1.61μs 0.413ns 1.54ns 0.0162 0 0 1.22 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.54μs 9.2ns 35.6ns 0.224 0 0 1.41 KB
master ExecuteAsync netcoreapp3.1 1.66μs 4.22ns 16.3ns 0.0178 0 0 1.34 KB
#3108 ExecuteAsync net472 2.76μs 9.29ns 33.5ns 0.224 0 0 1.41 KB
#3108 ExecuteAsync netcoreapp3.1 1.65μs 4.35ns 16.3ns 0.0182 0 0 1.34 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 5.69μs 24.4ns 94.5ns 0.439 0 0 2.77 KB
master SendAsync netcoreapp3.1 3.57μs 9.34ns 36.2ns 0.0351 0 0 2.6 KB
#3108 SendAsync net472 5.7μs 19.3ns 74.6ns 0.438 0 0 2.77 KB
#3108 SendAsync netcoreapp3.1 3.57μs 10.3ns 39.7ns 0.0355 0 0 2.6 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 3.13μs 2.34ns 9.05ns 0.287 0 0 1.81 KB
master EnrichedLog netcoreapp3.1 2.56μs 1.49ns 5.59ns 0.0242 0 0 1.85 KB
#3108 EnrichedLog net472 3.11μs 3.37ns 13ns 0.287 0 0 1.81 KB
#3108 EnrichedLog netcoreapp3.1 2.43μs 1.74ns 6.01ns 0.0246 0 0 1.85 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 152μs 128ns 494ns 0.686 0.229 0 4.65 KB
master EnrichedLog netcoreapp3.1 113μs 74.4ns 268ns 0.057 0 0 4.49 KB
#3108 EnrichedLog net472 152μs 147ns 571ns 0.688 0.229 0 4.65 KB
#3108 EnrichedLog netcoreapp3.1 118μs 135ns 521ns 0.0589 0 0 4.49 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.6μs 13.9ns 53.7ns 0.567 0.00278 0 3.59 KB
master EnrichedLog netcoreapp3.1 4.18μs 12.2ns 47.2ns 0.0538 0 0 3.91 KB
#3108 EnrichedLog net472 5.52μs 14.1ns 54.7ns 0.568 0.00278 0 3.59 KB
#3108 EnrichedLog netcoreapp3.1 4.28μs 22.5ns 110ns 0.0534 0 0 3.91 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.19μs 1.66ns 6.44ns 0.217 0 0 1.37 KB
master SendReceive netcoreapp3.1 1.77μs 0.673ns 2.52ns 0.0177 0 0 1.32 KB
#3108 SendReceive net472 2.25μs 1.93ns 7.49ns 0.217 0 0 1.37 KB
#3108 SendReceive netcoreapp3.1 1.78μs 0.731ns 2.64ns 0.0179 0 0 1.32 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.01μs 1.37ns 5.3ns 0.354 0 0 2.23 KB
master EnrichedLog netcoreapp3.1 4.26μs 2.74ns 10.3ns 0.0236 0 0 1.8 KB
#3108 EnrichedLog net472 4.95μs 1.76ns 6.8ns 0.352 0 0 2.23 KB
#3108 EnrichedLog netcoreapp3.1 4.11μs 1.84ns 6.89ns 0.0246 0 0 1.8 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 1.12μs 0.423ns 1.58ns 0.128 0 0 810 B
master StartFinishSpan netcoreapp3.1 919ns 0.317ns 1.23ns 0.0101 0 0 760 B
master StartFinishScope net472 1.37μs 0.299ns 1.16ns 0.141 0 0 891 B
master StartFinishScope netcoreapp3.1 1.05μs 0.281ns 1.05ns 0.0121 0 0 880 B
#3108 StartFinishSpan net472 1.14μs 0.473ns 1.83ns 0.129 0 0 810 B
#3108 StartFinishSpan netcoreapp3.1 911ns 0.315ns 1.22ns 0.01 0 0 760 B
#3108 StartFinishScope net472 1.38μs 0.35ns 1.31ns 0.141 0 0 891 B
#3108 StartFinishScope netcoreapp3.1 1.03μs 0.299ns 1.08ns 0.0118 0 0 880 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.51μs 0.321ns 1.24ns 0.141 0 0 891 B
master RunOnMethodBegin netcoreapp3.1 1.19μs 0.529ns 2.05ns 0.0119 0 0 880 B
#3108 RunOnMethodBegin net472 1.47μs 0.218ns 0.815ns 0.141 0 0 891 B
#3108 RunOnMethodBegin netcoreapp3.1 1.12μs 0.475ns 1.84ns 0.012 0 0 880 B

@andrewlock

andrewlock commented Aug 26, 2022

Copy link
Copy Markdown
Member

Code Coverage Report 📊

✔️ Merging #3108 into master will not change line coverage
✔️ Merging #3108 into master will not change branch coverage
⛔ Merging #3108 into master will will increase complexity by 3

master #3108 Change
Lines 17154 / 23534 17192 / 23546
Lines % 73% 73% 0% ✔️
Branches 10195 / 14538 10219 / 14540
Branches % 70% 70% 0% ✔️
Complexity 15640 15643 3 ⛔

View the full report for further details:

Datadog.Trace Breakdown ✔️

master #3108 Change
Lines % 73% 73% 0% ✔️
Branches % 70% 70% 0% ✔️
Complexity 15640 15643 3 ⛔

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Agent.DiscoveryService.DiscoveryService -8% ⛔ 0% ✔️ 0 ✔️
Datadog.Trace.Activity.DiagnosticSourceEventListener -6% ⛔ 0% ✔️ 0 ✔️
Datadog.Trace.Ci.CIVisibility 5% ✔️ 6% ✔️ 0 ✔️
Datadog.Trace.Ci.GitInfo 5% ✔️ 8% ✔️ 0 ✔️

View the full reports for further details:

@tonyredondo
tonyredondo merged commit 84c045b into master Aug 26, 2022
@tonyredondo
tonyredondo deleted the tony/disable-tracer-clr-profiler branch August 26, 2022 22:04
@github-actions github-actions Bot added this to the vNext milestone Aug 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@lucaspimentel lucaspimentel lucaspimentel approved these changes
@andrewlock andrewlock andrewlock approved these changes
+1 more reviewer
@nachoBonafonte nachoBonafonte nachoBonafonte approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

@tonyredondo tonyredondo

Labels

area:ci-visibility area:native-library Automatic instrumentation native C++ code (Datadog.Trace.ClrProfiler.Native) type:performance Performance, speed, latency, resource usage (CPU, memory)

Projects

None yet

Milestone

2.15.0

Development

Successfully merging this pull request may close these issues.

4 participants

@tonyredondo @andrewlock @lucaspimentel @nachoBonafonte