Skip to content

[Debugger] Resolving Live Debugger's code coverage degradation#3245

Merged
dudikeleti merged 2 commits into
masterfrom
matang/fixing-coverage-issue
Sep 22, 2022
Merged

[Debugger] Resolving Live Debugger's code coverage degradation#3245
dudikeleti merged 2 commits into
masterfrom
matang/fixing-coverage-issue

Conversation

@GreenMatan

@GreenMatan GreenMatan commented Sep 21, 2022 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary of changes

In #3164, our sample Samples.Probes accidently referenced Datadog.Trace.csproj and it affected our code coverage severely as our code coverage tool is patching the assembly on disk to perform the and apparently picked the wrong Datadog.Trace.dll.

The second change in this PR is renaming Live Debugger's (now called Dynamic Instrumentation) environment variables enablement from DD_DEBUGGER_* -> DD_DYNAMIC_INSTRUMENTATION_*.

@GreenMatan
GreenMatan requested a review from a team as a code owner September 21, 2022 16:31
@github-actions github-actions Bot added area:debugger area:tests unit tests, integration tests labels Sep 21, 2022
@GreenMatan
GreenMatan requested review from a team as code owners September 21, 2022 16:35
@GreenMatan GreenMatan changed the title [Debugger] Resolving Live Debugger's code coverage degradation [Debugger] Resolving Live Debugger's code coverage degradation + Renamed DD_DEBUGGER_* -> DD_DYNAMIC_INSTRUMENTATION_* Sep 21, 2022
@GreenMatan GreenMatan changed the title [Debugger] Resolving Live Debugger's code coverage degradation + Renamed DD_DEBUGGER_* -> DD_DYNAMIC_INSTRUMENTATION_* [Debugger] Resolving Live Debugger's code coverage degradation Sep 21, 2022
@andrewlock

andrewlock commented Sep 21, 2022

Copy link
Copy Markdown
Member

Benchmarks Report 🐌

Benchmarks for #3245 compared to master:

  • 1 benchmarks are slower, with geometric mean 1.131
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 710μs 506ns 1.96μs 0.357 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 459μs 275ns 1.06μs 0 0 0 2.58 KB
#3245 WriteAndFlushEnrichedTraces net472 722μs 531ns 2.06μs 0.357 0 0 3.18 KB
#3245 WriteAndFlushEnrichedTraces netcoreapp3.1 470μs 356ns 1.38μs 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #3245

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.AppSecBodyBenchmark.BodyExtractorSimpleBody‑net472 1.131 255.04 288.41

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 182ns 0.155ns 0.599ns 0.0675 9.18E-05 0 425 B
master AllCycleSimpleBody netcoreapp3.1 239ns 0.154ns 0.575ns 0.00577 0 0 424 B
master AllCycleMoreComplexBody net472 182ns 0.179ns 0.67ns 0.0637 0 0 401 B
master AllCycleMoreComplexBody netcoreapp3.1 245ns 0.304ns 1.18ns 0.00557 0 0 400 B
master BodyExtractorSimpleBody net472 255ns 0.209ns 0.783ns 0.0574 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 216ns 0.251ns 0.938ns 0.00367 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.8μs 9.99ns 36ns 1.21 0.0148 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12μs 18.8ns 72.7ns 0.09 0 0 6.75 KB
#3245 AllCycleSimpleBody net472 195ns 0.433ns 1.68ns 0.0675 0 0 425 B
#3245 AllCycleSimpleBody netcoreapp3.1 250ns 0.377ns 1.46ns 0.00578 0 0 424 B
#3245 AllCycleMoreComplexBody net472 195ns 0.32ns 1.24ns 0.0637 0 0 401 B
#3245 AllCycleMoreComplexBody netcoreapp3.1 256ns 0.323ns 1.25ns 0.00538 0 0 400 B
#3245 BodyExtractorSimpleBody net472 288ns 1.14ns 4.42ns 0.0573 0 0 361 B
#3245 BodyExtractorSimpleBody netcoreapp3.1 234ns 0.341ns 1.32ns 0.00365 0 0 272 B
#3245 BodyExtractorMoreComplexBody net472 15.6μs 28.2ns 109ns 1.21 0.0155 0 7.62 KB
#3245 BodyExtractorMoreComplexBody netcoreapp3.1 12.9μs 19.7ns 76.4ns 0.0891 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 178μs 163ns 588ns 0.268 0 0 20.58 KB
#3245 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3245 SendRequest netcoreapp3.1 181μs 191ns 713ns 0.271 0 0 20.6 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.84μs 1.08ns 4.17ns 0.15 0 0 947 B
master ExecuteNonQuery netcoreapp3.1 1.55μs 1.26ns 4.87ns 0.0129 0 0 936 B
#3245 ExecuteNonQuery net472 1.86μs 0.685ns 2.65ns 0.15 0 0 947 B
#3245 ExecuteNonQuery netcoreapp3.1 1.45μs 0.577ns 2.23ns 0.0124 0 0 936 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.5μs 1.45ns 5.43ns 0.183 0 0 1.16 KB
master CallElasticsearch netcoreapp3.1 1.56μs 0.979ns 3.66ns 0.0148 0 0 1.1 KB
master CallElasticsearchAsync net472 2.59μs 1.71ns 6.61ns 0.205 0 0 1.29 KB
master CallElasticsearchAsync netcoreapp3.1 1.69μs 0.671ns 2.42ns 0.0162 0 0 1.22 KB
#3245 CallElasticsearch net472 2.67μs 1.45ns 5.6ns 0.183 0 0 1.16 KB
#3245 CallElasticsearch netcoreapp3.1 1.59μs 0.768ns 2.77ns 0.0142 0 0 1.1 KB
#3245 CallElasticsearchAsync net472 2.73μs 2.84ns 11ns 0.205 0 0 1.29 KB
#3245 CallElasticsearchAsync netcoreapp3.1 1.57μs 1.18ns 4.4ns 0.0166 0 0 1.22 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.82μs 11ns 42.7ns 0.225 0 0 1.42 KB
master ExecuteAsync netcoreapp3.1 1.71μs 1.27ns 4.91ns 0.0179 0 0 1.34 KB
#3245 ExecuteAsync net472 2.86μs 2.27ns 8.48ns 0.226 0 0 1.42 KB
#3245 ExecuteAsync netcoreapp3.1 1.76μs 0.723ns 2.8ns 0.0184 0 0 1.34 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 5.85μs 3.31ns 12.8ns 0.438 0 0 2.77 KB
master SendAsync netcoreapp3.1 3.66μs 3.8ns 14.2ns 0.0349 0 0 2.6 KB
#3245 SendAsync net472 5.92μs 2.75ns 10.6ns 0.44 0 0 2.77 KB
#3245 SendAsync netcoreapp3.1 3.64μs 2.48ns 8.95ns 0.0347 0 0 2.6 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 3.24μs 3.36ns 13ns 0.288 0 0 1.81 KB
master EnrichedLog netcoreapp3.1 2.52μs 1.48ns 5.74ns 0.0252 0 0 1.85 KB
#3245 EnrichedLog net472 3.18μs 5.1ns 19.7ns 0.287 0 0 1.81 KB
#3245 EnrichedLog netcoreapp3.1 2.57μs 3.63ns 14.1ns 0.0245 0 0 1.85 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 152μs 155ns 600ns 0.685 0.228 0 4.65 KB
master EnrichedLog netcoreapp3.1 118μs 213ns 823ns 0.0579 0 0 4.49 KB
#3245 EnrichedLog net472 150μs 199ns 770ns 0.694 0.231 0 4.65 KB
#3245 EnrichedLog netcoreapp3.1 119μs 191ns 739ns 0.0592 0 0 4.49 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.72μs 14ns 54.3ns 0.57 0.00285 0 3.59 KB
master EnrichedLog netcoreapp3.1 4.39μs 3.4ns 12.3ns 0.0532 0.00213 0 3.91 KB
#3245 EnrichedLog net472 5.87μs 2.33ns 8.7ns 0.568 0.00294 0 3.59 KB
#3245 EnrichedLog netcoreapp3.1 4.43μs 3.13ns 12.1ns 0.0534 0 0 3.91 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.38μs 3.65ns 14.1ns 0.218 0 0 1.37 KB
master SendReceive netcoreapp3.1 1.76μs 0.62ns 2.4ns 0.0185 0 0 1.32 KB
#3245 SendReceive net472 2.32μs 2.83ns 10.9ns 0.218 0 0 1.37 KB
#3245 SendReceive netcoreapp3.1 1.88μs 1.02ns 3.8ns 0.0176 0 0 1.32 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 4.95μs 1.44ns 4.97ns 0.353 0 0 2.23 KB
master EnrichedLog netcoreapp3.1 4.3μs 3.04ns 11.4ns 0.0236 0 0 1.8 KB
#3245 EnrichedLog net472 5.12μs 2.05ns 7.66ns 0.354 0 0 2.23 KB
#3245 EnrichedLog netcoreapp3.1 4.4μs 1.32ns 5.12ns 0.0242 0 0 1.8 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 1.15μs 0.711ns 2.75ns 0.129 0 0 810 B
master StartFinishSpan netcoreapp3.1 985ns 0.614ns 2.38ns 0.0103 0 0 760 B
master StartFinishScope net472 1.45μs 0.422ns 1.52ns 0.141 0 0 891 B
master StartFinishScope netcoreapp3.1 1.05μs 0.371ns 1.44ns 0.0116 0 0 880 B
#3245 StartFinishSpan net472 1.1μs 0.6ns 2.24ns 0.128 0 0 810 B
#3245 StartFinishSpan netcoreapp3.1 924ns 0.359ns 1.34ns 0.0102 0 0 760 B
#3245 StartFinishScope net472 1.46μs 0.72ns 2.79ns 0.141 0 0 891 B
#3245 StartFinishScope netcoreapp3.1 1.07μs 0.502ns 1.88ns 0.0118 0 0 880 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.58μs 1.19ns 4.62ns 0.142 0 0 891 B
master RunOnMethodBegin netcoreapp3.1 1.18μs 0.571ns 2.21ns 0.0118 0 0 880 B
#3245 RunOnMethodBegin net472 1.51μs 0.793ns 2.86ns 0.141 0 0 891 B
#3245 RunOnMethodBegin netcoreapp3.1 1.19μs 0.615ns 2.22ns 0.0119 0 0 880 B

pierotibou
pierotibou approved these changes Sep 22, 2022
View reviewed changes
Comment thread tracer/src/Datadog.Trace/Configuration/ConfigurationKeys.Debugger.cs
@dudikeleti
dudikeleti merged commit f540639 into master Sep 22, 2022
@dudikeleti
dudikeleti deleted the matang/fixing-coverage-issue branch September 22, 2022 08:00
@github-actions github-actions Bot added this to the vNext milestone Sep 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@dudikeleti dudikeleti dudikeleti approved these changes
@pierotibou pierotibou pierotibou approved these changes

Assignees

No one assigned

Labels

area:debugger area:tests unit tests, integration tests

Projects

None yet

Milestone

2.15.0

Development

Successfully merging this pull request may close these issues.

4 participants

@GreenMatan @andrewlock @dudikeleti @pierotibou