Skip to content

Enable tool integration tests on Windows#3149

Merged
kevingosse merged 4 commits into
masterfrom
kevin/tool_tests
Sep 2, 2022
Merged

Enable tool integration tests on Windows#3149
kevingosse merged 4 commits into
masterfrom
kevin/tool_tests

Conversation

@kevingosse

@kevingosse kevingosse commented Sep 1, 2022 •
edited
Loading

Copy link
Copy Markdown
Contributor

Summary of changes

Add the required trait on the tool integration tests to run them on Windows.

@github-actions github-actions Bot added the area:tests unit tests, integration tests label Sep 1, 2022
@andrewlock

This comment has been minimized.

Sign in to view
@andrewlock

This comment has been minimized.

Sign in to view
@andrewlock

This comment has been minimized.

Sign in to view
@kevingosse kevingosse changed the title Try fixing tool integration tests Enable tool integration tests on Windows Sep 1, 2022
@kevingosse
kevingosse marked this pull request as ready for review September 2, 2022 08:08
@kevingosse
kevingosse requested a review from a team as a code owner September 2, 2022 08:08
@andrewlock

This comment has been minimized.

Sign in to view
@andrewlock

This comment has been minimized.

Sign in to view
@andrewlock

This comment has been minimized.

Sign in to view
andrewlock
andrewlock approved these changes Sep 2, 2022
View reviewed changes

@andrewlock andrewlock left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:nice:

Comment thread ...tadog.Trace.Tools.Runner.IntegrationTests/Datadog.Trace.Tools.Runner.IntegrationTests.csproj Outdated
@kevingosse @andrewlock
Update tracer/test/Datadog.Trace.Tools.Runner.IntegrationTests/Datado…
27629c5 
…g.Trace.Tools.Runner.IntegrationTests.csproj

Co-authored-by: Andrew Lock <[email protected]>
@andrewlock

andrewlock commented Sep 2, 2022

Copy link
Copy Markdown
Member

Benchmarks Report 🐌

Benchmarks for #3149 compared to master:

  • All benchmarks have the same speed
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 706μs 1.18μs 4.56μs 0.355 0 0 3.18 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 493μs 681ns 2.55μs 0 0 0 2.58 KB
#3149 WriteAndFlushEnrichedTraces net472 724μs 1.04μs 4.03μs 0.355 0 0 3.18 KB
#3149 WriteAndFlushEnrichedTraces netcoreapp3.1 463μs 517ns 2μs 0 0 0 2.58 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 184ns 0.233ns 0.841ns 0.0675 9.25E-05 0 425 B
master AllCycleSimpleBody netcoreapp3.1 239ns 0.298ns 1.15ns 0.00577 0 0 424 B
master AllCycleMoreComplexBody net472 184ns 0.198ns 0.741ns 0.0637 0 0 401 B
master AllCycleMoreComplexBody netcoreapp3.1 234ns 0.285ns 1.1ns 0.00562 0 0 400 B
master BodyExtractorSimpleBody net472 265ns 0.249ns 0.963ns 0.0574 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 221ns 0.243ns 0.877ns 0.00366 0 0 272 B
master BodyExtractorMoreComplexBody net472 14.2μs 12.2ns 45.5ns 1.21 0.0212 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 12.3μs 6.75ns 23.4ns 0.0923 0 0 6.75 KB
#3149 AllCycleSimpleBody net472 184ns 0.308ns 1.11ns 0.0676 9.2E-05 0 425 B
#3149 AllCycleSimpleBody netcoreapp3.1 237ns 0.203ns 0.785ns 0.00582 0 0 424 B
#3149 AllCycleMoreComplexBody net472 185ns 0.155ns 0.599ns 0.0637 0 0 401 B
#3149 AllCycleMoreComplexBody netcoreapp3.1 238ns 0.268ns 1.04ns 0.00548 0 0 400 B
#3149 BodyExtractorSimpleBody net472 260ns 0.439ns 1.7ns 0.0573 0 0 361 B
#3149 BodyExtractorSimpleBody netcoreapp3.1 223ns 0.225ns 0.873ns 0.00368 0 0 272 B
#3149 BodyExtractorMoreComplexBody net472 14.3μs 8.36ns 32.4ns 1.21 0.0214 0 7.62 KB
#3149 BodyExtractorMoreComplexBody netcoreapp3.1 12.2μs 12.9ns 46.4ns 0.0935 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Unknown 🤷 Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 183μs 126ns 489ns 0.273 0 0 20.57 KB
#3149 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#3149 SendRequest netcoreapp3.1 183μs 156ns 564ns 0.184 0 0 20.57 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.81μs 3.61ns 14ns 0.15 0.000889 0 947 B
master ExecuteNonQuery netcoreapp3.1 1.51μs 1.67ns 6.48ns 0.0128 0 0 936 B
#3149 ExecuteNonQuery net472 1.78μs 3.13ns 12.1ns 0.15 0.000883 0 947 B
#3149 ExecuteNonQuery netcoreapp3.1 1.42μs 1.07ns 4.14ns 0.0121 0 0 936 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.54μs 1.43ns 5.53ns 0.183 0 0 1.16 KB
master CallElasticsearch netcoreapp3.1 1.51μs 0.615ns 2.3ns 0.0153 0 0 1.1 KB
master CallElasticsearchAsync net472 2.73μs 0.493ns 1.91ns 0.205 0 0 1.29 KB
master CallElasticsearchAsync netcoreapp3.1 1.62μs 0.828ns 3.1ns 0.0162 0 0 1.22 KB
#3149 CallElasticsearch net472 2.55μs 1.05ns 4.08ns 0.184 0 0 1.16 KB
#3149 CallElasticsearch netcoreapp3.1 1.58μs 3.8ns 14.7ns 0.0148 0 0 1.1 KB
#3149 CallElasticsearchAsync net472 2.62μs 1.74ns 6.74ns 0.205 0 0 1.29 KB
#3149 CallElasticsearchAsync netcoreapp3.1 1.62μs 0.799ns 2.99ns 0.0162 0 0 1.22 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 2.54μs 4.45ns 16.7ns 0.224 0 0 1.41 KB
master ExecuteAsync netcoreapp3.1 1.72μs 3.89ns 14.5ns 0.0179 0 0 1.34 KB
#3149 ExecuteAsync net472 2.66μs 12.8ns 51.3ns 0.224 0 0 1.41 KB
#3149 ExecuteAsync netcoreapp3.1 1.7μs 2.59ns 9.69ns 0.0181 0 0 1.34 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 5.76μs 14.2ns 54.9ns 0.438 0 0 2.77 KB
master SendAsync netcoreapp3.1 3.62μs 8.85ns 34.3ns 0.0343 0 0 2.6 KB
#3149 SendAsync net472 5.62μs 9.4ns 35.2ns 0.438 0 0 2.77 KB
#3149 SendAsync netcoreapp3.1 3.6μs 10.8ns 40.2ns 0.0357 0 0 2.6 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 3.06μs 2.98ns 11.5ns 0.287 0 0 1.81 KB
master EnrichedLog netcoreapp3.1 2.47μs 1.59ns 5.96ns 0.0246 0 0 1.85 KB
#3149 EnrichedLog net472 3.17μs 3.54ns 13.2ns 0.287 0 0 1.81 KB
#3149 EnrichedLog netcoreapp3.1 2.49μs 0.771ns 2.89ns 0.0248 0 0 1.85 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 152μs 88.3ns 342ns 0.683 0.228 0 4.65 KB
master EnrichedLog netcoreapp3.1 116μs 138ns 533ns 0 0 0 4.49 KB
#3149 EnrichedLog net472 151μs 96.2ns 360ns 0.681 0.227 0 4.65 KB
#3149 EnrichedLog netcoreapp3.1 116μs 186ns 720ns 0.0584 0 0 4.49 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.66μs 9.49ns 36.7ns 0.568 0.00284 0 3.59 KB
master EnrichedLog netcoreapp3.1 4.39μs 7.56ns 28.3ns 0.0547 0 0 3.91 KB
#3149 EnrichedLog net472 5.63μs 16.7ns 64.8ns 0.567 0.00285 0 3.59 KB
#3149 EnrichedLog netcoreapp3.1 4.49μs 9.12ns 34.1ns 0.0534 0 0 3.91 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.22μs 1.12ns 4.18ns 0.218 0 0 1.37 KB
master SendReceive netcoreapp3.1 1.88μs 7.85ns 30.4ns 0.0176 0 0 1.32 KB
#3149 SendReceive net472 2.32μs 2.83ns 11ns 0.217 0 0 1.37 KB
#3149 SendReceive netcoreapp3.1 1.77μs 2.22ns 8.3ns 0.0178 0 0 1.32 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 5.06μs 0.834ns 3.12ns 0.352 0 0 2.23 KB
master EnrichedLog netcoreapp3.1 4.22μs 0.964ns 3.73ns 0.0233 0 0 1.8 KB
#3149 EnrichedLog net472 5.18μs 2.61ns 9.39ns 0.353 0 0 2.23 KB
#3149 EnrichedLog netcoreapp3.1 4.35μs 2.06ns 7.98ns 0.0242 0 0 1.8 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 1.18μs 0.447ns 1.67ns 0.129 0 0 810 B
master StartFinishSpan netcoreapp3.1 1.01μs 0.213ns 0.767ns 0.0101 0 0 760 B
master StartFinishScope net472 1.37μs 0.645ns 2.41ns 0.141 0 0 891 B
master StartFinishScope netcoreapp3.1 1.09μs 0.316ns 1.14ns 0.0116 0 0 880 B
#3149 StartFinishSpan net472 1.19μs 0.538ns 2.01ns 0.128 0 0 810 B
#3149 StartFinishSpan netcoreapp3.1 934ns 0.401ns 1.55ns 0.0102 0 0 760 B
#3149 StartFinishScope net472 1.44μs 0.36ns 1.39ns 0.141 0 0 891 B
#3149 StartFinishScope netcoreapp3.1 1.07μs 0.291ns 1.05ns 0.0118 0 0 880 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.51μs 0.623ns 2.41ns 0.141 0 0 891 B
master RunOnMethodBegin netcoreapp3.1 1.25μs 0.423ns 1.58ns 0.0119 0 0 880 B
#3149 RunOnMethodBegin net472 1.49μs 0.777ns 3.01ns 0.141 0 0 891 B
#3149 RunOnMethodBegin netcoreapp3.1 1.17μs 0.581ns 2.25ns 0.0123 0 0 880 B

pierotibou
pierotibou approved these changes Sep 2, 2022
View reviewed changes

@pierotibou pierotibou left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙇

@andrewlock

andrewlock commented Sep 2, 2022

Copy link
Copy Markdown
Member

Code Coverage Report 📊

✔️ Merging #3149 into master will not change line coverage
✔️ Merging #3149 into master will not change branch coverage
✔️ Merging #3149 into master will not change complexity

master #3149 Change
Lines 17462 / 24055 17417 / 24055
Lines % 73% 72% 0% ✔️
Branches 10318 / 14797 10310 / 14797
Branches % 70% 70% 0% ✔️
Complexity 15977 15977 0 ✔️

View the full report for further details:

Datadog.Trace Breakdown ✔️

master #3149 Change
Lines % 73% 72% 0% ✔️
Branches % 70% 70% 0% ✔️
Complexity 15977 15977 0 ✔️

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Agent.Transports.HttpClientResponse -23% ⛔ -38% ⛔ 0 ✔️
Datadog.Trace.Ci.GitInfo -17% ⛔ -11% ⛔ 0 ✔️
Datadog.Trace.Ci.CIVisibility 5% ✔️ 6% ✔️ 0 ✔️

View the full reports for further details:

@kevingosse
kevingosse merged commit fdc203d into master Sep 2, 2022
@kevingosse
kevingosse deleted the kevin/tool_tests branch September 2, 2022 15:23
@github-actions github-actions Bot added this to the vNext milestone Sep 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@andrewlock andrewlock andrewlock approved these changes
@pierotibou pierotibou pierotibou approved these changes

Assignees

No one assigned

Labels

area:tests unit tests, integration tests

Projects

None yet

Milestone

2.15.0

Development

Successfully merging this pull request may close these issues.

3 participants

@kevingosse @andrewlock @pierotibou