Skip to content

Make running integration tests on OSx great again#2708

Merged
pierotibou merged 1 commit into
masterfrom
pierre/native-loader-osx
Apr 21, 2022
Merged

Make running integration tests on OSx great again#2708
pierotibou merged 1 commit into
masterfrom
pierre/native-loader-osx

Conversation

@pierotibou

@pierotibou pierotibou commented Apr 20, 2022

Copy link
Copy Markdown
Contributor

Summary of changes

Allow running linux ITests locally on Mac, ie build nativeloader on this OS.

Reason for change

I have a Mac

Implementation details

Made a few changes for the native loader to build in MacOs (some variables renaming and declaring runtimeid_store definition). Thank you @gleocadie for the help ;)
Made a few modifications in the build process and EnvironmentHelper.

Test coverage

Tested locally

@pierotibou
pierotibou requested a review from a team as a code owner April 20, 2022 19:40
@pierotibou pierotibou changed the title Make the OSx integration tests great again Make running integration tests on OSx great again Apr 20, 2022
@pierotibou
pierotibou force-pushed the pierre/native-loader-osx branch from 4c3da90 to a320455 Compare April 20, 2022 19:42
@pierotibou pierotibou added the area:builds project files, build scripts, pipelines, versioning, releases, packages label Apr 20, 2022
andrewlock
andrewlock approved these changes Apr 20, 2022
View reviewed changes

@andrewlock andrewlock left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work, I can feel your pain 😂

@andrewlock

andrewlock commented Apr 20, 2022

Copy link
Copy Markdown
Member

Code Coverage Report 📊

✔️ Merging #2708 into master will not change line coverage
✔️ Merging #2708 into master will not change branch coverage
⛔ Merging #2708 into master will will increase complexity by 37

master #2708 Change
Lines 13549 / 18651 13622 / 18698
Lines % 73% 73% 0% ✔️
Branches 7821 / 11131 7874 / 11165
Branches % 70% 71% 0% ✔️
Complexity 12416 12453 37 ⛔

View the full report for further details:

Datadog.Trace Breakdown ✔️

master #2708 Change
Lines % 73% 73% 0% ✔️
Branches % 70% 71% 0% ✔️
Complexity 12416 12453 37 ⛔

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Telemetry.JsonHttpClientTelemetryTransport -26% ⛔ -38% ⛔ 0 ✔️
Datadog.Trace.ClrProfiler.NativeMethods 2% ✔️ 9% ✔️ 6 ⛔
Datadog.Trace.ClrProfiler.AutoInstrumentation.TraceAnnotations.TraceAnnotationInfoFactory 5% ✔️ 31% ✔️ 15 ⛔
Datadog.Trace.ClrProfiler.InstrumentationDefinitions 6% ✔️ 0% ✔️ 1 ⛔
Datadog.Trace.AppSec.Waf.Initialization.WafConfigurator 6% ✔️ 7% ✔️ 4 ⛔
Datadog.Trace.Ci.CIVisibility 7% ✔️ 7% ✔️ 0 ✔️
Datadog.Trace.Telemetry.JsonWebRequestTelemetryTransport 27% ✔️ 57% ✔️ 0 ✔️

View the full reports for further details:

zacharycmontoya
zacharycmontoya approved these changes Apr 20, 2022
View reviewed changes

@zacharycmontoya zacharycmontoya left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for doing this!

@andrewlock

andrewlock commented Apr 21, 2022

Copy link
Copy Markdown
Member

Benchmarks Report 🐌

Benchmarks for #2708 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.196
  • 1 benchmarks are slower, with geometric mean 1.214
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 947μs 4.93μs 24.7μs 0 0 0 3.16 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 738μs 3.8μs 19μs 0 0 0 2.57 KB
#2708 WriteAndFlushEnrichedTraces net472 945μs 3.71μs 14.4μs 0 0 0 3.16 KB
#2708 WriteAndFlushEnrichedTraces netcoreapp3.1 737μs 3.57μs 19.5μs 0 0 0 2.57 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 329ns 1.63ns 7.46ns 0.066 0 0 417 B
master AllCycleSimpleBody netcoreapp3.1 425ns 2.39ns 15.7ns 0.00556 0 0 416 B
master AllCycleMoreComplexBody net472 319ns 1.81ns 13.1ns 0.0622 0 0 393 B
master AllCycleMoreComplexBody netcoreapp3.1 407ns 2.11ns 12.3ns 0.00532 0 0 392 B
master BodyExtractorSimpleBody net472 453ns 2.55ns 16.7ns 0.0569 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 431ns 1.39ns 5.19ns 0.0039 0 0 272 B
master BodyExtractorMoreComplexBody net472 24.5μs 113ns 423ns 1.18 0.0131 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 21.8μs 119ns 735ns 0.0946 0 0 6.75 KB
#2708 AllCycleSimpleBody net472 332ns 1.12ns 4.19ns 0.066 0 0 417 B
#2708 AllCycleSimpleBody netcoreapp3.1 427ns 1.55ns 5.99ns 0.00562 0 0 416 B
#2708 AllCycleMoreComplexBody net472 335ns 1.92ns 14.5ns 0.062 0 0 393 B
#2708 AllCycleMoreComplexBody netcoreapp3.1 405ns 1.1ns 4.24ns 0.00541 0 0 392 B
#2708 BodyExtractorSimpleBody net472 459ns 2.33ns 11.4ns 0.0569 0 0 361 B
#2708 BodyExtractorSimpleBody netcoreapp3.1 450ns 2.04ns 7.9ns 0.00384 0 0 272 B
#2708 BodyExtractorMoreComplexBody net472 24.8μs 118ns 472ns 1.19 0.0123 0 7.62 KB
#2708 BodyExtractorMoreComplexBody netcoreapp3.1 21.9μs 116ns 581ns 0.0865 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 294μs 1.44μs 5.76μs 0.145 0 0 19.74 KB
#2708 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2708 SendRequest netcoreapp3.1 295μs 1.13μs 4.37μs 0.146 0 0 19.74 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.86μs 8.06ns 27.9ns 0.0936 0.000918 0 594 B
master ExecuteNonQuery netcoreapp3.1 1.56μs 8.18ns 43.3ns 0.00825 0 0 632 B
#2708 ExecuteNonQuery net472 1.87μs 5.5ns 20.6ns 0.0938 0 0 594 B
#2708 ExecuteNonQuery netcoreapp3.1 1.51μs 7.21ns 38.1ns 0.00911 0 0 632 B
Benchmarks.Trace.ElasticsearchBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #2708

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑net472 1.196 3,079.24 2,573.65

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 3.11μs 16.5ns 82.6ns 0.127 0 0 803 B
master CallElasticsearch netcoreapp3.1 1.85μs 9.87ns 68.3ns 0.0109 0 0 792 B
master CallElasticsearchAsync net472 3.17μs 16.6ns 77.9ns 0.149 0.00149 0 939 B
master CallElasticsearchAsync netcoreapp3.1 1.92μs 10.6ns 62.9ns 0.0123 0 0 912 B
#2708 CallElasticsearch net472 2.58μs 10.2ns 39.3ns 0.127 0 0 802 B
#2708 CallElasticsearch netcoreapp3.1 1.64μs 6.46ns 23.3ns 0.0106 0 0 792 B
#2708 CallElasticsearchAsync net472 3.05μs 17.4ns 123ns 0.146 0 0 939 B
#2708 CallElasticsearchAsync netcoreapp3.1 1.9μs 10.9ns 78.2ns 0.0128 0 0 912 B
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 3.1μs 15.8ns 72.5ns 0.168 0.00151 0 1.06 KB
master ExecuteAsync netcoreapp3.1 2.1μs 11.4ns 60.3ns 0.0138 0 0 1.03 KB
#2708 ExecuteAsync net472 3.37μs 9.57ns 33.1ns 0.167 0.00166 0 1.06 KB
#2708 ExecuteAsync netcoreapp3.1 2.33μs 8.45ns 30.5ns 0.0138 0 0 1.03 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 6.69μs 29.2ns 113ns 0.361 0 0 2.28 KB
master SendAsync netcoreapp3.1 5.06μs 23.5ns 99.5ns 0.031 0 0 2.21 KB
#2708 SendAsync net472 6.95μs 35ns 160ns 0.361 0 0 2.28 KB
#2708 SendAsync netcoreapp3.1 4.86μs 17.4ns 67.3ns 0.03 0 0 2.21 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 3.51μs 14.7ns 65.7ns 0.226 0 0 1.45 KB
master EnrichedLog netcoreapp3.1 3.12μs 10.4ns 40.2ns 0.0216 0 0 1.53 KB
#2708 EnrichedLog net472 3.51μs 18.1ns 86.9ns 0.228 0 0 1.45 KB
#2708 EnrichedLog netcoreapp3.1 3.12μs 9.63ns 36ns 0.0211 0 0 1.53 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 285μs 1.35μs 5.56μs 0.434 0.145 0 4.33 KB
master EnrichedLog netcoreapp3.1 231μs 1.23μs 6.6μs 0 0 0 4.21 KB
#2708 EnrichedLog net472 284μs 1.33μs 5.34μs 0.411 0.137 0 4.33 KB
#2708 EnrichedLog netcoreapp3.1 226μs 735ns 2.65μs 0 0 0 4.21 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 9.02μs 50.1ns 321ns 0.502 0 0 3.23 KB
master EnrichedLog netcoreapp3.1 6.6μs 36.3ns 224ns 0.0503 0 0 3.6 KB
#2708 EnrichedLog net472 9.01μs 51.8ns 398ns 0.503 0 0 3.23 KB
#2708 EnrichedLog netcoreapp3.1 6.59μs 36.8ns 232ns 0.0492 0 0 3.6 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.37μs 8.2ns 30.7ns 0.16 0 0 1.01 KB
master SendReceive netcoreapp3.1 2.17μs 10.4ns 44ns 0.0138 0 0 1.01 KB
#2708 SendReceive net472 2.3μs 11.2ns 46ns 0.16 0 0 1.01 KB
#2708 SendReceive netcoreapp3.1 2.01μs 3.13ns 10.8ns 0.0138 0 0 1.01 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 6.09μs 33.9ns 211ns 0.292 0 0 1.87 KB
master EnrichedLog netcoreapp3.1 5.33μs 26.4ns 156ns 0.0192 0 0 1.49 KB
#2708 EnrichedLog net472 6.08μs 29.3ns 114ns 0.292 0 0 1.87 KB
#2708 EnrichedLog netcoreapp3.1 5.32μs 16.7ns 62.5ns 0.0196 0 0 1.49 KB
Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #2708

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1 1.214 1,042.23 1,264.90 bimodal

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 914ns 4.9ns 26.4ns 0.0716 0 0 457 B
master StartFinishSpan netcoreapp3.1 882ns 4.38ns 22.7ns 0.00627 0 0 456 B
master StartFinishScope net472 1.08μs 4.92ns 18.4ns 0.0846 0 0 538 B
master StartFinishScope netcoreapp3.1 1.05μs 5.34ns 26.2ns 0.00786 0 0 576 B
#2708 StartFinishSpan net472 991ns 2.53ns 9.14ns 0.0715 0 0 457 B
#2708 StartFinishSpan netcoreapp3.1 887ns 4.19ns 16.8ns 0.00601 0 0 456 B
#2708 StartFinishScope net472 1.1μs 3.52ns 13.6ns 0.0841 0 0 538 B
#2708 StartFinishScope netcoreapp3.1 1.25μs 13.8ns 137ns 0.00795 0 0 576 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.28μs 6.7ns 36.7ns 0.0843 0 0 538 B
master RunOnMethodBegin netcoreapp3.1 1.17μs 5.73ns 23.6ns 0.0078 0 0 576 B
#2708 RunOnMethodBegin net472 1.27μs 5.76ns 40.3ns 0.0836 0 0 538 B
#2708 RunOnMethodBegin netcoreapp3.1 1.19μs 5.31ns 20.5ns 0.00785 0 0 576 B

@pierotibou
pierotibou merged commit 39c5773 into master Apr 21, 2022
@pierotibou
pierotibou deleted the pierre/native-loader-osx branch April 21, 2022 06:35
@github-actions github-actions Bot added this to the vNext milestone Apr 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zacharycmontoya zacharycmontoya zacharycmontoya approved these changes
@andrewlock andrewlock andrewlock approved these changes

Assignees

No one assigned

Labels

area:builds project files, build scripts, pipelines, versioning, releases, packages

Projects

None yet

Milestone

2.7.0

Development

Successfully merging this pull request may close these issues.

3 participants

@pierotibou @andrewlock @zacharycmontoya