Skip to content

Fix build in gitlab#2666

Merged
gleocadie merged 3 commits into
masterfrom
gleocadie/fix-build-in-gitlab
Apr 12, 2022
Merged

Fix build in gitlab#2666
gleocadie merged 3 commits into
masterfrom
gleocadie/fix-build-in-gitlab

Conversation

@gleocadie

@gleocadie gleocadie commented Apr 11, 2022 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Summary of changes

Reason for change

Currently in gitlab, building native code (Tracer and Native loader as of today) may crash with the following error message:

 c:\devtools\vstudio\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB8071: Cannot parse tool output 'c:\mnt\shared\src\native-lib\spdlog\include\spdlog\details\pattern_formatter-inl.h(1188,1): fatal error C1060: compiler is out of heap space (compiling source file ..\..\..\shared\src\native-lib\spdlog\src\spdlog.cpp)' with regex '^In file included from .*$': Exception of type 'System.OutOfMemoryException' was thrown. [c:\mnt\tracer\src\Datadog.AutoInstrumentation.NativeLoader\Datadog.AutoInstrumentation.NativeLoader.vcxproj]
  c:\devtools\vstudio\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003: The specified task executable "CL.exe" could not be run. System.IO.IOException: The process cannot access the file 'c:\mnt\tracer\src\Datadog.AutoInstrumentation.NativeLoader\obj\Release\x64\Datadog..48d12c26.tlog\CL.5220.read.1.tlog' because it is being used by another process. [c:\mnt\tracer\src\Datadog.AutoInstrumentation.NativeLoader\Datadog.AutoInstrumentation.NativeLoader.vcxproj]
c:\devtools\vstudio\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003:    at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath) [c:\mnt\tracer\src\Datadog.AutoInstrumentation.NativeLoader\Datadog.AutoInstrumentation.NativeLoader.vcxproj]
c:\devtools\vstudio\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003:    at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBUTES secAttrs, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost) [c:\mnt\tracer\src\Datadog.AutoInstrumentation.NativeLoader\Datadog.AutoInstrumentation.NativeLoader.vcxproj]
c:\devtools\vstudio\MSBuild\Microsoft\VC\v160\Microsoft.CppCommon.targets(687,5): error MSB6003:    at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options, String msgPath, Boolean bFromProxy, Boolean useLongPath, Boolean checkHost) [c:\mnt\tracer\src\Datadog.AutoInstrumentation.NativeLoader\Datadog.AutoInstrumentation.NativeLoader.vcxproj]

One explanation of the problem .

Implementation details

Just set to false the MultiProcessor Compilation option when running on Gitlab by setting the environment variable ENABLE_MULTIPROCESSOR_COMPILATION to false. By default, it's set to true

Test coverage

Other details

This will slow down the build of native libraries.

@gleocadie
gleocadie requested a review from a team as a code owner April 11, 2022 14:03
andrewlock
andrewlock approved these changes Apr 11, 2022
View reviewed changes

@andrewlock andrewlock left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder, could/should we set this via an env var? We could set it to false in GitLab (where taking the perf hit isn't a big deal), and it would be enabled by default in other envs (azdo/GH/local)?

@gleocadie

gleocadie commented Apr 11, 2022

Copy link
Copy Markdown
Collaborator Author

I wonder, could/should we set this via an env var? We could set it to false in GitLab (where taking the perf hit isn't a big deal), and it would be enabled by default in other envs (azdo/GH/local)?

Good point, let me do another try at this PR.

@andrewlock

This comment has been minimized.

Sign in to view
zacharycmontoya
zacharycmontoya approved these changes Apr 11, 2022
View reviewed changes

@zacharycmontoya zacharycmontoya left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM especially only disabling the flag in GitLab

@gleocadie
gleocadie force-pushed the gleocadie/fix-build-in-gitlab branch 4 times, most recently from c6389bc to 419323f Compare April 11, 2022 17:59
@gleocadie
gleocadie requested a review from a team as a code owner April 11, 2022 18:08
@andrewlock

This comment has been minimized.

Sign in to view
@andrewlock

andrewlock commented Apr 12, 2022

Copy link
Copy Markdown
Member

Benchmarks Report 🐌

Benchmarks for #2666 compared to master:

  • 1 benchmarks are faster, with geometric mean 1.127
  • All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net472 945μs 4.9μs 23.5μs 0 0 0 3.16 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 729μs 4.15μs 30.5μs 0 0 0 2.57 KB
#2666 WriteAndFlushEnrichedTraces net472 913μs 3.94μs 14.7μs 0 0 0 3.16 KB
#2666 WriteAndFlushEnrichedTraces netcoreapp3.1 724μs 3.52μs 14.5μs 0 0 0 2.57 KB
Benchmarks.Trace.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net472 294ns 1.58ns 8.38ns 0.0521 0 0 329 B
master AllCycleSimpleBody netcoreapp3.1 384ns 1.97ns 10.8ns 0.00449 0 0 328 B
master AllCycleMoreComplexBody net472 287ns 1.24ns 5.68ns 0.0482 0 0 305 B
master AllCycleMoreComplexBody netcoreapp3.1 379ns 1.89ns 8.22ns 0.00429 0 0 304 B
master BodyExtractorSimpleBody net472 426ns 2.07ns 8.8ns 0.0569 0 0 361 B
master BodyExtractorSimpleBody netcoreapp3.1 451ns 2.63ns 22.2ns 0.00369 0 0 272 B
master BodyExtractorMoreComplexBody net472 25.2μs 144ns 1.22μs 1.19 0.0122 0 7.62 KB
master BodyExtractorMoreComplexBody netcoreapp3.1 20.9μs 77.2ns 289ns 0.0922 0 0 6.75 KB
#2666 AllCycleSimpleBody net472 291ns 1.32ns 4.95ns 0.0519 0 0 329 B
#2666 AllCycleSimpleBody netcoreapp3.1 393ns 2.76ns 26.3ns 0.00457 0 0 328 B
#2666 AllCycleMoreComplexBody net472 290ns 1.51ns 7.68ns 0.0481 0 0 305 B
#2666 AllCycleMoreComplexBody netcoreapp3.1 373ns 1.88ns 10.5ns 0.00412 0 0 304 B
#2666 BodyExtractorSimpleBody net472 435ns 2.34ns 15.1ns 0.057 0 0 361 B
#2666 BodyExtractorSimpleBody netcoreapp3.1 442ns 2.18ns 9.24ns 0.00372 0 0 272 B
#2666 BodyExtractorMoreComplexBody net472 24μs 118ns 516ns 1.19 0.0117 0 7.62 KB
#2666 BodyExtractorMoreComplexBody netcoreapp3.1 21.2μs 81.8ns 317ns 0.0855 0 0 6.75 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
master SendRequest netcoreapp3.1 298μs 1.65μs 9.6μs 0.148 0 0 19.74 KB
#2666 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#2666 SendRequest netcoreapp3.1 295μs 1.44μs 6.28μs 0.285 0 0 19.73 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net472 1.79μs 8.92ns 48ns 0.0939 0 0 594 B
master ExecuteNonQuery netcoreapp3.1 1.56μs 9.02ns 80.2ns 0.00869 0 0 632 B
#2666 ExecuteNonQuery net472 1.67μs 7.26ns 27.1ns 0.094 0 0 594 B
#2666 ExecuteNonQuery netcoreapp3.1 1.47μs 7.03ns 46.1ns 0.00883 0 0 632 B
Benchmarks.Trace.ElasticsearchBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #2666

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑netcoreapp3.1 1.127 2,027.89 1,798.81

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net472 2.84μs 16.5ns 137ns 0.124 0 0 802 B
master CallElasticsearch netcoreapp3.1 1.65μs 7.68ns 28.7ns 0.0108 0 0 792 B
master CallElasticsearchAsync net472 3.07μs 19.3ns 191ns 0.147 0 0 939 B
master CallElasticsearchAsync netcoreapp3.1 2.03μs 5.68ns 19.7ns 0.0122 0 0 912 B
#2666 CallElasticsearch net472 2.87μs 16.5ns 138ns 0.125 0 0 802 B
#2666 CallElasticsearch netcoreapp3.1 1.69μs 9.4ns 61.6ns 0.0109 0 0 792 B
#2666 CallElasticsearchAsync net472 2.89μs 13.6ns 52.8ns 0.149 0 0 939 B
#2666 CallElasticsearchAsync netcoreapp3.1 1.8μs 5.31ns 19.1ns 0.0128 0 0 912 B
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net472 3.09μs 17.1ns 98.1ns 0.165 0 0 1.06 KB
master ExecuteAsync netcoreapp3.1 2.04μs 11.3ns 70.8ns 0.0141 0 0 1.03 KB
#2666 ExecuteAsync net472 3.39μs 18.6ns 104ns 0.167 0 0 1.06 KB
#2666 ExecuteAsync netcoreapp3.1 2.09μs 9.75ns 37.8ns 0.014 0 0 1.03 KB
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net472 6.86μs 38.2ns 238ns 0.356 0 0 2.28 KB
master SendAsync netcoreapp3.1 4.8μs 23.3ns 96.2ns 0.0298 0 0 2.21 KB
#2666 SendAsync net472 6.87μs 33.3ns 133ns 0.362 0 0 2.28 KB
#2666 SendAsync netcoreapp3.1 4.7μs 22ns 103ns 0.0307 0 0 2.21 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 3.42μs 18.8ns 147ns 0.226 0 0 1.45 KB
master EnrichedLog netcoreapp3.1 3.25μs 16ns 80.1ns 0.021 0 0 1.53 KB
#2666 EnrichedLog net472 3.37μs 18.2ns 94.4ns 0.227 0 0 1.45 KB
#2666 EnrichedLog netcoreapp3.1 3.1μs 16.5ns 88.6ns 0.021 0 0 1.53 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 296μs 1.61μs 9.53μs 0.452 0.151 0 4.33 KB
master EnrichedLog netcoreapp3.1 231μs 1.11μs 4.44μs 0 0 0 4.21 KB
#2666 EnrichedLog net472 316μs 2.94μs 29.4μs 0.438 0.146 0 4.33 KB
#2666 EnrichedLog netcoreapp3.1 238μs 1.28μs 6.67μs 0 0 0 4.21 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 7.91μs 42.1ns 245ns 0.506 0 0 3.23 KB
master EnrichedLog netcoreapp3.1 6.55μs 34.9ns 181ns 0.0501 0 0 3.6 KB
#2666 EnrichedLog net472 7.54μs 38.5ns 185ns 0.506 0 0 3.23 KB
#2666 EnrichedLog netcoreapp3.1 6.52μs 32.8ns 157ns 0.0481 0 0 3.6 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net472 2.29μs 9.68ns 33.5ns 0.16 0.00112 0 1.01 KB
master SendReceive netcoreapp3.1 2.19μs 8.2ns 31.8ns 0.0142 0 0 1.01 KB
#2666 SendReceive net472 2.34μs 11.7ns 51.1ns 0.16 0.00112 0 1.01 KB
#2666 SendReceive netcoreapp3.1 1.98μs 7.36ns 28.5ns 0.0135 0 0 1.01 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net472 6μs 30.5ns 136ns 0.29 0 0 1.87 KB
master EnrichedLog netcoreapp3.1 5.11μs 19.3ns 72.1ns 0.0202 0 0 1.49 KB
#2666 EnrichedLog net472 5.94μs 23.8ns 89.2ns 0.291 0 0 1.87 KB
#2666 EnrichedLog netcoreapp3.1 5.11μs 24.6ns 95.2ns 0.0211 0 0 1.49 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net472 909ns 5.08ns 33.3ns 0.0719 0 0 457 B
master StartFinishSpan netcoreapp3.1 915ns 4.96ns 27.6ns 0.00618 0 0 456 B
master StartFinishScope net472 1.08μs 2.65ns 9.55ns 0.0842 0 0 538 B
master StartFinishScope netcoreapp3.1 1.04μs 5.77ns 36ns 0.008 0 0 576 B
#2666 StartFinishSpan net472 894ns 4.85ns 26.1ns 0.0718 0 0 457 B
#2666 StartFinishSpan netcoreapp3.1 914ns 5.05ns 32ns 0.00605 0 0 456 B
#2666 StartFinishScope net472 1.09μs 4.7ns 18.2ns 0.0844 0 0 538 B
#2666 StartFinishScope netcoreapp3.1 1.02μs 4.17ns 16.1ns 0.00784 0 0 576 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net472 1.28μs 6.1ns 23.6ns 0.0844 0 0 538 B
master RunOnMethodBegin netcoreapp3.1 1.14μs 2.47ns 8.9ns 0.00801 0 0 576 B
#2666 RunOnMethodBegin net472 1.27μs 4.94ns 18.5ns 0.0841 0 0 538 B
#2666 RunOnMethodBegin netcoreapp3.1 1.19μs 6.76ns 47.8ns 0.00794 0 0 576 B

@gleocadie
gleocadie force-pushed the gleocadie/fix-build-in-gitlab branch from 83f1063 to 6f16f58 Compare April 12, 2022 08:59
@gleocadie gleocadie added the area:builds project files, build scripts, pipelines, versioning, releases, packages label Apr 12, 2022
chrisnas
chrisnas approved these changes Apr 12, 2022
View reviewed changes

@chrisnas chrisnas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@gleocadie
gleocadie merged commit 774621b into master Apr 12, 2022
@gleocadie
gleocadie deleted the gleocadie/fix-build-in-gitlab branch April 12, 2022 13:04
@github-actions github-actions Bot added this to the vNext milestone Apr 12, 2022
@andrewlock

andrewlock commented Apr 12, 2022

Copy link
Copy Markdown
Member

Code Coverage Report 📊

✔️ Merging #2666 into master will not change line coverage
✔️ Merging #2666 into master will not change branch coverage
✔️ Merging #2666 into master will not change complexity

master #2666 Change
Lines 13538 / 18627 13543 / 18627
Lines % 73% 73% 0% ✔️
Branches 7815 / 11125 7829 / 11125
Branches % 70% 70% 0% ✔️
Complexity 12431 12431 0 ✔️

View the full report for further details:

Datadog.Trace Breakdown ✔️

master #2666 Change
Lines % 73% 73% 0% ✔️
Branches % 70% 70% 0% ✔️
Complexity 12431 12431 0 ✔️

The following classes have significant coverage changes.

File Line coverage change Branch coverage change Complexity change
Datadog.Trace.Ci.CIVisibility 7% ✔️ 7% ✔️ 0 ✔️
Datadog.Trace.Telemetry.JsonHttpClientTelemetryTransport 26% ✔️ 38% ✔️ 0 ✔️

View the full reports for further details:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@chrisnas chrisnas chrisnas approved these changes
@zacharycmontoya zacharycmontoya zacharycmontoya approved these changes
@andrewlock andrewlock andrewlock approved these changes

Assignees

No one assigned

Labels

area:builds project files, build scripts, pipelines, versioning, releases, packages

Projects

None yet

Milestone

2.7.0

Development

Successfully merging this pull request may close these issues.

4 participants

@gleocadie @andrewlock @chrisnas @zacharycmontoya