[Profiler] Use Vcpkg download and install native dependencies (libdatadog) by gleocadie · Pull Request #6388 · DataDog/dd-trace-dotnet

gleocadie · 2024-12-03T13:51:57Z

Summary of changes

Use Vcpkg to manage native dependency instead of nuget.

Reason for change

Lately, the libdatadog nuget package crossed the 250MB-size limit imposed by nuget.org. Which means that recent version of libdatadog cannot be packaged and pushed to NuGet.org.
=> The windows .Net profiler cannot update its dependency to libdatadog.

This package contains shared libraries and static libraries (x86/x64, Release/Debug) of libdatadog. Even if we split into shared and static libraries, we are stuck (only the package containing the shared libraries can be pushed, the package with the static libraries is still to big, especially the debug ones).

The bigger became even bigger when the data-pipeline crate was added to the libraries.

So to fix that, we will use Vcpkg. Vcpkg is a well-known package manager for native libraries. It's based on Cmake and allows to download prebuilt binaries, build from github/gitlab/... and is well-integrated with Visual Studio.

Implementation details

Setup a local ports overlay (place where we describe ports/packages, how to build, download them....) in build/vcpk_local_ports
Create libdatadog description
Create a vcpkg.json to declare dependency towards libdatadog
Create a vcpkg-configuration.json to declare the local ports overlay.
Use Directory.Build.targets and Directory.Build.props files to setup vcpkg for native libraries and specifically for libdatadog
Remove the dependency on the libdatadog nuget package
Create SetupVcpkg to locate vcpkg on the machine, otherwise download and install it.

Test coverage

Profiler current tests must work.

Other details

In Visual Studio, you can install vcpkg using VS installer. Add the path to vcpkg to PATH.

datadog-ddstaging · 2024-12-03T15:41:44Z

Datadog Report

Branch report: gleocadie/vcpkg-libdatadog
Commit report: a7ee362
Test service: dd-trace-dotnet

❌ 1 Failed (0 Known Flaky), 455682 Passed, 3290 Skipped, 20h 58m 41.42s Total Time

❌ Failed Tests (1)

CheckTraceContextAreAttachedForWalltimeProfilerHumberOfThreads - Datadog.Profiler.IntegrationTests.CodeHotspot.CodeHotspotTest - Details
Expand for error
```
 Exit code of "Samples.BuggyBits" should be 0 instead of 134 (= 0x86)
 Expected: True
 Actual:   False
```

andrewlock · 2024-12-03T15:53:57Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6388) - mean (69ms)  : 66, 71
     .   : milestone, 69,
    master - mean (68ms)  : 65, 71
     .   : milestone, 68,

    section CallTarget+Inlining+NGEN
    This PR (6388) - mean (978ms)  : 952, 1004
     .   : milestone, 978,
    master - mean (977ms)  : 953, 1000
     .   : milestone, 977,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6388) - mean (107ms)  : 105, 109
     .   : milestone, 107,
    master - mean (107ms)  : 105, 109
     .   : milestone, 107,

    section CallTarget+Inlining+NGEN
    This PR (6388) - mean (678ms)  : 663, 692
     .   : milestone, 678,
    master - mean (677ms)  : 660, 694
     .   : milestone, 677,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6388) - mean (91ms)  : 88, 93
     .   : milestone, 91,
    master - mean (91ms)  : 89, 93
     .   : milestone, 91,

    section CallTarget+Inlining+NGEN
    This PR (6388) - mean (636ms)  : 618, 654
     .   : milestone, 636,
    master - mean (633ms)  : 615, 650
     .   : milestone, 633,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6388) - mean (193ms)  : 189, 197
     .   : milestone, 193,
    master - mean (194ms)  : 190, 198
     .   : milestone, 194,

    section CallTarget+Inlining+NGEN
    This PR (6388) - mean (1,102ms)  : 1071, 1133
     .   : milestone, 1102,
    master - mean (1,098ms)  : 1071, 1125
     .   : milestone, 1098,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6388) - mean (280ms)  : 274, 285
     .   : milestone, 280,
    master - mean (277ms)  : 271, 282
     .   : milestone, 277,

    section CallTarget+Inlining+NGEN
    This PR (6388) - mean (872ms)  : 837, 907
     .   : milestone, 872,
    master - mean (873ms)  : 838, 908
     .   : milestone, 873,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6388) - mean (266ms)  : 260, 272
     .   : milestone, 266,
    master - mean (266ms)  : 262, 270
     .   : milestone, 266,

    section CallTarget+Inlining+NGEN
    This PR (6388) - mean (854ms)  : 824, 884
     .   : milestone, 854,
    master - mean (853ms)  : 820, 885
     .   : milestone, 853,

andrewlock · 2024-12-04T20:30:38Z

Throughput/Crank Report ⚡

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6388) (11.180M)   : 0, 11180442
    master (11.221M)   : 0, 11220876
    benchmarks/2.9.0 (11.033M)   : 0, 11032866

    section Automatic
    This PR (6388) (7.235M)   : 0, 7235254
    master (7.241M)   : 0, 7241087
    benchmarks/2.9.0 (7.786M)   : 0, 7785853

    section Trace stats
    master (7.659M)   : 0, 7659179

    section Manual
    master (11.183M)   : 0, 11183048

    section Manual + Automatic
    This PR (6388) (6.741M)   : 0, 6741052
    master (6.809M)   : 0, 6809231

    section DD_TRACE_ENABLED=0
    master (10.320M)   : 0, 10320441

gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6388) (9.567M)   : 0, 9567102
    master (9.619M)   : 0, 9619474
    benchmarks/2.9.0 (9.495M)   : 0, 9494821

    section Automatic
    This PR (6388) (6.385M)   : 0, 6384827
    master (6.441M)   : 0, 6441493

    section Trace stats
    master (6.636M)   : 0, 6635629

    section Manual
    master (9.572M)   : 0, 9571909

    section Manual + Automatic
    This PR (6388) (5.846M)   : 0, 5845635
    master (5.944M)   : 0, 5944316

    section DD_TRACE_ENABLED=0
    master (8.814M)   : 0, 8813897

gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (6388) (9.991M)   : 0, 9991212
    master (10.190M)   : 0, 10190355
    benchmarks/2.9.0 (10.020M)   : 0, 10019592

    section Automatic
    This PR (6388) (6.528M)   : 0, 6527618
    master (6.498M)   : 0, 6498441
    benchmarks/2.9.0 (7.255M)   : 0, 7255257

    section Trace stats
    master (7.175M)   : 0, 7175288

    section Manual
    master (9.943M)   : 0, 9943361

    section Manual + Automatic
    This PR (6388) (5.909M)   : 0, 5909476
    master (5.907M)   : 0, 5906581

    section DD_TRACE_ENABLED=0
    master (9.404M)   : 0, 9404038

andrewlock · 2024-12-04T23:45:35Z

Benchmarks Report for tracer 🐌

Benchmarks for #6388 compared to master:

3 benchmarks are slower, with geometric mean 1.177
1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	8.05μs	46.2ns	349ns	0.0158	0.00791	0	5.61 KB
master	`StartStopWithChild`	netcoreapp3.1	10.2μs	52.2ns	245ns	0.0197	0.00987	0	5.79 KB
master	`StartStopWithChild`	net472	16.4μs	58.4ns	226ns	1.04	0.306	0.0992	6.21 KB
#6388	`StartStopWithChild`	net6.0	8.26μs	46.4ns	322ns	0.0172	0.00859	0	5.61 KB
#6388	`StartStopWithChild`	netcoreapp3.1	10.3μs	41.5ns	150ns	0.0196	0.00979	0	5.81 KB
#6388	`StartStopWithChild`	net472	16.4μs	45.6ns	177ns	1.03	0.305	0.0907	6.21 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	502μs	2.15μs	8.34μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	650μs	353ns	1.32μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	851μs	509ns	1.97μs	0.422	3.3 KB
#6388	`WriteAndFlushEnrichedTraces`	net6.0	490μs	275ns	1.03μs	0	2.7 KB
#6388	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	649μs	648ns	2.51μs	0	2.7 KB
#6388	`WriteAndFlushEnrichedTraces`	net472	852μs	341ns	1.27μs	0.428	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	153μs	862ns	6.33μs	0.145	14.47 KB
master	`SendRequest`	netcoreapp3.1	169μs	979ns	9.23μs	0.168	17.27 KB
master	`SendRequest`	net472	0.00213ns	0.000857ns	0.00332ns	0	0 b
#6388	`SendRequest`	net6.0	153μs	967ns	9.58μs	0.142	14.47 KB
#6388	`SendRequest`	netcoreapp3.1	172μs	1.1μs	10.9μs	0.166	17.27 KB
#6388	`SendRequest`	net472	0.000964ns	0.000352ns	0.00132ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #6388

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0	41.52 KB	41.87 KB	354 B	0.85%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	555μs	2.78μs	11.8μs	0.576	0	0	41.52 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	655μs	2.55μs	14μs	0.324	0	0	41.74 KB
master	`WriteAndFlushEnrichedTraces`	net472	841μs	3.87μs	15μs	8.22	2.47	0.411	53.29 KB
#6388	`WriteAndFlushEnrichedTraces`	net6.0	589μs	3.16μs	16.7μs	0.607	0	0	41.87 KB
#6388	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	739μs	4.22μs	31.3μs	0.349	0	0	41.72 KB
#6388	`WriteAndFlushEnrichedTraces`	net472	868μs	4.1μs	16.4μs	8.04	2.23	0.446	53.28 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteNonQuery`	net6.0	1.34μs	1.27ns	4.93ns	0.0141	0	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	1.76μs	1.31ns	4.91ns	0.0138	0	1.02 KB
master	`ExecuteNonQuery`	net472	2.07μs	2.93ns	11.3ns	0.156	0.00103	987 B
#6388	`ExecuteNonQuery`	net6.0	1.29μs	1.41ns	5.46ns	0.0146	0	1.02 KB
#6388	`ExecuteNonQuery`	netcoreapp3.1	1.73μs	2.05ns	7.95ns	0.0138	0	1.02 KB
#6388	`ExecuteNonQuery`	net472	2.08μs	2.04ns	7.89ns	0.156	0.00104	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.26μs	0.398ns	1.49ns	0.0139	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.59μs	1.62ns	5.83ns	0.0128	976 B
master	`CallElasticsearch`	net472	2.54μs	2.1ns	8.12ns	0.157	995 B
master	`CallElasticsearchAsync`	net6.0	1.38μs	0.563ns	2.11ns	0.0131	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.66μs	2.26ns	8.74ns	0.0141	1.02 KB
master	`CallElasticsearchAsync`	net472	2.73μs	1.97ns	7.39ns	0.166	1.05 KB
#6388	`CallElasticsearch`	net6.0	1.35μs	3.39ns	13.1ns	0.0134	976 B
#6388	`CallElasticsearch`	netcoreapp3.1	1.58μs	1.28ns	4.96ns	0.0126	976 B
#6388	`CallElasticsearch`	net472	2.57μs	2.16ns	8.08ns	0.157	995 B
#6388	`CallElasticsearchAsync`	net6.0	1.26μs	0.621ns	2.4ns	0.0131	952 B
#6388	`CallElasticsearchAsync`	netcoreapp3.1	1.62μs	0.574ns	2.07ns	0.0137	1.02 KB
#6388	`CallElasticsearchAsync`	net472	2.62μs	2.26ns	8.77ns	0.166	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.39μs	0.827ns	3.2ns	0.0132	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.77μs	1.18ns	4.43ns	0.0123	952 B
master	`ExecuteAsync`	net472	1.77μs	0.324ns	1.21ns	0.145	915 B
#6388	`ExecuteAsync`	net6.0	1.26μs	3.94ns	15.2ns	0.0133	952 B
#6388	`ExecuteAsync`	netcoreapp3.1	1.64μs	0.959ns	3.59ns	0.0131	952 B
#6388	`ExecuteAsync`	net472	1.8μs	0.672ns	2.51ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.52μs	1.94ns	7ns	0.0319	2.31 KB
master	`SendAsync`	netcoreapp3.1	5.22μs	3.42ns	13.2ns	0.0391	2.85 KB
master	`SendAsync`	net472	7.35μs	2.09ns	8.11ns	0.495	3.12 KB
#6388	`SendAsync`	net6.0	4.53μs	2.7ns	10.1ns	0.0314	2.31 KB
#6388	`SendAsync`	netcoreapp3.1	5.17μs	2.64ns	10.2ns	0.0388	2.85 KB
#6388	`SendAsync`	net472	7.37μs	1.51ns	5.66ns	0.494	3.12 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.55μs	1.01ns	3.93ns	0.0224	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.31μs	1.34ns	5ns	0.0219	1.64 KB
master	`EnrichedLog`	net472	2.64μs	1.3ns	5.03ns	0.249	1.57 KB
#6388	`EnrichedLog`	net6.0	1.56μs	1.06ns	3.98ns	0.0228	1.64 KB
#6388	`EnrichedLog`	netcoreapp3.1	2.19μs	1.73ns	6.48ns	0.022	1.64 KB
#6388	`EnrichedLog`	net472	2.72μs	0.945ns	3.54ns	0.249	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	118μs	340ns	1.32μs	0.0595	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	123μs	152ns	590ns	0	0	4.28 KB
master	`EnrichedLog`	net472	153μs	45.3ns	157ns	0.689	0.23	4.46 KB
#6388	`EnrichedLog`	net6.0	118μs	168ns	650ns	0	0	4.28 KB
#6388	`EnrichedLog`	netcoreapp3.1	120μs	260ns	1.01μs	0	0	4.28 KB
#6388	`EnrichedLog`	net472	151μs	91ns	352ns	0.677	0.226	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.9μs	1.23ns	4.75ns	0.0308	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.23μs	1.34ns	5.18ns	0.0295	2.2 KB
master	`EnrichedLog`	net472	5.02μs	1.42ns	5.52ns	0.321	2.02 KB
#6388	`EnrichedLog`	net6.0	3.1μs	1.15ns	4.45ns	0.031	2.2 KB
#6388	`EnrichedLog`	netcoreapp3.1	4.03μs	1.21ns	4.36ns	0.0304	2.2 KB
#6388	`EnrichedLog`	net472	4.86μs	1.26ns	4.87ns	0.321	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.36μs	1.65ns	6.39ns	0.0156	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.79μs	4.72ns	18.3ns	0.0149	1.14 KB
master	`SendReceive`	net472	2.07μs	1.55ns	6.02ns	0.183	1.16 KB
#6388	`SendReceive`	net6.0	1.35μs	0.919ns	3.56ns	0.0162	1.14 KB
#6388	`SendReceive`	netcoreapp3.1	1.8μs	0.596ns	2.06ns	0.0154	1.14 KB
#6388	`SendReceive`	net472	2.17μs	1.35ns	5.24ns	0.183	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.71μs	0.889ns	3.21ns	0.023	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.87μs	1.45ns	5.63ns	0.0212	1.65 KB
master	`EnrichedLog`	net472	4.4μs	3.92ns	15.2ns	0.322	2.04 KB
#6388	`EnrichedLog`	net6.0	2.79μs	9.26ns	35.9ns	0.0231	1.6 KB
#6388	`EnrichedLog`	netcoreapp3.1	3.81μs	2.7ns	10.5ns	0.021	1.65 KB
#6388	`EnrichedLog`	net472	4.42μs	4.3ns	16.6ns	0.324	2.04 KB

Benchmarks.Trace.SpanBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6388

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1	1.174	646.73	759.43
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net472	1.132	833.90	944.12

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	399ns	0.211ns	0.761ns	0.008	576 B
master	`StartFinishSpan`	netcoreapp3.1	625ns	3.31ns	17.2ns	0.008	576 B
master	`StartFinishSpan`	net472	670ns	0.415ns	1.55ns	0.0917	578 B
master	`StartFinishScope`	net6.0	501ns	0.385ns	1.49ns	0.00964	696 B
master	`StartFinishScope`	netcoreapp3.1	646ns	0.705ns	2.73ns	0.00954	696 B
master	`StartFinishScope`	net472	833ns	0.993ns	3.85ns	0.105	658 B
#6388	`StartFinishSpan`	net6.0	402ns	0.183ns	0.71ns	0.00804	576 B
#6388	`StartFinishSpan`	netcoreapp3.1	572ns	0.313ns	1.21ns	0.0079	576 B
#6388	`StartFinishSpan`	net472	672ns	0.375ns	1.45ns	0.0915	578 B
#6388	`StartFinishScope`	net6.0	551ns	0.32ns	1.24ns	0.0098	696 B
#6388	`StartFinishScope`	netcoreapp3.1	758ns	0.955ns	3.7ns	0.00949	696 B
#6388	`StartFinishScope`	net472	944ns	0.912ns	3.53ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #6388

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0	1.226	592.35	726.03

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	592ns	0.333ns	1.29ns	0.00978	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	923ns	0.403ns	1.56ns	0.0094	696 B
master	`RunOnMethodBegin`	net472	1.16μs	0.531ns	2.06ns	0.104	658 B
#6388	`RunOnMethodBegin`	net6.0	726ns	0.409ns	1.58ns	0.00982	696 B
#6388	`RunOnMethodBegin`	netcoreapp3.1	920ns	0.437ns	1.69ns	0.00922	696 B
#6388	`RunOnMethodBegin`	net472	1.05μs	0.665ns	2.57ns	0.104	658 B

gleocadie · 2024-12-05T18:51:18Z

+                var tempFolder = TempDirectory / "vcpkg_temp";
+                CompressionTasks.UncompressZip(vcpkgZip, tempFolder);
+
+                CopyDirectoryRecursively(tempFolder / $"vcpkg-{vcpkgVersion}", destinationFolder);


could not move the directory? (not enough permissions 🤷 )

Weird 🤔 Meh, not a big deal (unless it's a particularly big file)

I can retry if you want and we can investigate this (currently looking at the size of the extracted package)

andrewlock

Awesome, nice job! A bunch of questions, mostly about paths etc but nothing blocking 🙂

andrewlock · 2024-12-06T11:04:13Z

+# Vcpkg Packages
+**/vcpkg_installed/*
+# when building using Nuke and vcpkg is not installed (or in the PATH)
+# the target SetupVcpkg will install it at the root of the repo


Just a nit, but can we control these paths? In general we're moving towards the "new" artifacts layout. i.e. everything goes in the artifacts folder, with a sub folder for "category" e.g. currently we have

artifacts |___ bin |___ Project1 |___ Project2 |___ obj |___ Project1 |___ Project2 |___ publish |___ Project1 |___ Project2 |___ build_data |___ logs |___ results

It would be nice if we could have (for example) a deps or dependencies folder, with a vcpkg subfolder perhaps? Something like that?

Currently we're only putting the samples and things in artifacts but ultimately I'd like to get everything building there, as it will give us some CI wins apart from anything else 🙂

good idea. I'll get to it.

for this specific one, it might be tricky but I'll check

andrewlock · 2024-12-06T11:04:42Z

-	ProjectSection(SolutionItems) = preProject
-		test\Directory.Build.props = test\Directory.Build.props
-	EndProjectSection


I don't know if this one should have gone? 🤔 But also, meh, who knows with .sln files 😅

andrewlock · 2024-12-06T11:22:31Z

+      <!-- VCPKG setup -->
+      <VcpkgEnableManifest>true</VcpkgEnableManifest>
+      <VcpkgUseStatic>true</VcpkgUseStatic>
+      <VcpkgIntermediateFolder>$(DOTNET-TRACER-REPO-ROOT-PATH)obj\vcpkg</VcpkgIntermediateFolder>


As mentioned earlier, would love to move this into artifacts/obj or artifacts/deps or anything really 😄

andrewlock · 2024-12-06T11:23:35Z

  </ItemGroup>
  <Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
  <ImportGroup Label="ExtensionTargets" />
-  <Target Name="EnsureNuGetPackageBuildImports" BeforeTargets="PrepareForBuild">


Was this the only nuget requirement we had on the native side do you know? Do any of the other native projects have nuget deps? 🤔

the profiler vcxproj (test, static lib and dll) files have libdatadog as nuget deps.
The test has googletest too.

andrewlock · 2024-12-06T11:26:44Z

 <packages>
  <package id="gmock" version="1.11.0" targetFramework="native" />
-  <package id="libdatadog" version="13.1.0" targetFramework="native" />
  <package id="Microsoft.googletest.v140.windesktop.msvcstl.static.rt-static" version="1.8.1.7" targetFramework="native" />


Was this the only nuget requirement we had on the native side do you know
oh, boo, I guess not 😄 Just wondering outloud can/should we switch to vcpkg dependencies for these too? Would that make sense?

For context, NuGet with native side is a bit of a pain, because you can't use the built-in dotnet version, you have to use the standalone NuGet etc, which adds a bit of extra complexity in places. Not a big deal, not for this PR, just wondering out loud

yeah I remember 😓

andrewlock · 2024-12-06T11:32:37Z

+                var tempFolder = TempDirectory / "vcpkg_temp";
+                CompressionTasks.UncompressZip(vcpkgZip, tempFolder);
+
+                CopyDirectoryRecursively(tempFolder / $"vcpkg-{vcpkgVersion}", destinationFolder);


Weird 🤔 Meh, not a big deal (unless it's a particularly big file)

andrewlock · 2024-12-06T11:34:00Z

Just wondering, do these have to be in the root folder? We have a vcpkg.json file in the build/ folder for example, but I assume this one is different...

in build/ is the description of the libdatadog package (+ instruction to download and create the folder structure on disk)
This one expresses the dependency for the all the subfolder. We could have put it in the profiler/ folder but I was foreseeing dependency from the tracer too.

I guess I was thinking that /build is "top-level" (i.e. profiler + tracer), so we could put it in there? But meh, was mostly just wondering if VS requires it to be in the root. Ultimately would be nice to rearrange the repo a bit at some point (e.g. move the tracer/build folder to be top-level build given it contains Nuke that's used by everything) and just want to make sure we understand the constraints 🙂

chrisnas

LGTM

chrisnas · 2024-12-16T16:11:01Z

    <ClInclude Include="RuntimeIdStoreHelper.h">
      <Filter>Helpers</Filter>
    </ClInclude>
-    <ClInclude Include="ThreadsCpuManagerHelper.h">


I see references of this type in ProviderTest and SamplesCollectorTest

…adog) (#6388)

github-actions Bot added the area:profiler Issues related to the continous-profiler label Dec 3, 2024

gleocadie force-pushed the gleocadie/vcpkg-libdatadog branch 3 times, most recently from 214005f to cf2ccd0 Compare December 3, 2024 15:02

gleocadie force-pushed the gleocadie/vcpkg-libdatadog branch 10 times, most recently from dbeac62 to 11647ae Compare December 4, 2024 17:12

gleocadie force-pushed the gleocadie/vcpkg-libdatadog branch from 11647ae to 668372e Compare December 4, 2024 22:02

gleocadie force-pushed the gleocadie/vcpkg-libdatadog branch 9 times, most recently from 10638b1 to ee43064 Compare December 5, 2024 16:53

gleocadie commented Dec 5, 2024

View reviewed changes

gleocadie changed the title ~~[Profiler] Try VCPKG 🤞~~ [Profiler] Use Vcpkg download and install native dependencies (libdatadog) Dec 5, 2024

andrewlock approved these changes Dec 6, 2024

View reviewed changes

gleocadie force-pushed the gleocadie/vcpkg-libdatadog branch 7 times, most recently from be13705 to 79ab416 Compare December 13, 2024 07:48

gleocadie force-pushed the gleocadie/vcpkg-libdatadog branch from 79ab416 to b69e808 Compare December 16, 2024 08:37

chrisnas approved these changes Dec 16, 2024

View reviewed changes

gleocadie added 13 commits December 16, 2024 20:48

Try VCPKG :crossfinder:

0ec02bc

Install and setup VCPKG

f9d6f64

use vcpkg for libdatadog step 2

2cec697

Shared the configuration for libdatadog static

a39b8b8

Set VCPKG_ROOT to <repo>/obj/vcpkg to avoid polluting C:"

8ea645c

Setup vcpkg if not installed on the machine

4ec1006

Clean up

69d3851

Fix final package

2caf1c9

Use the correct url to download libdatadog binaires

211bb88

Fix crashtracking test

ef2470a

Try following the artifacts layout

78861df

Adjust vcpkg setup when not found on the disk

fe9c901

Check gitlab for moving the vcpkg folder

a7ee362

gleocadie force-pushed the gleocadie/vcpkg-libdatadog branch from b69e808 to a7ee362 Compare December 16, 2024 19:48

Address PR comment

e59fe2b

gleocadie merged commit f0567c1 into master Dec 17, 2024

gleocadie deleted the gleocadie/vcpkg-libdatadog branch December 17, 2024 10:05

github-actions Bot added this to the vNext-v3 milestone Dec 17, 2024

BaptisteFoy mentioned this pull request Feb 1, 2025

upgrade(tracer): Support & use libdatadog's library_config crate #6615

Closed

5 tasks

chojomok pushed a commit that referenced this pull request Jul 15, 2025

[Profiler] Use Vcpkg download and install native dependencies (libdat…

2caf299

…adog) (#6388)

Uh oh!

Conversation

gleocadie commented Dec 3, 2024 • edited by chrisnas Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

Uh oh!

datadog-ddstaging Bot commented Dec 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Datadog Report

❌ Failed Tests (1)

Uh oh!

andrewlock commented Dec 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Execution-Time Benchmarks Report ⏱️

Uh oh!

andrewlock commented Dec 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Throughput/Crank Report ⚡

Uh oh!

andrewlock commented Dec 4, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

More allocations ⚠️ in #6388

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #6388

Raw results

Slower ⚠️ in #6388

Raw results

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

andrewlock left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gleocadie commented Dec 3, 2024 •

edited by chrisnas

Loading

datadog-ddstaging Bot commented Dec 3, 2024 •

edited

Loading

andrewlock commented Dec 3, 2024 •

edited

Loading

andrewlock commented Dec 4, 2024 •

edited

Loading

andrewlock commented Dec 4, 2024 •

edited

Loading