Replace fpm with nfpm#5905
Conversation
Datadog ReportBranch report: ✅ 0 Failed, 297628 Passed, 1604 Skipped, 12h 11m 2s Total Time |
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing the following branches/commits: Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). |
Benchmarks Report for tracer 🐌Benchmarks for #5905 compared to master:
The following thresholds were used for comparing the benchmark speeds:
Allocation changes below 0.5% are ignored. Benchmark detailsBenchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations
|
| Benchmark | Base Allocated | Diff Allocated | Change | Change % |
|---|---|---|---|---|
| Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 | 41.61 KB | 41.92 KB | 306 B | 0.74% |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | WriteAndFlushEnrichedTraces |
net6.0 | 564μs | 2.25μs | 8.73μs | 0.581 | 0 | 0 | 41.47 KB |
| master | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 670μs | 3.11μs | 11.6μs | 0.342 | 0 | 0 | 41.61 KB |
| master | WriteAndFlushEnrichedTraces |
net472 | 848μs | 3.4μs | 13.2μs | 8.42 | 2.66 | 0.443 | 53.3 KB |
| #5905 | WriteAndFlushEnrichedTraces |
net6.0 | 558μs | 2.55μs | 9.87μs | 0.566 | 0 | 0 | 41.48 KB |
| #5905 | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 684μs | 3.38μs | 14μs | 0.338 | 0 | 0 | 41.92 KB |
| #5905 | WriteAndFlushEnrichedTraces |
net472 | 859μs | 4.06μs | 15.7μs | 8.36 | 2.64 | 0.44 | 53.33 KB |
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | ExecuteNonQuery |
net6.0 | 1.2μs | 1.04ns | 3.89ns | 0.0145 | 0 | 0 | 1.02 KB |
| master | ExecuteNonQuery |
netcoreapp3.1 | 1.68μs | 1.1ns | 4.12ns | 0.0133 | 0 | 0 | 1.02 KB |
| master | ExecuteNonQuery |
net472 | 2.02μs | 2.08ns | 8.06ns | 0.156 | 0 | 0 | 987 B |
| #5905 | ExecuteNonQuery |
net6.0 | 1.32μs | 1.5ns | 5.81ns | 0.0145 | 0 | 0 | 1.02 KB |
| #5905 | ExecuteNonQuery |
netcoreapp3.1 | 1.75μs | 1.23ns | 4.78ns | 0.0131 | 0 | 0 | 1.02 KB |
| #5905 | ExecuteNonQuery |
net472 | 1.91μs | 1.65ns | 6.39ns | 0.156 | 0 | 0 | 987 B |
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | CallElasticsearch |
net6.0 | 1.21μs | 0.493ns | 1.91ns | 0.0133 | 0 | 0 | 976 B |
| master | CallElasticsearch |
netcoreapp3.1 | 1.55μs | 0.529ns | 1.98ns | 0.0132 | 0 | 0 | 976 B |
| master | CallElasticsearch |
net472 | 2.45μs | 1.51ns | 5.84ns | 0.158 | 0 | 0 | 995 B |
| master | CallElasticsearchAsync |
net6.0 | 1.27μs | 1.24ns | 4.62ns | 0.0133 | 0 | 0 | 952 B |
| master | CallElasticsearchAsync |
netcoreapp3.1 | 1.6μs | 0.662ns | 2.48ns | 0.0137 | 0 | 0 | 1.02 KB |
| master | CallElasticsearchAsync |
net472 | 2.54μs | 2.69ns | 10.4ns | 0.167 | 0 | 0 | 1.05 KB |
| #5905 | CallElasticsearch |
net6.0 | 1.14μs | 0.632ns | 2.45ns | 0.0136 | 0 | 0 | 976 B |
| #5905 | CallElasticsearch |
netcoreapp3.1 | 1.49μs | 0.39ns | 1.46ns | 0.0127 | 0 | 0 | 976 B |
| #5905 | CallElasticsearch |
net472 | 2.45μs | 1.39ns | 5.39ns | 0.157 | 0 | 0 | 995 B |
| #5905 | CallElasticsearchAsync |
net6.0 | 1.32μs | 1.24ns | 4.8ns | 0.0132 | 0 | 0 | 952 B |
| #5905 | CallElasticsearchAsync |
netcoreapp3.1 | 1.59μs | 0.88ns | 3.29ns | 0.0135 | 0 | 0 | 1.02 KB |
| #5905 | CallElasticsearchAsync |
net472 | 2.61μs | 2.08ns | 8.07ns | 0.166 | 0 | 0 | 1.05 KB |
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | ExecuteAsync |
net6.0 | 1.22μs | 0.768ns | 2.97ns | 0.0134 | 0 | 0 | 952 B |
| master | ExecuteAsync |
netcoreapp3.1 | 1.66μs | 0.944ns | 3.65ns | 0.0124 | 0 | 0 | 952 B |
| master | ExecuteAsync |
net472 | 1.77μs | 4.48ns | 17.3ns | 0.145 | 0 | 0 | 915 B |
| #5905 | ExecuteAsync |
net6.0 | 1.29μs | 0.768ns | 2.97ns | 0.0132 | 0 | 0 | 952 B |
| #5905 | ExecuteAsync |
netcoreapp3.1 | 1.56μs | 0.294ns | 1.1ns | 0.0131 | 0 | 0 | 952 B |
| #5905 | ExecuteAsync |
net472 | 1.72μs | 0.817ns | 3.17ns | 0.145 | 0 | 0 | 915 B |
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | SendAsync |
net6.0 | 4.33μs | 1.55ns | 5.57ns | 0.0304 | 0 | 0 | 2.22 KB |
| master | SendAsync |
netcoreapp3.1 | 5.13μs | 1.3ns | 5.03ns | 0.0358 | 0 | 0 | 2.76 KB |
| master | SendAsync |
net472 | 7.75μs | 1.47ns | 5.69ns | 0.5 | 0 | 0 | 3.15 KB |
| #5905 | SendAsync |
net6.0 | 4.09μs | 2.6ns | 9.74ns | 0.0308 | 0 | 0 | 2.22 KB |
| #5905 | SendAsync |
netcoreapp3.1 | 5.02μs | 1.68ns | 6.27ns | 0.0375 | 0 | 0 | 2.76 KB |
| #5905 | SendAsync |
net472 | 7.73μs | 1.78ns | 6.9ns | 0.499 | 0 | 0 | 3.15 KB |
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 1.57μs | 0.627ns | 2.34ns | 0.0229 | 0 | 0 | 1.64 KB |
| master | EnrichedLog |
netcoreapp3.1 | 2.28μs | 0.822ns | 2.96ns | 0.0219 | 0 | 0 | 1.64 KB |
| master | EnrichedLog |
net472 | 2.67μs | 1.98ns | 7.68ns | 0.249 | 0 | 0 | 1.57 KB |
| #5905 | EnrichedLog |
net6.0 | 1.5μs | 0.439ns | 1.58ns | 0.0233 | 0 | 0 | 1.64 KB |
| #5905 | EnrichedLog |
netcoreapp3.1 | 2.13μs | 6.66ns | 24.9ns | 0.0223 | 0 | 0 | 1.64 KB |
| #5905 | EnrichedLog |
net472 | 2.69μs | 1.03ns | 3.85ns | 0.249 | 0 | 0 | 1.57 KB |
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 116μs | 218ns | 845ns | 0 | 0 | 0 | 4.28 KB |
| master | EnrichedLog |
netcoreapp3.1 | 118μs | 199ns | 717ns | 0 | 0 | 0 | 4.28 KB |
| master | EnrichedLog |
net472 | 147μs | 254ns | 982ns | 0.654 | 0.218 | 0 | 4.46 KB |
| #5905 | EnrichedLog |
net6.0 | 115μs | 346ns | 1.34μs | 0.057 | 0 | 0 | 4.28 KB |
| #5905 | EnrichedLog |
netcoreapp3.1 | 120μs | 327ns | 1.27μs | 0 | 0 | 0 | 4.28 KB |
| #5905 | EnrichedLog |
net472 | 148μs | 315ns | 1.22μs | 0.664 | 0.221 | 0 | 4.46 KB |
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 3.04μs | 0.63ns | 2.44ns | 0.0305 | 0 | 0 | 2.2 KB |
| master | EnrichedLog |
netcoreapp3.1 | 4.21μs | 10.7ns | 41.4ns | 0.0296 | 0 | 0 | 2.2 KB |
| master | EnrichedLog |
net472 | 5.03μs | 2.59ns | 10ns | 0.32 | 0 | 0 | 2.02 KB |
| #5905 | EnrichedLog |
net6.0 | 2.96μs | 1.19ns | 4.61ns | 0.0311 | 0 | 0 | 2.2 KB |
| #5905 | EnrichedLog |
netcoreapp3.1 | 4.27μs | 0.994ns | 3.85ns | 0.0299 | 0 | 0 | 2.2 KB |
| #5905 | EnrichedLog |
net472 | 4.97μs | 1.81ns | 7.01ns | 0.32 | 0 | 0 | 2.02 KB |
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | SendReceive |
net6.0 | 1.38μs | 1.18ns | 4.59ns | 0.0158 | 0 | 0 | 1.14 KB |
| master | SendReceive |
netcoreapp3.1 | 1.76μs | 0.717ns | 2.78ns | 0.0149 | 0 | 0 | 1.14 KB |
| master | SendReceive |
net472 | 2.16μs | 2.48ns | 9.61ns | 0.183 | 0.00109 | 0 | 1.16 KB |
| #5905 | SendReceive |
net6.0 | 1.35μs | 0.605ns | 2.34ns | 0.0162 | 0 | 0 | 1.14 KB |
| #5905 | SendReceive |
netcoreapp3.1 | 1.88μs | 1.25ns | 4.86ns | 0.015 | 0 | 0 | 1.14 KB |
| #5905 | SendReceive |
net472 | 2.25μs | 1.61ns | 6.02ns | 0.183 | 0.00113 | 0 | 1.16 KB |
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 2.76μs | 1.17ns | 4.36ns | 0.0221 | 0 | 0 | 1.6 KB |
| master | EnrichedLog |
netcoreapp3.1 | 3.89μs | 1.53ns | 5.94ns | 0.0213 | 0 | 0 | 1.65 KB |
| master | EnrichedLog |
net472 | 4.43μs | 2.83ns | 11ns | 0.323 | 0 | 0 | 2.04 KB |
| #5905 | EnrichedLog |
net6.0 | 2.71μs | 0.587ns | 2.2ns | 0.0217 | 0 | 0 | 1.6 KB |
| #5905 | EnrichedLog |
netcoreapp3.1 | 4.05μs | 3.34ns | 12.9ns | 0.0218 | 0 | 0 | 1.65 KB |
| #5905 | EnrichedLog |
net472 | 4.3μs | 3.11ns | 11.6ns | 0.323 | 0 | 0 | 2.04 KB |
Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️
Faster 🎉 in #5905
Benchmark
base/diff
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net472
1.142
652.79
571.51
| Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality |
|---|---|---|---|---|
| Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net472 | 1.142 | 652.79 | 571.51 |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | StartFinishSpan |
net6.0 | 398ns | 0.184ns | 0.711ns | 0.008 | 0 | 0 | 576 B |
| master | StartFinishSpan |
netcoreapp3.1 | 592ns | 0.386ns | 1.5ns | 0.00778 | 0 | 0 | 576 B |
| master | StartFinishSpan |
net472 | 653ns | 0.175ns | 0.676ns | 0.0917 | 0 | 0 | 578 B |
| master | StartFinishScope |
net6.0 | 547ns | 0.231ns | 0.832ns | 0.00956 | 0 | 0 | 696 B |
| master | StartFinishScope |
netcoreapp3.1 | 737ns | 0.518ns | 2.01ns | 0.00918 | 0 | 0 | 696 B |
| master | StartFinishScope |
net472 | 879ns | 0.903ns | 3.5ns | 0.105 | 0 | 0 | 658 B |
| #5905 | StartFinishSpan |
net6.0 | 406ns | 0.768ns | 2.87ns | 0.00799 | 0 | 0 | 576 B |
| #5905 | StartFinishSpan |
netcoreapp3.1 | 614ns | 0.419ns | 1.51ns | 0.00774 | 0 | 0 | 576 B |
| #5905 | StartFinishSpan |
net472 | 572ns | 0.938ns | 3.63ns | 0.0917 | 0 | 0 | 578 B |
| #5905 | StartFinishScope |
net6.0 | 552ns | 0.493ns | 1.91ns | 0.0099 | 0 | 0 | 696 B |
| #5905 | StartFinishScope |
netcoreapp3.1 | 773ns | 0.952ns | 3.69ns | 0.00955 | 0 | 0 | 696 B |
| #5905 | StartFinishScope |
net472 | 918ns | 4.05ns | 15.7ns | 0.105 | 0 | 0 | 658 B |
Benchmarks.Trace.TraceAnnotationsBenchmark - Faster 🎉 Same allocations ✔️
Faster 🎉 in #5905
Benchmark
base/diff
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑netcoreapp3.1
1.139
1,011.99
888.54
| Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality |
|---|---|---|---|---|
| Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑netcoreapp3.1 | 1.139 | 1,011.99 | 888.54 |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | RunOnMethodBegin |
net6.0 | 647ns | 0.413ns | 1.6ns | 0.0098 | 0 | 0 | 696 B |
| master | RunOnMethodBegin |
netcoreapp3.1 | 1.01μs | 0.299ns | 1.12ns | 0.00962 | 0 | 0 | 696 B |
| master | RunOnMethodBegin |
net472 | 1.07μs | 0.593ns | 2.3ns | 0.104 | 0 | 0 | 658 B |
| #5905 | RunOnMethodBegin |
net6.0 | 605ns | 0.564ns | 2.19ns | 0.00983 | 0 | 0 | 696 B |
| #5905 | RunOnMethodBegin |
netcoreapp3.1 | 888ns | 0.325ns | 1.17ns | 0.00932 | 0 | 0 | 696 B |
| #5905 | RunOnMethodBegin |
net472 | 1.03μs | 0.987ns | 3.56ns | 0.104 | 0 | 0 | 658 B |
fe1376b to
9001630
Compare
85e665e to
3d85dec
Compare
Throughput/Crank Report ⚡Throughput results for AspNetCoreSimpleController comparing the following branches/commits: Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red. Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards! gantt
title Throughput Linux x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (5905) (11.456M) : 0, 11456165
master (11.567M) : 0, 11566509
benchmarks/2.9.0 (11.883M) : 0, 11882814
section Automatic
This PR (5905) (7.740M) : 0, 7739659
master (7.841M) : 0, 7841471
benchmarks/2.9.0 (8.446M) : 0, 8445613
section Trace stats
master (8.125M) : 0, 8124886
section Manual
master (11.708M) : 0, 11708359
section Manual + Automatic
This PR (5905) (7.198M) : 0, 7198061
master (7.216M) : 0, 7216432
section DD_TRACE_ENABLED=0
master (10.773M) : 0, 10773417
gantt
title Throughput Linux arm64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (5905) (9.662M) : 0, 9661526
master (9.828M) : 0, 9828340
benchmarks/2.9.0 (9.760M) : 0, 9759697
section Automatic
This PR (5905) (6.573M) : 0, 6572556
master (6.642M) : 0, 6641935
section Trace stats
master (6.912M) : 0, 6911773
section Manual
master (9.501M) : 0, 9500687
section Manual + Automatic
This PR (5905) (6.032M) : 0, 6032343
master (6.162M) : 0, 6161652
section DD_TRACE_ENABLED=0
master (8.890M) : 0, 8889645
gantt
title Throughput Windows x64 (Total requests)
dateFormat X
axisFormat %s
section Baseline
This PR (5905) (10.345M) : 0, 10345255
section Automatic
This PR (5905) (6.895M) : 0, 6895351
section Manual + Automatic
This PR (5905) (6.310M) : 0, 6310476
|
…rtifacts\after-install.sh
f91433b to
0dcd714
Compare
## Summary of changes Adds support for the tracer on alpine on ARM64 ## Reason for change We currently support x64 on glibc/alpine but only glibc on arm64. This closes that matrix which makes it easier to understand ## Implementation details There's several layers, but at a high level - Anywhere where we detect platform, make sure to include `linux-musl-arm64` as well as `linux-arm64` - Build the tracer + profiler on alpine arm64 There were several difficulties - The ruby-based fpm bundler we use doesn't work on alpine arm64, hence #5905 - I couldn't get our existing alpine.build.dockerfile to compile on arm64 (where we currently compile llvm/clang16 on alpine14). - To work around it, created a separate alpine.build.arm64.dockerfile image which uses alpine:18 (the first version that includes clang16) as the base. - Obviously this gives us different requirements in arm64, but I think that's fine because a) this is a new feature b) we already have differences on the glibc requirements for arm64 c) We only support .NET 5+ anyway on arm64 (.NET 6 really given .NET 5 is mostly gone) and they ship alpine:18 images for .NET 6+ - I created a multi arch base image from the two base images ([as some bloke with a blog describes here](https://andrewlock.net/combining-multiple-docker-images-into-a-multi-arch-image/)) - I had difficulty compiling the profiler on arm64 initially (we require it for some crash tracking functionality), as we were getting `undefined reference to `_Uaarch64_get_accessors_int'` when trying to compile libunwind. @gleocadie tracked it down to [this issue](libunwind/libunwind#788), and fixed it by just not building the libunwind tests 🙇♂️ - The sqlite IAST instrumentation tests on alpine arm64 were failing in .NET 5. I _think_ that's because the sqlite package we're using there doesn't support it (which makes sense) so just disabled the tests for that scenario. They pass in .NET 6+ so I'm not worried about it. - Removed the "noop" smoke tests stage (there's nothing to run in it currently) - We can't even install .NET Core 2.1 using the dotnet install script on alpine arm64, so bailing out. We can probably bail out of more too, but that's not _necessary_ so haven't done it yet. There's still one outstanding issue I can't get my head around, on _debian x64_ (i.e. not something that should have changed) The `OnEolFrameworkInSsi_WhenForwarderPathExists_CallsForwarderWithExpectedTelemetry` tests is failing by crashing. It's bizarre, I can't figure it out, if anyone has any ideas, please let me know 😅 ## Test coverage I've run full installer + full TFM tests [here](https://dev.azure.com/datadoghq/dd-trace-dotnet/_build/results?buildId=162869&view=results) 🤞 ## Other details Stacked on - #5770 - #5905 as they both change key parts of the build + will mean we need to rebuild the VMs Fixes #3850 --------- Co-authored-by: Gregory LEOCADIE <[email protected]>
## Summary of changes Replaces the ruby tool `fpm` with `nfpm` and `tar` ## Reason for change `fpm` is a ruby tool, which means we have to install ruby. This has historically caused a bunch of issues at various points when dependencies change (e.g. we add/update anything or change dockerfiles). [`nfpm` is a go tool](https://github.com/goreleaser/nfpm) created specifically to do pretty much the same thing as `fpm`, but without requiring ruby. The only slightly annoying thing is that it doesn't support raw "tar" so we have to use the built-in `tar` for that (although that's not a big hardship). ## Implementation details - Install `nfpm` in the docker images and remove the `fpm` install (and removing all the other ruby-related code) - Call `tar` to pack the `tar` images, and `nfpm` to pack the `deb/rpm` packages - `nfpm` is a "config" based tool, so we dump the yaml config to a file, and pass that to the execution - I have validated the nfpm and fpm deb output are essentially identical, there are just a few nuances - `fpm` adds a `License` field to the `.deb` header, but that's non standard. [The correct approach](https://www.debian.org/doc/debian-policy/ch-docs.html#copyright-information) is to include a specific license file. See [this issue](goreleaser/nfpm#847) for more details. - Similarly, `fpm` adds a `Vendor` field that is non standard (but `Maintainer` is standard and is added). - `fpm` adds `Relocations: /var/datadog` to the `.rpm` header, so I have replicated that for nfpm, but I don't know that we actually _want_ that... If we let people relocate the headers, then the documentation is wrong for the profiler env vars etc. Left it as-is anyway, to reduce chance of breaking people, just something I spotted. - The `"Apache License 2.0"` license string used for `.rpm` previously is not a valid value, as defined by fedora, so switched it to the supported moniker [here](https://docs.fedoraproject.org/en-US/legal/allowed-licenses/). - The `tar` files were "accidentally" including the before/after scripts, so I've made sure to still include them, incase anyone was relying on those (they run the `createLogPath.sh` file, and create a symlink to dd-dotnet in `/usr/bin`) ## Test coverage [Ran a full test with all installers here](https://dev.azure.com/datadoghq/dd-trace-dotnet/_build/results?buildId=162539&view=results) - ignore the couple of failures in the dotnet tool smoke tests, those are failures on `master` I need to fix 😅 Will manually compare the output of this run too: - [x] tar (`tar -tzvf *.tar.gz`) - The same, except the before/after scripts weren't being made executable. Fixed - [x] deb (`dpkg --info *.deb` and `dpkg -c *.deb`) - Minor differences: - nfpm adds the directories as entries - nfpm doesn't have the non-standard License field (unlike fpm) - nfpm adds the license in the required copyright path - nfpm doesn't include a dummy changelog.gz file (unlike fpm) - [x] rpm (`rpm -qipl --scripts *.rpm`) - Minor differences, nfpm adds the directories as entries, nothing significant though ## Other details Stacked on - #5770 because we need to change the dockerfiles, and we need the Centos7 fixes in there at the very least. We'll need to update the VMSS images though after making these changes so I've been delaying until that makes sense up to this point.
## Summary of changes The `release/2.x` branch is broken since I updated the VMs ## Reason for change I updated VMs yesterday, which cleared the build cache. I knew that would slow down release/2.x builds but I forgot it would completely break them due to the centos7 mess. ## Implementation details Backported some PRs to fix the layer caching etc - #5770 - #5905 - #5933 (just the changes in the alpine dockerfile) ## Test coverage This is the test, hopefully it works 🤞
Summary of changes
Replaces the ruby tool
fpmwithnfpmandtarReason for change
fpmis a ruby tool, which means we have to install ruby. This has historically caused a bunch of issues at various points when dependencies change (e.g. we add/update anything or change dockerfiles).nfpmis a go tool created specifically to do pretty much the same thing asfpm, but without requiring ruby. The only slightly annoying thing is that it doesn't support raw "tar" so we have to use the built-intarfor that (although that's not a big hardship).Implementation details
nfpmin the docker images and remove thefpminstall (and removing all the other ruby-related code)tarto pack thetarimages, andnfpmto pack thedeb/rpmpackagesnfpmis a "config" based tool, so we dump the yaml config to a file, and pass that to the executionfpmadds aLicensefield to the.debheader, but that's non standard. The correct approach is to include a specific license file. See this issue for more details.fpmadds aVendorfield that is non standard (butMaintaineris standard and is added).fpmaddsRelocations: /var/datadogto the.rpmheader, so I have replicated that for nfpm, but I don't know that we actually want that... If we let people relocate the headers, then the documentation is wrong for the profiler env vars etc. Left it as-is anyway, to reduce chance of breaking people, just something I spotted."Apache License 2.0"license string used for.rpmpreviously is not a valid value, as defined by fedora, so switched it to the supported moniker here.tarfiles were "accidentally" including the before/after scripts, so I've made sure to still include them, incase anyone was relying on those (they run thecreateLogPath.shfile, and create a symlink to dd-dotnet in/usr/bin)Test coverage
Ran a full test with all installers here - ignore the couple of failures in the dotnet tool smoke tests, those are failures on
masterI need to fix 😅Will manually compare the output of this run too:
tar -tzvf *.tar.gz)dpkg --info *.debanddpkg -c *.deb)rpm -qipl --scripts *.rpm)Other details
Stacked on
because we need to change the dockerfiles, and we need the Centos7 fixes in there at the very least. We'll need to update the VMSS images though after making these changes so I've been delaying until that makes sense up to this point.