Skip to content

[build] Build AWS Lambda layers (with ReadyToRun)#4573

Closed
lucaspimentel wants to merge 17 commits into
masterfrom
lpimentel/serverless-readytorun
Closed

[build] Build AWS Lambda layers (with ReadyToRun)#4573
lucaspimentel wants to merge 17 commits into
masterfrom
lpimentel/serverless-readytorun

Conversation

@lucaspimentel

@lucaspimentel lucaspimentel commented Aug 31, 2023

Copy link
Copy Markdown
Member

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

@lucaspimentel lucaspimentel added the area:builds project files, build scripts, pipelines, versioning, releases, packages label Aug 31, 2023
@datadog-ddstaging

datadog-ddstaging Bot commented Aug 31, 2023 •
edited
Loading

Copy link
Copy Markdown

Datadog Report

Branch report: lpimentel/serverless-readytorun
Commit report: 7c5abfc

✅ dd-trace-dotnet: 0 Failed, 0 New Flaky, 301351 Passed, 1115 Skipped, 22m 34.52s Wall Time

@andrewlock

andrewlock commented Aug 31, 2023 •
edited
Loading

Copy link
Copy Markdown
Member

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4573) - mean (70ms)  : 62, 77
     .   : milestone, 70,
    master - mean (68ms)  : 63, 74
     .   : milestone, 68,

    section CallTarget+Inlining+NGEN
    This PR (4573) - mean (1,032ms)  : 996, 1067
     .   : milestone, 1032,
    master - mean (1,030ms)  : 998, 1062
     .   : milestone, 1030,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4573) - mean (106ms)  : 100, 111
     .   : milestone, 106,
    master - mean (106ms)  : 100, 112
     .   : milestone, 106,

    section CallTarget+Inlining+NGEN
    This PR (4573) - mean (719ms)  : 687, 750
     .   : milestone, 719,
    master - mean (723ms)  : 697, 750
     .   : milestone, 723,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4573) - mean (90ms)  : 86, 94
     .   : milestone, 90,
    master - mean (90ms)  : 87, 93
     .   : milestone, 90,

    section CallTarget+Inlining+NGEN
    This PR (4573) - mean (682ms)  : 652, 712
     .   : milestone, 682,
    master - mean (684ms)  : 652, 716
     .   : milestone, 684,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4573) - mean (192ms)  : 186, 197
     .   : milestone, 192,
    master - mean (191ms)  : 185, 197
     .   : milestone, 191,

    section CallTarget+Inlining+NGEN
    This PR (4573) - mean (1,122ms)  : 1096, 1148
     .   : milestone, 1122,
    master - mean (1,126ms)  : 1097, 1155
     .   : milestone, 1126,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4573) - mean (273ms)  : 266, 280
     .   : milestone, 273,
    master - mean (272ms)  : 266, 278
     .   : milestone, 272,

    section CallTarget+Inlining+NGEN
    This PR (4573) - mean (1,084ms)  : 1049, 1118
     .   : milestone, 1084,
    master - mean (1,083ms)  : 1047, 1120
     .   : milestone, 1083,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (4573) - mean (263ms)  : 255, 270
     .   : milestone, 263,
    master - mean (261ms)  : 253, 269
     .   : milestone, 261,

    section CallTarget+Inlining+NGEN
    This PR (4573) - mean (1,040ms)  : 1005, 1075
     .   : milestone, 1040,
    master - mean (1,030ms)  : 997, 1063
     .   : milestone, 1030,
Loading

@andrewlock

andrewlock commented Aug 31, 2023 •
edited
Loading

Copy link
Copy Markdown
Member

Benchmarks Report 🐌

Benchmarks for #4573 compared to master:

  • 3 benchmarks are faster, with geometric mean 1.133
  • 2 benchmarks are slower, with geometric mean 1.144
  • 1 benchmarks have fewer allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 8.27μs 44.6ns 256ns 0.0208 0.00832 0 7.3 KB
master StartStopWithChild netcoreapp3.1 10μs 48.5ns 194ns 0.0239 0.00955 0 7.39 KB
master StartStopWithChild net472 15.6μs 51.7ns 200ns 1.26 0.281 0.078 7.67 KB
#4573 StartStopWithChild net6.0 8.14μs 43.4ns 234ns 0.0165 0.00823 0 7.3 KB
#4573 StartStopWithChild netcoreapp3.1 9.86μs 51.5ns 268ns 0.0252 0.0101 0 7.39 KB
#4573 StartStopWithChild net472 15.9μs 52ns 201ns 1.28 0.339 0.102 7.67 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 478μs 230ns 892ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 644μs 274ns 1.06μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 792μs 380ns 1.47μs 0.393 0 0 3.3 KB
#4573 WriteAndFlushEnrichedTraces net6.0 492μs 376ns 1.41μs 0 0 0 2.7 KB
#4573 WriteAndFlushEnrichedTraces netcoreapp3.1 636μs 152ns 570ns 0 0 0 2.7 KB
#4573 WriteAndFlushEnrichedTraces net472 790μs 284ns 1.1μs 0.396 0 0 3.3 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #4573

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody‑net6.0 1.126 133.04 149.79

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 38.7μs 14.7ns 56.8ns 0.0195 0 0 1.69 KB
master AllCycleSimpleBody netcoreapp3.1 41.1μs 20.5ns 76.7ns 0.0205 0 0 1.67 KB
master AllCycleSimpleBody net472 44.3μs 51.5ns 200ns 0.259 0 0 1.73 KB
master AllCycleMoreComplexBody net6.0 230μs 311ns 1.2μs 0.115 0 0 9.26 KB
master AllCycleMoreComplexBody netcoreapp3.1 241μs 515ns 1.99μs 0.12 0 0 9.16 KB
master AllCycleMoreComplexBody net472 247μs 90.1ns 349ns 1.36 0 0 9.33 KB
master ObjectExtractorSimpleBody net6.0 134ns 0.328ns 1.27ns 0.00394 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 172ns 0.464ns 1.8ns 0.00378 0 0 272 B
master ObjectExtractorSimpleBody net472 147ns 0.162ns 0.627ns 0.0446 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 2.98μs 1.43ns 5.35ns 0.054 0 0 3.88 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 3.99μs 1.33ns 4.97ns 0.0499 0 0 3.78 KB
master ObjectExtractorMoreComplexBody net472 4.08μs 4.17ns 16.1ns 0.618 0.00612 0 3.89 KB
#4573 AllCycleSimpleBody net6.0 39.4μs 12.4ns 47.9ns 0.0196 0 0 1.69 KB
#4573 AllCycleSimpleBody netcoreapp3.1 41.4μs 137ns 511ns 0.0205 0 0 1.67 KB
#4573 AllCycleSimpleBody net472 42.9μs 9.59ns 34.6ns 0.255 0 0 1.73 KB
#4573 AllCycleMoreComplexBody net6.0 232μs 73.4ns 274ns 0.116 0 0 9.26 KB
#4573 AllCycleMoreComplexBody netcoreapp3.1 240μs 226ns 846ns 0.121 0 0 9.16 KB
#4573 AllCycleMoreComplexBody net472 251μs 194ns 750ns 1.37 0 0 9.33 KB
#4573 ObjectExtractorSimpleBody net6.0 150ns 0.0585ns 0.227ns 0.00389 0 0 280 B
#4573 ObjectExtractorSimpleBody netcoreapp3.1 173ns 0.0977ns 0.378ns 0.00374 0 0 272 B
#4573 ObjectExtractorSimpleBody net472 147ns 0.113ns 0.438ns 0.0446 0 0 281 B
#4573 ObjectExtractorMoreComplexBody net6.0 3.01μs 0.86ns 3.22ns 0.0543 0 0 3.88 KB
#4573 ObjectExtractorMoreComplexBody netcoreapp3.1 4.07μs 3.38ns 13.1ns 0.0511 0 0 3.78 KB
#4573 ObjectExtractorMoreComplexBody net472 4.11μs 3.69ns 13.8ns 0.618 0.00612 0 3.89 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWaf(args=NestedMap (10)) net6.0 150ms 248μs 927μs 0 0 0 33.94 MB
master RunWaf(args=NestedMap (10)) netcoreapp3.1 166ms 399μs 1.49ms 0 0 0 33.94 MB
master RunWaf(args=NestedMap (10)) net472 236ms 199μs 689μs 6E+03 3E+03 2E+03 35.2 MB
master RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 167ms 192μs 745μs 0 0 0 33.95 MB
master RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 184ms 371μs 1.44ms 0 0 0 33.95 MB
master RunWafWithAttack(args=Neste(...)tack) [22]) net472 251ms 306μs 1.14ms 5E+03 2E+03 1E+03 35.19 MB
master RunWaf(args=NestedMap (100)) net6.0 299ms 971μs 3.63ms 0 0 0 73.6 MB
master RunWaf(args=NestedMap (100)) netcoreapp3.1 331ms 745μs 2.79ms 0 0 0 74.91 MB
master RunWaf(args=NestedMap (100)) net472 502ms 2.44ms 9.77ms 1.1E+04 5E+03 2E+03 78.66 MB
master RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 319ms 339μs 1.27ms 0 0 0 73.6 MB
master RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 359ms 452μs 1.57ms 0 0 0 74.91 MB
master RunWafWithAttack(args=Neste(...)tack) [23]) net472 530ms 1.15ms 4.44ms 1.1E+04 5E+03 2E+03 78.68 MB
master RunWaf(args=NestedMap (1000)) net6.0 297ms 784μs 2.93ms 0 0 0 73.6 MB
master RunWaf(args=NestedMap (1000)) netcoreapp3.1 344ms 958μs 3.71ms 0 0 0 74.91 MB
master RunWaf(args=NestedMap (1000)) net472 506ms 1.38ms 5.36ms 1.1E+04 5E+03 2E+03 78.67 MB
master RunWafWithAttack(args=Neste(...)tack) [24]) net6.0 319ms 899μs 3.48ms 0 0 0 73.69 MB
master RunWafWithAttack(args=Neste(...)tack) [24]) netcoreapp3.1 365ms 743μs 2.88ms 0 0 0 74.91 MB
master RunWafWithAttack(args=Neste(...)tack) [24]) net472 534ms 1.62ms 6.29ms 1.1E+04 5E+03 2E+03 78.68 MB
#4573 RunWaf(args=NestedMap (10)) net6.0 152ms 234μs 905μs 0 0 0 33.94 MB
#4573 RunWaf(args=NestedMap (10)) netcoreapp3.1 165ms 279μs 1.08ms 0 0 0 33.94 MB
#4573 RunWaf(args=NestedMap (10)) net472 237ms 198μs 767μs 6E+03 3E+03 2E+03 35.2 MB
#4573 RunWafWithAttack(args=Neste(...)tack) [22]) net6.0 166ms 197μs 763μs 0 0 0 33.95 MB
#4573 RunWafWithAttack(args=Neste(...)tack) [22]) netcoreapp3.1 182ms 196μs 760μs 0 0 0 33.95 MB
#4573 RunWafWithAttack(args=Neste(...)tack) [22]) net472 249ms 223μs 834μs 5E+03 2E+03 1E+03 35.19 MB
#4573 RunWaf(args=NestedMap (100)) net6.0 296ms 682μs 2.64ms 0 0 0 73.68 MB
#4573 RunWaf(args=NestedMap (100)) netcoreapp3.1 340ms 840μs 3.03ms 0 0 0 74.91 MB
#4573 RunWaf(args=NestedMap (100)) net472 508ms 1.52ms 5.87ms 1.1E+04 5E+03 2E+03 78.67 MB
#4573 RunWafWithAttack(args=Neste(...)tack) [23]) net6.0 324ms 716μs 2.77ms 0 0 0 73.6 MB
#4573 RunWafWithAttack(args=Neste(...)tack) [23]) netcoreapp3.1 360ms 342μs 1.28ms 0 0 0 74.91 MB
#4573 RunWafWithAttack(args=Neste(...)tack) [23]) net472 528ms 411μs 1.54ms 1.1E+04 5E+03 2E+03 78.66 MB
#4573 RunWaf(args=NestedMap (1000)) net6.0 298ms 347μs 1.25ms 0 0 0 73.6 MB
#4573 RunWaf(args=NestedMap (1000)) netcoreapp3.1 334ms 370μs 1.34ms 0 0 0 74.99 MB
#4573 RunWaf(args=NestedMap (1000)) net472 504ms 1.59ms 6.17ms 1.1E+04 5E+03 2E+03 78.67 MB
#4573 RunWafWithAttack(args=Neste(...)tack) [24]) net6.0 324ms 686μs 2.57ms 0 0 0 73.6 MB
#4573 RunWafWithAttack(args=Neste(...)tack) [24]) netcoreapp3.1 361ms 908μs 3.52ms 0 0 0 74.91 MB
#4573 RunWafWithAttack(args=Neste(...)tack) [24]) net472 540ms 2.11ms 8.19ms 1.1E+04 5E+03 2E+03 78.73 MB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 170μs 67.8ns 254ns 0.254 0 0 18.09 KB
master SendRequest netcoreapp3.1 192μs 408ns 1.58μs 0.19 0 0 20.25 KB
master SendRequest net472 0.00148ns 0.000519ns 0.00194ns 0 0 0 0 b
#4573 SendRequest net6.0 170μs 49.5ns 178ns 0.17 0 0 18.09 KB
#4573 SendRequest netcoreapp3.1 192μs 162ns 607ns 0.19 0 0 20.25 KB
#4573 SendRequest net472 0.0015ns 0.000599ns 0.00232ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #4573

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0 41.72 KB 41.49 KB -232 B -0.56%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 517μs 858ns 3.32μs 0.523 0 0 41.72 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 627μs 1.88μs 7.27μs 0.314 0 0 41.7 KB
master WriteAndFlushEnrichedTraces net472 771μs 2.29μs 8.88μs 8.15 2.33 0.388 53.22 KB
#4573 WriteAndFlushEnrichedTraces net6.0 508μs 330ns 1.24μs 0.519 0 0 41.49 KB
#4573 WriteAndFlushEnrichedTraces netcoreapp3.1 634μs 659ns 2.55μs 0.312 0 0 41.88 KB
#4573 WriteAndFlushEnrichedTraces net472 756μs 2.32μs 8.97μs 8.38 2.67 0.381 53.26 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 969ns 0.511ns 1.98ns 0.0106 0 0 768 B
master ExecuteNonQuery netcoreapp3.1 1.32μs 0.525ns 1.96ns 0.0105 0 0 768 B
master ExecuteNonQuery net472 1.57μs 0.956ns 3.7ns 0.115 0.000785 0 730 B
#4573 ExecuteNonQuery net6.0 1.06μs 0.581ns 2.25ns 0.0107 0 0 768 B
#4573 ExecuteNonQuery netcoreapp3.1 1.35μs 0.562ns 2.18ns 0.0102 0 0 768 B
#4573 ExecuteNonQuery net472 1.62μs 0.646ns 2.5ns 0.115 0.000807 0 730 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.18μs 1.01ns 3.77ns 0.0137 0 0 992 B
master CallElasticsearch netcoreapp3.1 1.54μs 0.435ns 1.63ns 0.0131 0 0 992 B
master CallElasticsearch net472 2.26μs 0.823ns 3.08ns 0.159 0.00113 0 1 KB
master CallElasticsearchAsync net6.0 1.29μs 0.421ns 1.52ns 0.0134 0 0 968 B
master CallElasticsearchAsync netcoreapp3.1 1.53μs 0.557ns 1.93ns 0.0137 0 0 1.04 KB
master CallElasticsearchAsync net472 2.61μs 0.678ns 2.63ns 0.167 0.00131 0 1.06 KB
#4573 CallElasticsearch net6.0 1.23μs 0.151ns 0.544ns 0.0142 0 0 992 B
#4573 CallElasticsearch netcoreapp3.1 1.52μs 0.578ns 2.24ns 0.0136 0 0 992 B
#4573 CallElasticsearch net472 2.41μs 0.42ns 1.63ns 0.158 0.0012 0 1 KB
#4573 CallElasticsearchAsync net6.0 1.33μs 0.394ns 1.42ns 0.0133 0 0 968 B
#4573 CallElasticsearchAsync netcoreapp3.1 1.6μs 0.485ns 1.81ns 0.0144 0 0 1.04 KB
#4573 CallElasticsearchAsync net472 2.43μs 1.11ns 4.14ns 0.168 0.00122 0 1.06 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.16μs 0.789ns 3.05ns 0.0128 0 0 912 B
master ExecuteAsync netcoreapp3.1 1.47μs 0.708ns 2.65ns 0.0124 0 0 912 B
master ExecuteAsync net472 1.7μs 0.52ns 1.95ns 0.139 0.000853 0 875 B
#4573 ExecuteAsync net6.0 1.21μs 1.55ns 6.02ns 0.0128 0 0 912 B
#4573 ExecuteAsync netcoreapp3.1 1.5μs 0.587ns 2.27ns 0.012 0 0 912 B
#4573 ExecuteAsync net472 1.66μs 1.12ns 4.33ns 0.138 0.000828 0 875 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 3.78μs 1.27ns 4.75ns 0.0264 0 0 1.94 KB
master SendAsync netcoreapp3.1 4.41μs 3.1ns 12ns 0.0335 0 0 2.48 KB
master SendAsync net472 7.11μs 5.56ns 20.8ns 0.481 0 0 3.05 KB
#4573 SendAsync net6.0 3.7μs 20ns 108ns 0.0273 0 0 1.94 KB
#4573 SendAsync netcoreapp3.1 4.51μs 2.7ns 9.74ns 0.0336 0 0 2.48 KB
#4573 SendAsync net472 7.13μs 3.04ns 11.4ns 0.485 0 0 3.05 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 1.31μs 0.515ns 1.93ns 0.023 0 0 1.62 KB
master EnrichedLog netcoreapp3.1 2.02μs 0.6ns 2.24ns 0.0222 0 0 1.62 KB
master EnrichedLog net472 2.3μs 3.27ns 12.7ns 0.244 0 0 1.54 KB
#4573 EnrichedLog net6.0 1.36μs 0.83ns 3.21ns 0.0225 0 0 1.62 KB
#4573 EnrichedLog netcoreapp3.1 1.91μs 5.76ns 21.6ns 0.0217 0 0 1.62 KB
#4573 EnrichedLog net472 2.26μs 7.86ns 30.4ns 0.245 0 0 1.54 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 113μs 120ns 448ns 0.0561 0 0 4.21 KB
master EnrichedLog netcoreapp3.1 120μs 122ns 472ns 0.0599 0 0 4.21 KB
master EnrichedLog net472 149μs 208ns 805ns 0.668 0.223 0 4.38 KB
#4573 EnrichedLog net6.0 112μs 116ns 451ns 0.0561 0 0 4.21 KB
#4573 EnrichedLog netcoreapp3.1 120μs 130ns 486ns 0.0595 0 0 4.21 KB
#4573 EnrichedLog net472 147μs 204ns 789ns 0.656 0.219 0 4.38 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.91μs 1.22ns 4.71ns 0.0303 0 0 2.18 KB
master EnrichedLog netcoreapp3.1 3.84μs 1.25ns 4.5ns 0.0285 0 0 2.18 KB
master EnrichedLog net472 4.54μs 3.51ns 13.6ns 0.315 0 0 1.99 KB
#4573 EnrichedLog net6.0 2.78μs 1.18ns 4.57ns 0.0301 0 0 2.18 KB
#4573 EnrichedLog netcoreapp3.1 3.94μs 1.79ns 6.94ns 0.0294 0 0 2.18 KB
#4573 EnrichedLog net472 4.61μs 2.11ns 7.9ns 0.314 0 0 1.99 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.25μs 0.825ns 3.09ns 0.0162 0 0 1.16 KB
master SendReceive netcoreapp3.1 1.68μs 0.986ns 3.69ns 0.0151 0 0 1.16 KB
master SendReceive net472 2.01μs 3.01ns 11.7ns 0.185 0.000999 0 1.16 KB
#4573 SendReceive net6.0 1.3μs 0.646ns 2.42ns 0.0164 0 0 1.16 KB
#4573 SendReceive netcoreapp3.1 1.55μs 1.45ns 5.62ns 0.0161 0 0 1.16 KB
#4573 SendReceive net472 2.02μs 1.53ns 5.73ns 0.184 0.00102 0 1.16 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.56μs 0.698ns 2.61ns 0.0217 0 0 1.53 KB
master EnrichedLog netcoreapp3.1 3.71μs 1.03ns 3.87ns 0.0204 0 0 1.58 KB
master EnrichedLog net472 4.09μs 1.18ns 4.56ns 0.311 0 0 1.96 KB
#4573 EnrichedLog net6.0 2.67μs 0.953ns 3.57ns 0.0215 0 0 1.53 KB
#4573 EnrichedLog netcoreapp3.1 3.48μs 1.7ns 6.58ns 0.0207 0 0 1.58 KB
#4573 EnrichedLog net472 4.01μs 0.94ns 3.64ns 0.31 0 0 1.96 KB
Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #4573

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0 1.148 551.89 480.66
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1 1.117 743.79 665.67

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 421ns 0.0958ns 0.358ns 0.00752 0 0 536 B
master StartFinishSpan netcoreapp3.1 580ns 1.56ns 5.85ns 0.0072 0 0 536 B
master StartFinishSpan net472 676ns 0.17ns 0.635ns 0.0852 0 0 538 B
master StartFinishScope net6.0 552ns 0.133ns 0.516ns 0.00921 0 0 656 B
master StartFinishScope netcoreapp3.1 744ns 0.306ns 1.19ns 0.00896 0 0 656 B
master StartFinishScope net472 842ns 0.381ns 1.42ns 0.0979 0 0 618 B
#4573 StartFinishSpan net6.0 421ns 0.213ns 0.824ns 0.00755 0 0 536 B
#4573 StartFinishSpan netcoreapp3.1 580ns 0.268ns 1.04ns 0.00732 0 0 536 B
#4573 StartFinishSpan net472 669ns 0.273ns 1.06ns 0.0854 0 0 538 B
#4573 StartFinishScope net6.0 481ns 0.217ns 0.842ns 0.00911 0 0 656 B
#4573 StartFinishScope netcoreapp3.1 665ns 0.831ns 3.11ns 0.00886 0 0 656 B
#4573 StartFinishScope net472 855ns 0.439ns 1.7ns 0.0978 0 0 618 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #4573

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑netcoreapp3.1 1.162 724.51 841.89

Faster 🎉 in #4573

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 1.134 632.17 557.30

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 632ns 0.26ns 0.973ns 0.00925 0 0 656 B
master RunOnMethodBegin netcoreapp3.1 725ns 0.349ns 1.35ns 0.00876 0 0 656 B
master RunOnMethodBegin net472 948ns 0.697ns 2.7ns 0.0979 0 0 618 B
#4573 RunOnMethodBegin net6.0 557ns 0.229ns 0.887ns 0.00902 0 0 656 B
#4573 RunOnMethodBegin netcoreapp3.1 842ns 0.337ns 1.22ns 0.00884 0 0 656 B
#4573 RunOnMethodBegin net472 1.04μs 0.283ns 1.06ns 0.0981 0 0 618 B

@andrewlock

andrewlock commented Aug 31, 2023 •
edited
Loading

Copy link
Copy Markdown
Member

Throughput/Crank Report:zap:

Throughput results for AspNetCoreSimpleController comparing the following branches/commits:

Cases where throughput results for the PR are worse than latest master (5% drop or greater), results are shown in red.

Note that these results are based on a single point-in-time result for each branch. For full results, see one of the many, many dashboards!

gantt
    title Throughput Linux x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4573) (11.244M)   : 0, 11243911
    master (11.294M)   : 0, 11294499
    benchmarks/2.37.0 (11.329M)   : 0, 11328644
    benchmarks/2.9.0 (11.385M)   : 0, 11384959

    section Automatic
    This PR (4573) (7.819M)   : 0, 7819132
    master (7.935M)   : 0, 7935342
    benchmarks/2.37.0 (7.961M)   : 0, 7961200
    benchmarks/2.9.0 (8.282M)   : 0, 8281551

    section Trace stats
    master (8.006M)   : 0, 8006141
    benchmarks/2.37.0 (7.902M)   : 0, 7902358

    section Manual
    This PR (4573) (9.934M)   : 0, 9934407
    master (10.125M)   : 0, 10124706
    benchmarks/2.37.0 (10.029M)   : 0, 10028769

    section Manual + Automatic
    This PR (4573) (7.508M)   : 0, 7508308
    master (7.666M)   : 0, 7665905
    benchmarks/2.37.0 (7.525M)   : 0, 7524503

    section Version Conflict
    master (6.871M)   : 0, 6870971
    benchmarks/2.37.0 (6.896M)   : 0, 6896060
Loading 
gantt
    title Throughput Linux arm64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4573) (9.458M)   : 0, 9457842
    master (9.768M)   : 0, 9767842
    benchmarks/2.37.0 (9.500M)   : 0, 9499914
    benchmarks/2.9.0 (9.505M)   : 0, 9504964

    section Automatic
    This PR (4573) (6.760M)   : 0, 6760017
    master (6.690M)   : 0, 6689913
    benchmarks/2.37.0 (6.711M)   : 0, 6710788

    section Trace stats
    master (6.778M)   : 0, 6777919
    benchmarks/2.37.0 (6.720M)   : 0, 6720194

    section Manual
    This PR (4573) (8.617M)   : 0, 8616850
    master (8.495M)   : 0, 8494830
    benchmarks/2.37.0 (8.540M)   : 0, 8539648

    section Manual + Automatic
    This PR (4573) (6.473M)   : 0, 6472895
    master (6.524M)   : 0, 6523526
    benchmarks/2.37.0 (6.398M)   : 0, 6398298

    section Version Conflict
    master (5.969M)   : 0, 5969418
    benchmarks/2.37.0 (5.962M)   : 0, 5961541
Loading 
gantt
    title Throughput Windows x64 (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4573) (10.039M)   : 0, 10038623
    master (10.364M)   : 0, 10363612
    benchmarks/2.37.0 (10.455M)   : 0, 10455227
    benchmarks/2.9.0 (10.491M)   : 0, 10490658

    section Automatic
    This PR (4573) (7.441M)   : 0, 7440719
    master (7.321M)   : 0, 7320645
    benchmarks/2.37.0 (7.444M)   : 0, 7443822
    benchmarks/2.9.0 (7.474M)   : 0, 7474016

    section Trace stats
    master (7.388M)   : 0, 7388260
    benchmarks/2.37.0 (7.455M)   : 0, 7454746

    section Manual
    This PR (4573) (9.070M)   : 0, 9069732
    master (9.227M)   : 0, 9226538
    benchmarks/2.37.0 (9.214M)   : 0, 9213981

    section Manual + Automatic
    This PR (4573) (7.072M)   : 0, 7071707
    master (7.239M)   : 0, 7239198
    benchmarks/2.37.0 (7.314M)   : 0, 7313712

    section Version Conflict
    master (6.459M)   : 0, 6459309
    benchmarks/2.37.0 (6.856M)   : 0, 6856254
Loading 
gantt
    title Throughput Linux x64 (ASM) (Total requests) 
    dateFormat  X
    axisFormat %s
    section Baseline
    This PR (4573) (7.472M)   : 0, 7472437
    master (7.328M)   : 0, 7328037
    benchmarks/2.37.0 (7.348M)   : 0, 7347922
    benchmarks/2.9.0 (7.671M)   : 0, 7671167

    section No attack
    This PR (4573) (2.119M)   : 0, 2118686
    master (2.132M)   : 0, 2131777
    benchmarks/2.37.0 (2.134M)   : 0, 2133940
    benchmarks/2.9.0 (3.175M)   : 0, 3174779

    section Attack
    This PR (4573) (1.675M)   : 0, 1675191
    master (1.640M)   : 0, 1640285
    benchmarks/2.37.0 (1.651M)   : 0, 1651416
    benchmarks/2.9.0 (2.518M)   : 0, 2518177

    section Blocking
    This PR (4573) (3.083M)   : 0, 3083207
    master (3.092M)   : 0, 3092137
    benchmarks/2.37.0 (3.122M)   : 0, 3122110
Loading

@lucaspimentel
lucaspimentel force-pushed the lpimentel/serverless-readytorun branch from 79a57a3 to 3d177c4 Compare September 1, 2023 17:48
andrewlock
andrewlock reviewed Sep 4, 2023
View reviewed changes
Comment thread tracer/build/_build/Build.Steps.Debugger.cs
andrewlock
andrewlock reviewed Sep 4, 2023
View reviewed changes
Comment thread tracer/build/_build/Build.Steps.Debugger.cs
andrewlock
andrewlock reviewed Sep 4, 2023
View reviewed changes
Comment thread tracer/build/_build/Build.Shared.Steps.cs Outdated
.Unlisted()
.OnlyWhenStatic(() => IsLinux)
.After(CompileNativeLoader)
.After(CompileNativeLoader, CompileNativeLoaderLinux)

@andrewlock andrewlock Sep 4, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: CompileNativeLoader includes CompileNativeLoaderLinux so this isn't necessary

Suggested change
.After(CompileNativeLoader, CompileNativeLoaderLinux)
.After(CompileNativeLoader)

@lucaspimentel lucaspimentel Sep 5, 2023

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was using Depends(CompileNativeLoaderLinux) directly somewhere because for AWS Lambda it will always be Linux. I don't think I'm doing that anymore.

@lucaspimentel lucaspimentel Sep 6, 2023

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I take that back. I had to replace PublishNativeTracer and PublishNativeLoader with Lambda-specific versions (PublishNativeTracerAwsLambda and PublishNativeLoaderAwsLambda), so I'm calling CompileTracerNativeSrcLinux and CompileNativeLoaderLinux directly:

.DependsOn(CompileTracerNativeSrcLinux)   // CompileNativeSrc
.DependsOn(PublishNativeTracerAwsLambda)  // PublishNativeTracer

.DependsOn(CompileNativeLoaderLinux)      // BuildNativeLoader -> CompileNativeLoader
.DependsOn(PublishNativeLoaderAwsLambda); // BuildNativeLoader -> PublishNativeLoader

@andrewlock andrewlock Sep 7, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would still suggest you depend on the non-platform specific versions, i.e. CompileNativeLoader. The high-level targets were added to avoid having to do exactly this 😄 Otherwise we should remove the sub-targets and do if else inside the main targets, as they're just adding noise and complexity otherwise 🤷

andrewlock
andrewlock reviewed Sep 4, 2023
View reviewed changes
Comment thread tracer/build/_build/Build.Steps.cs Outdated
andrewlock
andrewlock reviewed Sep 4, 2023
View reviewed changes
Comment thread tracer/build/_build/Build.Steps.cs Outdated
andrewlock
andrewlock reviewed Sep 4, 2023
View reviewed changes
Comment thread tracer/build/_build/Build.Steps.cs Outdated
andrewlock
andrewlock reviewed Sep 4, 2023
View reviewed changes
Comment thread tracer/build/_build/Build.Steps.cs Outdated
andrewlock
andrewlock reviewed Sep 4, 2023
View reviewed changes
Comment thread tracer/build/_build/Build.Steps.cs Outdated
andrewlock
andrewlock reviewed Sep 4, 2023
View reviewed changes
Comment thread tracer/build/_build/Build.Steps.cs Outdated
andrewlock
andrewlock reviewed Sep 4, 2023
View reviewed changes
Comment thread tracer/build/_build/Build.Steps.cs
andrewlock
andrewlock reviewed Sep 4, 2023
View reviewed changes
Comment thread tracer/build/_build/Build.Utilities.cs Outdated
andrewlock
andrewlock reviewed Sep 4, 2023
View reviewed changes
Comment thread tracer/build/_build/Build.Utilities.cs Outdated
andrewlock
andrewlock reviewed Sep 4, 2023
View reviewed changes
Comment thread tracer/build/_build/Build.cs Outdated
Comment on lines +180 to +185
.DependsOn(CompileTracerNativeSrcLinux) // CompileNativeSrc
.DependsOn(PublishNativeTracerUnix) // PublishNativeTracer
//.DependsOn(DownloadLibDdwaf)
//.DependsOn(CopyLibDdwaf)
.DependsOn(CompileNativeLoaderLinux) // BuildNativeLoader -> CompileNativeLoader
.DependsOn(PublishNativeLoaderUnix); // BuildNativeLoader -> PublishNativeLoader

@andrewlock andrewlock Sep 4, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Prefer the non-Linux specific versions, as that takes of everything for you. Any reason you didn't do that here?

Suggested change
.DependsOn(CompileTracerNativeSrcLinux) // CompileNativeSrc
.DependsOn(PublishNativeTracerUnix) // PublishNativeTracer
//.DependsOn(DownloadLibDdwaf)
//.DependsOn(CopyLibDdwaf)
.DependsOn(CompileNativeLoaderLinux) // BuildNativeLoader -> CompileNativeLoader
.DependsOn(PublishNativeLoaderUnix); // BuildNativeLoader -> PublishNativeLoader
.DependsOn(CompileNativeSrc)
.DependsOn(PublishNativeTracer)
//.DependsOn(DownloadLibDdwaf)
//.DependsOn(CopyLibDdwaf)
.DependsOn(CompileNativeLoader)
.DependsOn(PublishNativeLoader);

@lucaspimentel lucaspimentel Sep 5, 2023

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seemed redundant since it's always Linux when building the AWS Lambda layer. I reverted like you asked here, but then I ended up with Lambda-specific versions of PublishNativeTracer and PublishNativeLoader, so we'll see.

andrewlock
andrewlock reviewed Sep 4, 2023
View reviewed changes
Comment thread tracer/src/Datadog.Trace/Datadog.Trace.csproj Outdated
andrewlock
andrewlock reviewed Sep 4, 2023
View reviewed changes
Comment thread tracer/build/_build/docker/centos7.dockerfile
Comment on lines +88 to +93
COPY *.csproj *.props *.targets /build/
RUN dotnet restore /build
COPY . /build
RUN dotnet build /build
RUN dotnet build /build --no-restore

@andrewlock andrewlock Sep 4, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😭 FYI, we don't cache between builds... so this will actually slow down the build compared to the previous (more layers) unless you also go through and rebuild all the VMSS vms in Azure and AWS...😬 which you're welcome to do of course! 😄

@lucaspimentel lucaspimentel Sep 5, 2023

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to know. I'll keep this change local-only, then. It was shaving off ~20 seconds on each local build by not restoring every time.

@andrewlock andrewlock Sep 6, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I agree this is a good change to make, just be aware that there's more involved 😄

@andrewlock andrewlock Nov 1, 2023

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI, I incorporated this suggestion in the dockerfiles when I did the .NET 8 update

@lucaspimentel
lucaspimentel force-pushed the lpimentel/serverless-readytorun branch 3 times, most recently from e408106 to ffaa380 Compare September 7, 2023 20:13
@lucaspimentel lucaspimentel changed the title [build] Create AWS Lambda layers (with ReadyToRun) [build] Build AWS Lambda layers (with ReadyToRun) Sep 8, 2023
@lucaspimentel
lucaspimentel force-pushed the lpimentel/serverless-readytorun branch from 7140994 to f28b27d Compare September 8, 2023 20:25
@lucaspimentel
lucaspimentel force-pushed the lpimentel/serverless-readytorun branch from f28b27d to 8932bbe Compare October 18, 2023 11:18
@lucaspimentel
lucaspimentel force-pushed the lpimentel/serverless-readytorun branch from 8932bbe to f6913ce Compare October 26, 2023 18:39
@lucaspimentel
lucaspimentel force-pushed the lpimentel/serverless-readytorun branch from f6913ce to 9a2bb89 Compare October 30, 2023 17:14
@lucaspimentel
lucaspimentel force-pushed the lpimentel/serverless-readytorun branch from 9a2bb89 to ae3b0f6 Compare October 30, 2023 17:19
@duncanista duncanista mentioned this pull request Aug 29, 2024
Merged
@lucaspimentel

lucaspimentel commented Aug 29, 2024

Copy link
Copy Markdown
Member Author

closing this PR, superseded by #5962

duncanista added a commit that referenced this pull request Sep 6, 2024
@duncanista @andrewlock
[build] Build tracer with ReadyToRun (#5962)
5c5bec1 
## Summary of changes

Allows tracer publishing to be compiled with
[ReadyToRun](https://learn.microsoft.com/en-us/dotnet/core/deploying/ready-to-run)
to improve Serverless workloads init duration.

## Reason for change

It has showcased a 500ms init duration improvement for AWS Lambda.
Potentially could be used for other workloads in the future.

## Implementation details

Followed #4573 and
[ReadyToRun](https://learn.microsoft.com/en-us/dotnet/core/deploying/ready-to-run)
docs.

## Test coverage

- TBD
- Tested manually in AWS Lambda.

## Other details
Increases tracer size by 3x.

<!-- ⚠️ Note: where possible, please obtain 2 approvals prior to
merging. Unless CODEOWNERS specifies otherwise, for external teams it is
typically best to have one review from a team member, and one review
from apm-dotnet. Trivial changes do not require 2 reviews. -->

---------

Co-authored-by: Andrew Lock <[email protected]>
@duncanista duncanista mentioned this pull request Sep 6, 2024
Merged
duncanista added a commit that referenced this pull request Sep 10, 2024
@duncanista
[backport][build] Build tracer with ReadyToRun (5962 => v2) (#6005)
da56242 
cherry-pick `5c5bec1b893b52c2364a2bf2b66c58a92e6d7d83`
[5962](#5962)

## Summary of changes

Allows tracer publishing to be compiled with

[ReadyToRun](https://learn.microsoft.com/en-us/dotnet/core/deploying/ready-to-run)
to improve Serverless workloads init duration.

## Reason for change

It has showcased a 500ms init duration improvement for AWS Lambda.
Potentially could be used for other workloads in the future.

## Implementation details

Followed #4573 and

[ReadyToRun](https://learn.microsoft.com/en-us/dotnet/core/deploying/ready-to-run)
docs.

## Test coverage

- Tested manually in AWS Lambda.

## Other details
Increases tracer size by 3x.

<!-- ⚠️ Note: where possible, please obtain 2 approvals prior to
merging. Unless CODEOWNERS specifies otherwise, for external teams it is
typically best to have one review from a team member, and one review
from apm-dotnet. Trivial changes do not require 2 reviews. -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@andrewlock andrewlock andrewlock left review comments

Assignees

No one assigned

Labels

area:builds project files, build scripts, pipelines, versioning, releases, packages

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@lucaspimentel @andrewlock