[Code Origin] Fix smoke test path mismatches on macOS and Windows IIS#8659
Conversation
- macOS smoke tests build on the host (no Docker), so add /p:PathMap to the publish step so PDB paths emit /src/ValuesController.cs. - Windows IIS Fleet Dockerfile already remaps via /p:PathMap=C:\src=/src, so update smoke_test_iis_snapshots.json to expect /src/ValuesController.cs. Fixes regressions from #8272 (Code Origin on by default).
Snapshots difference summaryThe following differences have been observed in committed snapshots. It is meant to help the reviewer. 1 occurrences of : - "_dd.code_origin.frames.0.file": "C:\\src\\ValuesController.cs",
+ "_dd.code_origin.frames.0.file": "/src/ValuesController.cs",
|
andrewlock
left a comment
There was a problem hiding this comment.
Thanks, though we don't run the macos tests on PRs currently, so I started a run here to test it: https://dev.azure.com/datadoghq/dd-trace-dotnet/_build/results?buildId=201909&view=results
If that passes, then we can merge this!
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing This PR (8659) and master. ✅ No regressions detected - check the details below Full Metrics ComparisonFakeDbCommand
HttpMessageHandler
Comparison explanationExecution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). Duration chartsFakeDbCommand (.NET Framework 4.8)gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8659) - mean (74ms) : 71, 78
master - mean (75ms) : 71, 79
section Bailout
This PR (8659) - mean (77ms) : 75, 79
master - mean (77ms) : 75, 80
section CallTarget+Inlining+NGEN
This PR (8659) - mean (1,115ms) : 1055, 1174
master - mean (1,107ms) : 1054, 1161
FakeDbCommand (.NET Core 3.1)gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8659) - mean (115ms) : 109, 121
master - mean (116ms) : 111, 121
section Bailout
This PR (8659) - mean (116ms) : 113, 119
master - mean (115ms) : 112, 118
section CallTarget+Inlining+NGEN
This PR (8659) - mean (792ms) : 767, 817
master - mean (793ms) : 766, 820
FakeDbCommand (.NET 6)gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8659) - mean (104ms) : 98, 110
master - mean (104ms) : 99, 109
section Bailout
This PR (8659) - mean (102ms) : 99, 104
master - mean (106ms) : 101, 111
section CallTarget+Inlining+NGEN
This PR (8659) - mean (950ms) : 904, 996
master - mean (958ms) : 921, 995
FakeDbCommand (.NET 8)gantt
title Execution time (ms) FakeDbCommand (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8659) - mean (101ms) : 95, 106
master - mean (99ms) : 96, 102
section Bailout
This PR (8659) - mean (105ms) : 101, 109
master - mean (104ms) : 97, 110
section CallTarget+Inlining+NGEN
This PR (8659) - mean (825ms) : 781, 868
master - mean (824ms) : 777, 872
HttpMessageHandler (.NET Framework 4.8)gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8659) - mean (202ms) : 197, 206
master - mean (202ms) : 196, 208
section Bailout
This PR (8659) - mean (205ms) : 201, 208
master - mean (205ms) : 199, 210
section CallTarget+Inlining+NGEN
This PR (8659) - mean (1,206ms) : 1162, 1249
master - mean (1,197ms) : 1151, 1244
HttpMessageHandler (.NET Core 3.1)gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8659) - mean (289ms) : 279, 299
master - mean (287ms) : 277, 298
section Bailout
This PR (8659) - mean (290ms) : 286, 295
master - mean (289ms) : 280, 297
section CallTarget+Inlining+NGEN
This PR (8659) - mean (965ms) : 941, 989
master - mean (963ms) : 945, 981
HttpMessageHandler (.NET 6)gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8659) - mean (282ms) : 275, 288
master - mean (279ms) : 271, 287
section Bailout
This PR (8659) - mean (283ms) : 277, 289
master - mean (280ms) : 274, 285
section CallTarget+Inlining+NGEN
This PR (8659) - mean (1,165ms) : 1130, 1200
master - mean (1,157ms) : 1122, 1193
HttpMessageHandler (.NET 8)gantt
title Execution time (ms) HttpMessageHandler (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8659) - mean (278ms) : 272, 284
master - mean (278ms) : 271, 285
section Bailout
This PR (8659) - mean (278ms) : 270, 286
master - mean (279ms) : 272, 285
section CallTarget+Inlining+NGEN
This PR (8659) - mean (1,037ms) : 997, 1077
master - mean (1,035ms) : 999, 1072
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
BenchmarksBenchmark execution time: 2026-05-19 16:21:35 Comparing candidate commit 40dc31c in PR branch Some scenarios are present only in baseline or only in candidate runs. If you didn't create or remove some scenarios in your branch, this maybe a sign of crashed benchmarks 💥💥💥 Scenarios present only in baseline:
Found 1 performance improvements and 3 performance regressions! Performance is the same for 49 metrics, 19 unstable metrics, 88 known flaky benchmarks, 38 flaky benchmarks without significant changes.
|
Summary of changes
/p:PathMap=$(smokeTestAppDir)=/srcto the macOS smoke-test publish step in.azure-pipelines/ultimate-pipeline.yml, mirroring thePathMapalready used by every Linux/Windows smoke-test Dockerfile.tracer/build/smoke_test_snapshots/smoke_test_iis_snapshots.jsonso_dd.code_origin.frames.0.fileis/src/ValuesController.cs(the value the Windows IIS Fleet Dockerfile actually emits).Reason for change
smoke_macos_tool_testsfailed on master for all 13macos*jobs because the macOS pipeline buildsAspNetCoreSmokeTestdirectly on the host (no Docker, noPathMap), so PDB sequence points contained/Users/runner/work/1/s/tracer/test/test-applications/regression/AspNetCoreSmokeTest/ValuesController.csinstead of the snapshot's expected/src/ValuesController.cs./p:PathMap=C:\src=/srctosmoke.windows.iis.fleet-installer.dockerfile, so the runtime now emits/src/ValuesController.cs, butsmoke_test_iis_snapshots.jsonwas approved withC:\\src\\ValuesController.cs, breakingsmoke_win_fleet_tests.Implementation details
$(smokeTestAppDir)resolves to$(System.DefaultWorkingDirectory)/tracer/test/test-applications/regression/AspNetCoreSmokeTest, soPathMap=$(smokeTestAppDir)=/srcrewrites every embedded source path under that directory to/src/.... This matches the layout produced by the LinuxWORKDIR /srcDockerfiles, keeping a single sharedsmoke_test_snapshots.jsonfor all Linux/macOS smoke tests.WindowsFleetInstallerIisScenarioreadssmoke_test_iis_snapshots, and its Dockerfile already appliesPathMap=C:\src=/src, so updating the snapshot to/src/ValuesController.csaligns the expected value with actual runtime output rather than re-introducing a divergent path.+2 / -2).