Fix dd_dotnet.ArtifactTests.CreatedumpTests.DisableTelemetry flakiness#8354
Conversation
BenchmarksBenchmark execution time: 2026-03-24 10:31:47 Comparing candidate commit 367f679 in PR branch Found 7 performance improvements and 3 performance regressions! Performance is the same for 260 metrics, 18 unstable metrics.
|
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing This PR (8354) and master. ✅ No regressions detected - check the details below Full Metrics ComparisonFakeDbCommand
HttpMessageHandler
Comparison explanationExecution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). Duration chartsFakeDbCommand (.NET Framework 4.8)gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8354) - mean (69ms) : 67, 71
master - mean (69ms) : 66, 71
section Bailout
This PR (8354) - mean (73ms) : 71, 75
master - mean (73ms) : 71, 74
section CallTarget+Inlining+NGEN
This PR (8354) - mean (1,038ms) : 1001, 1075
master - mean (1,046ms) : 977, 1115
FakeDbCommand (.NET Core 3.1)gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8354) - mean (107ms) : 104, 110
master - mean (107ms) : 104, 110
section Bailout
This PR (8354) - mean (108ms) : 106, 109
master - mean (108ms) : 106, 110
section CallTarget+Inlining+NGEN
This PR (8354) - mean (772ms) : 753, 790
master - mean (769ms) : 751, 788
FakeDbCommand (.NET 6)gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8354) - mean (94ms) : 92, 97
master - mean (94ms) : 92, 97
section Bailout
This PR (8354) - mean (96ms) : 94, 98
master - mean (96ms) : 94, 97
section CallTarget+Inlining+NGEN
This PR (8354) - mean (750ms) : 731, 770
master - mean (751ms) : 730, 771
FakeDbCommand (.NET 8)gantt
title Execution time (ms) FakeDbCommand (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8354) - mean (94ms) : 91, 97
master - mean (93ms) : 91, 95
section Bailout
This PR (8354) - mean (95ms) : 92, 97
master - mean (95ms) : 93, 96
section CallTarget+Inlining+NGEN
This PR (8354) - mean (657ms) : 639, 675
master - mean (655ms) : 636, 674
HttpMessageHandler (.NET Framework 4.8)gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8354) - mean (193ms) : 189, 197
master - mean (194ms) : 190, 198
section Bailout
This PR (8354) - mean (196ms) : 193, 199
master - mean (196ms) : 193, 199
section CallTarget+Inlining+NGEN
This PR (8354) - mean (1,157ms) : 1090, 1224
master - mean (1,153ms) : 1094, 1212
HttpMessageHandler (.NET Core 3.1)gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8354) - mean (276ms) : 270, 281
master - mean (276ms) : 271, 280
section Bailout
This PR (8354) - mean (277ms) : 272, 281
master - mean (276ms) : 271, 281
section CallTarget+Inlining+NGEN
This PR (8354) - mean (939ms) : 919, 959
master - mean (937ms) : 913, 961
HttpMessageHandler (.NET 6)gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8354) - mean (270ms) : 266, 274
master - mean (270ms) : 264, 276
section Bailout
This PR (8354) - mean (271ms) : 266, 276
master - mean (272ms) : 263, 280
section CallTarget+Inlining+NGEN
This PR (8354) - mean (962ms) : 940, 983
master - mean (966ms) : 936, 995
HttpMessageHandler (.NET 8)gantt
title Execution time (ms) HttpMessageHandler (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8354) - mean (269ms) : 263, 275
master - mean (268ms) : 262, 273
section Bailout
This PR (8354) - mean (269ms) : 265, 272
master - mean (268ms) : 264, 271
section CallTarget+Inlining+NGEN
This PR (8354) - mean (845ms) : 825, 865
master - mean (848ms) : 825, 870
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
andrewlock
left a comment
There was a problem hiding this comment.
Thanks!
For future us:
when dd-dotnet is spawned, we go over the threads to capture the crash information, but we do no stop all the threads. For each thread, we suspend them, collect info, detach and resume.
When attaching to a process, we could instruct to stop all the threads, and resume them when done
Summary of changes
Fixes intermittent
CreatedumpTestsfailures on arm64 Linux by relaxing the createdump output assertion when the .NET runtime'screatedumptool fails due to a known ptrace race condition.Reason for change
On arm64 Linux,
createdumpintermittently fails withptrace(ATTACH, <tid>) FAILED No such process— a race condition where threads exit before createdump can attach to them. When this happens, createdump aborts without writing"Writing minidump with heap to file /dev/null", causing the test assertion to fail even though our crash tracking code correctly invoked createdump.This is a known .NET runtime limitation tracked in multiple issues:
Flaky test identified via CI Test Management.
Implementation details
AssertCreatedumpWasInvoked()helper that, only on arm64 Linux, also accepts"[createdump] Gathering state for process"as evidence that createdump was invoked (even if it failed due to the ptrace race). On all other platforms, the strict assertion is preserved.Passthrough,BashScript,DoNothingIfNotEnabled, andDisableTelemetry.Test coverage
Other details
The test's purpose is to verify that our crash tracking code correctly invokes createdump, not that createdump itself succeeds. The relaxed assertion still validates that createdump was started by our code.