[CI] Retry docker pull of ddapm-test-agent image on transient failure#8755
Conversation
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing This PR (8755) and master. ✅ No regressions detected - check the details below Full Metrics ComparisonFakeDbCommand
HttpMessageHandler
Comparison explanationExecution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). Duration chartsFakeDbCommand (.NET Framework 4.8)gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8755) - mean (75ms) : 71, 78
master - mean (74ms) : 70, 79
section Bailout
This PR (8755) - mean (77ms) : 75, 80
master - mean (77ms) : 75, 80
section CallTarget+Inlining+NGEN
This PR (8755) - mean (1,112ms) : 1060, 1164
master - mean (1,106ms) : 1050, 1161
FakeDbCommand (.NET Core 3.1)gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8755) - mean (114ms) : 110, 117
master - mean (114ms) : 108, 121
section Bailout
This PR (8755) - mean (114ms) : 112, 116
master - mean (113ms) : 111, 116
section CallTarget+Inlining+NGEN
This PR (8755) - mean (789ms) : 761, 817
master - mean (793ms) : 760, 825
FakeDbCommand (.NET 6)gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8755) - mean (104ms) : 99, 110
master - mean (101ms) : 98, 105
section Bailout
This PR (8755) - mean (103ms) : 101, 105
master - mean (102ms) : 99, 105
section CallTarget+Inlining+NGEN
This PR (8755) - mean (950ms) : 913, 987
master - mean (948ms) : 905, 991
FakeDbCommand (.NET 8)gantt
title Execution time (ms) FakeDbCommand (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8755) - mean (101ms) : 96, 105
master - mean (101ms) : 96, 106
section Bailout
This PR (8755) - mean (103ms) : 99, 108
master - mean (100ms) : 97, 102
section CallTarget+Inlining+NGEN
This PR (8755) - mean (820ms) : 786, 853
master - mean (824ms) : 778, 869
HttpMessageHandler (.NET Framework 4.8)gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8755) - mean (198ms) : 194, 202
master - mean (196ms) : 192, 200
section Bailout
This PR (8755) - mean (202ms) : 198, 205
master - mean (200ms) : 196, 203
section CallTarget+Inlining+NGEN
This PR (8755) - mean (1,198ms) : 1158, 1239
master - mean (1,196ms) : 1159, 1232
HttpMessageHandler (.NET Core 3.1)gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8755) - mean (285ms) : 279, 292
master - mean (285ms) : 278, 292
section Bailout
This PR (8755) - mean (286ms) : 280, 291
master - mean (283ms) : 277, 290
section CallTarget+Inlining+NGEN
This PR (8755) - mean (960ms) : 943, 977
master - mean (956ms) : 934, 979
HttpMessageHandler (.NET 6)gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8755) - mean (276ms) : 271, 281
master - mean (276ms) : 269, 283
section Bailout
This PR (8755) - mean (277ms) : 272, 282
master - mean (275ms) : 270, 280
section CallTarget+Inlining+NGEN
This PR (8755) - mean (1,152ms) : 1115, 1189
master - mean (1,161ms) : 1128, 1194
HttpMessageHandler (.NET 8)gantt
title Execution time (ms) HttpMessageHandler (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8755) - mean (276ms) : 268, 284
master - mean (277ms) : 268, 287
section Bailout
This PR (8755) - mean (277ms) : 270, 284
master - mean (276ms) : 270, 283
section CallTarget+Inlining+NGEN
This PR (8755) - mean (1,041ms) : 993, 1089
master - mean (1,038ms) : 993, 1083
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
BenchmarksBenchmark execution time: 2026-06-05 09:51:57 Comparing candidate commit 7e79c6e in PR branch Found 0 performance improvements and 1 performance regressions! Performance is the same for 71 metrics, 0 unstable metrics, 59 known flaky benchmarks, 67 flaky benchmarks without significant changes.
|
andrewlock
left a comment
There was a problem hiding this comment.
Thanks! We should probably pin these and rebuild the VMs sometime soon so that we don't have to pull at all! 😄
…#8755) ## Summary of changes Add `retryCountOnTaskFailure: 3` to both `docker pull ddapm-test-agent:latest` steps (x64 and ARM64) in `ultimate-pipeline.yml`. ## Reason for change These standalone `script` steps had no retry, so [a transient registry/network error](https://dev.azure.com/datadoghq/a51c4863-3eb4-4c5d-878a-58b41a049e4e/_apis/build/builds/202885/logs/6041) during the pull (e.g. `failed to copy: failed to send write: EOF`) failed the whole job. Retry mechanisms exist elsewhere in the pipeline (`retryCountOnTaskFailure`, `retryCountForRunCommand`) but were never wired into these two steps. ## Implementation details `retryCountOnTaskFailure` is the native Azure DevOps step-level retry; it re-runs the step up to 3 times on failure. ## Test coverage N/A — CI configuration change. ## Other details
Summary of changes
Add
retryCountOnTaskFailure: 3to bothdocker pull ddapm-test-agent:lateststeps (x64 and ARM64) inultimate-pipeline.yml.Reason for change
These standalone
scriptsteps had no retry, so a transient registry/network error during the pull (e.g.failed to copy: failed to send write: EOF) failed the whole job. Retry mechanisms exist elsewhere in the pipeline (retryCountOnTaskFailure,retryCountForRunCommand) but were never wired into these two steps.Implementation details
retryCountOnTaskFailureis the native Azure DevOps step-level retry; it re-runs the step up to 3 times on failure.Test coverage
N/A — CI configuration change.
Other details