Skip to content

ci: prevent running macos tests on release branches#1765

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit intomainfrom
julio/publishing/prevent-running-macos-when-releasing
Mar 19, 2026
Merged

ci: prevent running macos tests on release branches#1765
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit intomainfrom
julio/publishing/prevent-running-macos-when-releasing

Conversation

@hoolioh
Copy link
Copy Markdown
Contributor

@hoolioh hoolioh commented Mar 19, 2026

What does this PR do?

Avoid to wait for macos runners when releasing.

Motivation

Since the release process only modifies depencies and changelogs it's a waste of time running the tests on MacOS.

@hoolioh hoolioh requested a review from a team as a code owner March 19, 2026 10:31
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Mar 19, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.43%. Comparing base (2db515d) to head (904ad8c).

Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #1765      +/-   ##
==========================================
- Coverage   71.44%   71.43%   -0.01%     
==========================================
  Files         430      430              
  Lines       64152    64152              
==========================================
- Hits        45835    45829       -6     
- Misses      18317    18323       +6
Components Coverage Δ
libdd-crashtracker 63.83% <ø> (ø)
libdd-crashtracker-ffi 17.10% <ø> (ø)
libdd-alloc 98.77% <ø> (ø)
libdd-data-pipeline 87.69% <ø> (ø)
libdd-data-pipeline-ffi 73.57% <ø> (ø)
libdd-common 79.73% <ø> (ø)
libdd-common-ffi 73.40% <ø> (ø)
libdd-telemetry 62.48% <ø> (ø)
libdd-telemetry-ffi 16.75% <ø> (ø)
libdd-dogstatsd-client 82.64% <ø> (ø)
datadog-ipc 80.35% <ø> (-0.12%) ⬇️
libdd-profiling 81.47% <ø> (-0.02%) ⬇️
libdd-profiling-ffi 63.65% <ø> (ø)
datadog-sidecar 33.61% <ø> (ø)
datdog-sidecar-ffi 13.01% <ø> (ø)
spawn-worker 54.69% <ø> (ø)
libdd-tinybytes 93.16% <ø> (ø)
libdd-trace-normalization 81.71% <ø> (ø)
libdd-trace-obfuscation 91.80% <ø> (ø)
libdd-trace-protobuf 68.25% <ø> (ø)
libdd-trace-utils 89.05% <ø> (ø)
datadog-tracer-flare 86.88% <ø> (ø)
libdd-log 74.69% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pr-commenter
Copy link
Copy Markdown

pr-commenter bot commented Mar 19, 2026 •
edited
Loading

Benchmarks

Comparison

Benchmark execution time: 2026-03-19 10:47:04

Comparing candidate commit 904ad8c in PR branch julio/publishing/prevent-running-macos-when-releasing with baseline commit 2db515d in branch main.

Found 3 performance improvements and 0 performance regressions! Performance is the same for 56 metrics, 2 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:credit_card/is_card_number/378282246310005

  • 🟩 execution_time [-15.520µs; -15.483µs] or [-19.326%; -19.280%]
  • 🟩 throughput [+2975462.722op/s; +2982212.555op/s] or [+23.894%; +23.948%]

scenario:normalization/normalize_name/normalize_name/good

  • 🟩 throughput [+3878849.643op/s; +4084452.045op/s] or [+4.145%; +4.365%]

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 904ad8c 1773915977 julio/publishing/prevent-running-macos-when-releasing
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
profile_add_sample_frames_x1000 execution_time 4.218ms 4.221ms ± 0.002ms 4.221ms ± 0.001ms 4.222ms 4.224ms 4.227ms 4.230ms 0.22% 1.354 3.688 0.04% 0.000ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
profile_add_sample_frames_x1000 execution_time [4.221ms; 4.222ms] or [-0.006%; +0.006%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 904ad8c 1773915977 julio/publishing/prevent-running-macos-when-releasing
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
ip_address/quantize_peer_ip_address_benchmark execution_time 5.064µs 5.141µs ± 0.053µs 5.128µs ± 0.040µs 5.191µs 5.231µs 5.233µs 5.233µs 2.06% 0.483 -1.179 1.02% 0.004µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
ip_address/quantize_peer_ip_address_benchmark execution_time [5.134µs; 5.148µs] or [-0.142%; +0.142%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 904ad8c 1773915977 julio/publishing/prevent-running-macos-when-releasing
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 32.430µs 33.140µs ± 1.295µs 32.540µs ± 0.034µs 32.616µs 35.909µs 35.942µs 36.936µs 13.51% 1.706 0.969 3.90% 0.092µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [32.961µs; 33.319µs] or [-0.541%; +0.541%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 904ad8c 1773915977 julio/publishing/prevent-running-macos-when-releasing
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sdk_test_data/rules-based execution_time 145.075µs 146.964µs ± 1.625µs 146.651µs ± 0.551µs 147.292µs 148.560µs 152.746µs 161.673µs 10.24% 5.441 40.607 1.10% 0.115µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sdk_test_data/rules-based execution_time [146.739µs; 147.189µs] or [-0.153%; +0.153%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 904ad8c 1773915977 julio/publishing/prevent-running-macos-when-releasing
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 18.132µs 26.273µs ± 9.446µs 18.455µs ± 0.307µs 34.378µs 43.394µs 44.752µs 58.845µs 218.85% 0.716 -0.442 35.86% 0.668µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [24.964µs; 27.582µs] or [-4.983%; +4.983%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 904ad8c 1773915977 julio/publishing/prevent-running-macos-when-releasing
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
receiver_entry_point/report/2598 execution_time 3.389ms 3.425ms ± 0.020ms 3.420ms ± 0.010ms 3.432ms 3.467ms 3.475ms 3.484ms 1.87% 1.024 0.312 0.59% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
receiver_entry_point/report/2598 execution_time [3.422ms; 3.428ms] or [-0.082%; +0.082%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 904ad8c 1773915977 julio/publishing/prevent-running-macos-when-releasing
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 3.896µs 3.915µs ± 0.003µs 3.914µs ± 0.001µs 3.916µs 3.919µs 3.921µs 3.925µs 0.27% -0.773 12.782 0.07% 0.000µs 1 200
credit_card/is_card_number/ throughput 254803088.074op/s 255457521.378op/s ± 176167.929op/s 255487353.377op/s ± 81637.472op/s 255546816.181op/s 255642504.084op/s 255685077.900op/s 256704302.565op/s 0.48% 0.802 12.964 0.07% 12456.954op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 79.569µs 80.147µs ± 0.125µs 80.150µs ± 0.038µs 80.184µs 80.267µs 80.400µs 81.245µs 1.37% 2.211 32.807 0.16% 0.009µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 12308453.387op/s 12477097.282op/s ± 19390.051op/s 12476632.515op/s ± 5869.595op/s 12482954.216op/s 12491008.680op/s 12552547.918op/s 12567750.228op/s 0.73% -2.072 31.820 0.16% 1371.084op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 67.835µs 67.923µs ± 0.072µs 67.916µs ± 0.032µs 67.946µs 68.006µs 68.094µs 68.622µs 1.04% 5.238 44.407 0.11% 0.005µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 14572652.055op/s 14722598.012op/s ± 15603.247op/s 14724119.254op/s ± 6907.251op/s 14731408.353op/s 14738353.447op/s 14740569.811op/s 14741645.197op/s 0.12% -5.178 43.624 0.11% 1103.316op/s 1 200
credit_card/is_card_number/37828224631 execution_time 3.897µs 3.916µs ± 0.003µs 3.916µs ± 0.001µs 3.917µs 3.920µs 3.922µs 3.925µs 0.22% -1.489 13.564 0.07% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 254804369.306op/s 255365793.567op/s ± 169361.926op/s 255364989.461op/s ± 82485.452op/s 255450904.982op/s 255581760.840op/s 255654890.630op/s 256605661.702op/s 0.49% 1.516 13.763 0.07% 11975.697op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 64.641µs 64.802µs ± 0.072µs 64.792µs ± 0.052µs 64.849µs 64.928µs 64.958µs 64.993µs 0.31% 0.321 -0.466 0.11% 0.005µs 1 200
credit_card/is_card_number/378282246310005 throughput 15386338.044op/s 15431656.137op/s ± 17128.160op/s 15434045.693op/s ± 12278.389op/s 15444823.369op/s 15457286.418op/s 15465065.376op/s 15469980.503op/s 0.23% -0.316 -0.469 0.11% 1211.144op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 45.369µs 45.737µs ± 0.105µs 45.744µs ± 0.065µs 45.805µs 45.894µs 45.943µs 45.964µs 0.48% -0.468 0.362 0.23% 0.007µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 21756254.253op/s 21864159.289op/s ± 50269.453op/s 21860558.088op/s ± 30956.371op/s 21893550.712op/s 21946180.690op/s 21964892.165op/s 22041406.644op/s 0.83% 0.483 0.392 0.23% 3554.587op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 6.563µs 6.626µs ± 0.015µs 6.632µs ± 0.007µs 6.636µs 6.641µs 6.646µs 6.653µs 0.31% -1.307 1.880 0.22% 0.001µs 1 200
credit_card/is_card_number/x371413321323331 throughput 150309357.747op/s 150929514.756op/s ± 338230.169op/s 150782655.296op/s ± 168547.976op/s 151143075.777op/s 151670897.975op/s 151879590.116op/s 152373814.615op/s 1.06% 1.321 1.940 0.22% 23916.485op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 3.897µs 3.915µs ± 0.003µs 3.914µs ± 0.001µs 3.916µs 3.919µs 3.922µs 3.933µs 0.49% 0.627 16.055 0.07% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 254232953.805op/s 255450251.482op/s ± 183300.960op/s 255471655.423op/s ± 83274.471op/s 255544956.747op/s 255651761.279op/s 255714719.622op/s 256599245.508op/s 0.44% -0.589 16.033 0.07% 12961.335op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 65.034µs 65.645µs ± 0.177µs 65.710µs ± 0.082µs 65.777µs 65.835µs 65.867µs 65.886µs 0.27% -1.114 0.391 0.27% 0.013µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 15177794.470op/s 15233488.198op/s ± 41171.791op/s 15218459.495op/s ± 18987.737op/s 15254023.613op/s 15318215.833op/s 15336329.757op/s 15376669.612op/s 1.04% 1.123 0.418 0.27% 2911.285op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 53.366µs 53.418µs ± 0.028µs 53.413µs ± 0.018µs 53.434µs 53.476µs 53.496µs 53.501µs 0.17% 0.714 0.287 0.05% 0.002µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 18691072.170op/s 18720301.593op/s ± 9899.870op/s 18721932.351op/s ± 6348.321op/s 18727507.690op/s 18734433.884op/s 18736671.568op/s 18738511.449op/s 0.09% -0.711 0.282 0.05% 700.026op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 3.895µs 3.915µs ± 0.003µs 3.915µs ± 0.001µs 3.916µs 3.919µs 3.921µs 3.923µs 0.21% -1.902 19.531 0.06% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 254914922.394op/s 255443463.750op/s ± 165618.515op/s 255456353.840op/s ± 81534.404op/s 255535158.934op/s 255620438.122op/s 255667293.509op/s 256768653.956op/s 0.51% 1.936 19.826 0.06% 11710.998op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 50.154µs 50.212µs ± 0.066µs 50.200µs ± 0.020µs 50.224µs 50.274µs 50.338µs 50.793µs 1.18% 6.782 56.120 0.13% 0.005µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 19687843.338op/s 19915480.609op/s ± 26061.660op/s 19920465.549op/s ± 7759.957op/s 19927147.842op/s 19933031.195op/s 19936101.982op/s 19938400.312op/s 0.09% -6.734 55.548 0.13% 1842.838op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 45.479µs 45.733µs ± 0.084µs 45.737µs ± 0.060µs 45.794µs 45.863µs 45.890µs 45.897µs 0.35% -0.161 -0.559 0.18% 0.006µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 21787786.364op/s 21866061.652op/s ± 40171.447op/s 21863994.404op/s ± 28511.998op/s 21896371.784op/s 21933572.407op/s 21953729.631op/s 21988104.134op/s 0.57% 0.169 -0.552 0.18% 2840.550op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 6.564µs 6.628µs ± 0.016µs 6.635µs ± 0.007µs 6.639µs 6.644µs 6.648µs 6.655µs 0.31% -1.289 1.489 0.25% 0.001µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 150259284.279op/s 150887154.688op/s ± 374583.415op/s 150722545.539op/s ± 162518.790op/s 151131650.768op/s 151623192.930op/s 151962852.302op/s 152338911.431op/s 1.07% 1.302 1.543 0.25% 26487.047op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [3.914µs; 3.915µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/ throughput [255433106.197op/s; 255481936.558op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [80.130µs; 80.164µs] or [-0.022%; +0.022%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [12474410.007op/s; 12479784.556op/s] or [-0.022%; +0.022%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [67.913µs; 67.933µs] or [-0.015%; +0.015%] None None None
credit_card/is_card_number/ 378282246310005 throughput [14720435.552op/s; 14724760.472op/s] or [-0.015%; +0.015%] None None None
credit_card/is_card_number/37828224631 execution_time [3.916µs; 3.916µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number/37828224631 throughput [255342321.633op/s; 255389265.501op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number/378282246310005 execution_time [64.792µs; 64.812µs] or [-0.015%; +0.015%] None None None
credit_card/is_card_number/378282246310005 throughput [15429282.339op/s; 15434029.935op/s] or [-0.015%; +0.015%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [45.723µs; 45.752µs] or [-0.032%; +0.032%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [21857192.427op/s; 21871126.152op/s] or [-0.032%; +0.032%] None None None
credit_card/is_card_number/x371413321323331 execution_time [6.624µs; 6.628µs] or [-0.031%; +0.031%] None None None
credit_card/is_card_number/x371413321323331 throughput [150882639.307op/s; 150976390.204op/s] or [-0.031%; +0.031%] None None None
credit_card/is_card_number_no_luhn/ execution_time [3.914µs; 3.915µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number_no_luhn/ throughput [255424847.732op/s; 255475655.232op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [65.621µs; 65.670µs] or [-0.037%; +0.037%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [15227782.184op/s; 15239194.213op/s] or [-0.037%; +0.037%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [53.414µs; 53.422µs] or [-0.007%; +0.007%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [18718929.566op/s; 18721673.620op/s] or [-0.007%; +0.007%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [3.914µs; 3.915µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [255420510.616op/s; 255466416.883op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [50.203µs; 50.221µs] or [-0.018%; +0.018%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [19911868.714op/s; 19919092.505op/s] or [-0.018%; +0.018%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [45.721µs; 45.745µs] or [-0.025%; +0.025%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [21860494.276op/s; 21871629.029op/s] or [-0.025%; +0.025%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [6.625µs; 6.630µs] or [-0.034%; +0.034%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [150835241.029op/s; 150939068.346op/s] or [-0.034%; +0.034%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 904ad8c 1773915977 julio/publishing/prevent-running-macos-when-releasing
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 47.595ms 47.941ms ± 1.134ms 47.781ms ± 0.057ms 47.846ms 48.011ms 55.697ms 57.845ms 21.06% 7.884 61.791 2.36% 0.080ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [47.784ms; 48.098ms] or [-0.328%; +0.328%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 904ad8c 1773915977 julio/publishing/prevent-running-macos-when-releasing
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 161.155µs 162.007µs ± 0.602µs 161.885µs ± 0.170µs 162.144µs 162.486µs 162.855µs 169.491µs 4.70% 9.727 117.879 0.37% 0.043µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [161.923µs; 162.090µs] or [-0.051%; +0.051%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 904ad8c 1773915977 julio/publishing/prevent-running-macos-when-releasing
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching serializing traces from their internal representation to msgpack execution_time 14.429ms 14.482ms ± 0.029ms 14.479ms ± 0.014ms 14.494ms 14.527ms 14.577ms 14.664ms 1.27% 1.932 8.518 0.20% 0.002ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching serializing traces from their internal representation to msgpack execution_time [14.478ms; 14.486ms] or [-0.027%; +0.027%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 904ad8c 1773915977 julio/publishing/prevent-running-macos-when-releasing
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 12.888ms 12.913ms ± 0.015ms 12.911ms ± 0.008ms 12.919ms 12.940ms 12.968ms 12.975ms 0.49% 1.451 3.088 0.11% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [12.911ms; 12.915ms] or [-0.016%; +0.016%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 904ad8c 1773915977 julio/publishing/prevent-running-macos-when-releasing
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.403µs 2.458µs ± 0.020µs 2.457µs ± 0.009µs 2.466µs 2.501µs 2.508µs 2.512µs 2.24% 0.279 0.962 0.82% 0.001µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.456µs; 2.461µs] or [-0.113%; +0.113%] None None None

Group 13

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 904ad8c 1773915977 julio/publishing/prevent-running-macos-when-releasing
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 236.330ns 248.429ns ± 13.529ns 244.010ns ± 5.897ns 252.771ns 284.447ns 287.111ns 288.741ns 18.33% 1.659 1.862 5.43% 0.957ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [246.554ns; 250.304ns] or [-0.755%; +0.755%] None None None

Group 14

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 904ad8c 1773915977 julio/publishing/prevent-running-macos-when-releasing
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 85.633µs 85.879µs ± 0.267µs 85.846µs ± 0.082µs 85.941µs 86.075µs 86.424µs 89.184µs 3.89% 9.689 116.560 0.31% 0.019µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [85.842µs; 85.916µs] or [-0.043%; +0.043%] None None None

Group 15

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 904ad8c 1773915977 julio/publishing/prevent-running-macos-when-releasing
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 495.455µs 497.168µs ± 2.238µs 496.272µs ± 0.325µs 496.721µs 500.667µs 506.992µs 507.651µs 2.29% 2.462 6.273 0.45% 0.158µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 1969857.244op/s 2011430.703op/s ± 8954.381op/s 2015024.357op/s ± 1319.827op/s 2015978.873op/s 2017199.868op/s 2017941.292op/s 2018345.188op/s 0.16% -2.433 6.078 0.44% 633.170op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 369.847µs 370.442µs ± 0.250µs 370.417µs ± 0.164µs 370.591µs 370.847µs 371.138µs 371.363µs 0.26% 0.512 0.510 0.07% 0.018µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2692780.550op/s 2699481.847op/s ± 1819.435op/s 2699659.004op/s ± 1194.729op/s 2700792.868op/s 2702064.654op/s 2702957.743op/s 2703822.994op/s 0.15% -0.507 0.501 0.07% 128.653op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 168.634µs 168.956µs ± 0.172µs 168.937µs ± 0.109µs 169.050µs 169.234µs 169.291µs 170.199µs 0.75% 2.071 12.301 0.10% 0.012µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5875485.616op/s 5918720.502op/s ± 5995.971op/s 5919361.909op/s ± 3805.382op/s 5922939.257op/s 5926766.230op/s 5927921.574op/s 5930016.156op/s 0.18% -2.041 12.028 0.10% 423.979op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 36.822µs 37.025µs ± 0.108µs 37.061µs ± 0.070µs 37.110µs 37.156µs 37.185µs 37.205µs 0.39% -0.550 -1.094 0.29% 0.008µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 26878047.218op/s 27009016.338op/s ± 79275.395op/s 26982516.148op/s ± 50646.360op/s 27092935.761op/s 27144346.042op/s 27153858.402op/s 27157553.817op/s 0.65% 0.555 -1.091 0.29% 5605.617op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 46.216µs 46.297µs ± 0.103µs 46.280µs ± 0.023µs 46.311µs 46.374µs 46.472µs 47.608µs 2.87% 10.353 127.829 0.22% 0.007µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 21004684.753op/s 21599870.343op/s ± 47173.013op/s 21607563.910op/s ± 10846.134op/s 21617081.575op/s 21628467.334op/s 21633365.263op/s 21637612.297op/s 0.14% -10.202 125.197 0.22% 3335.636op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [496.858µs; 497.479µs] or [-0.062%; +0.062%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [2010189.712op/s; 2012671.694op/s] or [-0.062%; +0.062%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [370.407µs; 370.476µs] or [-0.009%; +0.009%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2699229.691op/s; 2699734.004op/s] or [-0.009%; +0.009%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [168.932µs; 168.979µs] or [-0.014%; +0.014%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5917889.519op/s; 5919551.486op/s] or [-0.014%; +0.014%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [37.010µs; 37.040µs] or [-0.041%; +0.041%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [26998029.531op/s; 27020003.145op/s] or [-0.041%; +0.041%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [46.282µs; 46.311µs] or [-0.031%; +0.031%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [21593332.617op/s; 21606408.069op/s] or [-0.030%; +0.030%] None None None

Group 16

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 904ad8c 1773915977 julio/publishing/prevent-running-macos-when-releasing
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 186.036µs 186.426µs ± 0.208µs 186.408µs ± 0.159µs 186.590µs 186.773µs 186.907µs 187.085µs 0.36% 0.337 -0.302 0.11% 0.015µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 5345149.678op/s 5364078.586op/s ± 5986.078op/s 5364574.110op/s ± 4584.114op/s 5368630.108op/s 5373119.890op/s 5374655.412op/s 5375307.316op/s 0.20% -0.332 -0.310 0.11% 423.280op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 17.868µs 17.935µs ± 0.048µs 17.925µs ± 0.016µs 17.947µs 18.001µs 18.092µs 18.373µs 2.50% 4.681 35.087 0.27% 0.003µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 54427692.810op/s 55757743.870op/s ± 148427.743op/s 55788044.815op/s ± 50802.717op/s 55825266.915op/s 55912496.287op/s 55949894.702op/s 55966288.541op/s 0.32% -4.560 33.603 0.27% 10495.426op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 10.148µs 10.250µs ± 0.050µs 10.247µs ± 0.035µs 10.284µs 10.333µs 10.372µs 10.418µs 1.68% 0.418 0.143 0.49% 0.004µs 1 200
normalization/normalize_name/normalize_name/good throughput 95984548.686op/s 97559928.706op/s ± 477367.400op/s 97593246.252op/s ± 332147.044op/s 97893204.124op/s 98238589.652op/s 98509646.066op/s 98539805.568op/s 0.97% -0.390 0.098 0.49% 33754.973op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [186.397µs; 186.454µs] or [-0.015%; +0.015%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [5363248.973op/s; 5364908.199op/s] or [-0.015%; +0.015%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [17.928µs; 17.942µs] or [-0.037%; +0.037%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [55737173.212op/s; 55778314.528op/s] or [-0.037%; +0.037%] None None None
normalization/normalize_name/normalize_name/good execution_time [10.243µs; 10.257µs] or [-0.068%; +0.068%] None None None
normalization/normalize_name/normalize_name/good throughput [97493770.175op/s; 97626087.236op/s] or [-0.068%; +0.068%] None None None

Group 17

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 904ad8c 1773915977 julio/publishing/prevent-running-macos-when-releasing
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
single_flag_killswitch/rules-based execution_time 190.754ns 192.930ns ± 2.015ns 192.686ns ± 1.507ns 193.878ns 196.906ns 199.880ns 201.090ns 4.36% 1.274 1.817 1.04% 0.142ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
single_flag_killswitch/rules-based execution_time [192.651ns; 193.209ns] or [-0.145%; +0.145%] None None None

Group 18

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 904ad8c 1773915977 julio/publishing/prevent-running-macos-when-releasing
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
profile_add_sample_timestamped_x1000 execution_time 4.140ms 4.143ms ± 0.007ms 4.142ms ± 0.001ms 4.144ms 4.146ms 4.151ms 4.233ms 2.19% 11.887 154.211 0.16% 0.000ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
profile_add_sample_timestamped_x1000 execution_time [4.142ms; 4.144ms] or [-0.023%; +0.023%] None None None

Group 19

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 904ad8c 1773915977 julio/publishing/prevent-running-macos-when-releasing
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.208µs 3.247µs ± 1.446µs 3.011µs ± 0.030µs 3.046µs 3.756µs 14.172µs 14.864µs 393.70% 7.244 54.044 44.41% 0.102µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [3.046µs; 3.447µs] or [-6.171%; +6.171%] None None None

Group 20

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 904ad8c 1773915977 julio/publishing/prevent-running-macos-when-releasing
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
profile_add_sample2_frames_x1000 execution_time 739.502µs 740.480µs ± 0.470µs 740.406µs ± 0.273µs 740.717µs 741.334µs 741.818µs 742.179µs 0.24% 0.743 0.750 0.06% 0.033µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
profile_add_sample2_frames_x1000 execution_time [740.415µs; 740.545µs] or [-0.009%; +0.009%] None None None

Baseline

Omitted due to size.

@hoolioh
Copy link
Copy Markdown
Contributor Author

hoolioh commented Mar 19, 2026

/merge

@gh-worker-devflow-routing-ef8351
Copy link
Copy Markdown

gh-worker-devflow-routing-ef8351 bot commented Mar 19, 2026 •
edited
Loading

View all feedbacks in Devflow UI.

2026-03-19 11:02:25 UTC ℹ️ Start processing command /merge

2026-03-19 11:02:31 UTC ℹ️ MergeQueue: waiting for PR to be ready

This pull request is not mergeable according to GitHub. Common reasons include pending required checks, missing approvals, or merge conflicts — but it could also be blocked by other repository rules or settings.
It will be added to the queue as soon as checks pass and/or get approvals. View in MergeQueue UI.
Note: if you pushed new commits since the last approval, you may need additional approval.
You can remove it from the waiting list with /remove command.

2026-03-19 11:12:15 UTC ℹ️ MergeQueue: merge request added to the queue

The expected merge time in main is approximately 45m (p90).

2026-03-19 12:06:06 UTC ℹ️ MergeQueue: Readding this merge request to the queue because another merge request processed with yours failed. No action is needed from your side.

2026-03-19 12:45:28 UTC ℹ️ MergeQueue: This merge request was merged

@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts bot commented Mar 19, 2026

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 8.70 MB 8.70 MB 0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 100.36 MB 100.36 MB 0% (0 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 11.28 MB 11.28 MB 0% (0 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 117.03 MB 117.03 MB 0% (0 B) 👌
libdatadog-x64-windows
Artifact Baseline Commit Change
libdatadog-x86-windows
Artifact Baseline Commit Change
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 87.57 MB 87.57 MB 0% (0 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 10.22 MB 10.22 MB 0% (0 B) 👌
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 109.90 MB 109.90 MB 0% (0 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 11.97 MB 11.97 MB 0% (0 B) 👌

@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d bot merged commit b6dfc1f into main Mar 19, 2026
146 checks passed
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d bot deleted the julio/publishing/prevent-running-macos-when-releasing branch March 19, 2026 12:45
bwoebi added a commit that referenced this pull request Mar 20, 2026
@bwoebi
Merge branch 'main' of github.com:DataDog/libdatadog into bob/prevent…
5d5ae75 
…-unprocessed

* 'main' of github.com:DataDog/libdatadog:
  feat(sidecar): add thread mode as fallback connection for restricted environments (#1447)
  feat(profiling-ffi): ProfilesDictionary_insert_strs (#1764)
  chore(release): merge release branch to main (#1760)
  fix(libdd-crashtracker-ffi)!: add missing fields for endpoint configuration (#1758)
  ci: prevent running macos tests on release branches (#1765)
  chore(datadog-tracer-flare): remove unnecessary features/deps (#1761)
  fix(profiling-ffi): Windows extern statics need __declspec(dllimport) (#1468)
  feat(profiling): thread id/name as well-known strs (#1757)
  ci: switch to ephemeral branches (#1731)
  chore(crashtracker): use weaker mem ordering for OP_COUNTERS (#1744)
  refactor(trace-utils)!: change header name type to accept dynamic values (#1722)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Aaalibaba42 Aaalibaba42 Aaalibaba42 approved these changes

Assignees

No one assigned

Labels

ci-build mergequeue-status: done

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@hoolioh @codecov-commenter @Aaalibaba42