Skip to content

fix(ci): handle new crate addition in semver-check#1769

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit intomainfrom
yannham/fix-semver-check-new-crate
Mar 20, 2026
Merged

fix(ci): handle new crate addition in semver-check#1769
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit intomainfrom
yannham/fix-semver-check-new-crate

Conversation

@yannham
Copy link
Copy Markdown
Contributor

@yannham yannham commented Mar 20, 2026

What does this PR do?

When a crate doesn't exist in the baseline (new crate being added), cargo semver-checks exits with code 1 and prints package 'X' not found. The semver-level script had no handler for this case, causing it to exit with Error: unknown level ().

Added an elif branch to detect this message and treat it as a minor change — a new crate is purely additive.

Motivation

Fixes the semver-check CI failure on #1624, which adds the new libdd-http-client crate.

Additional Notes

N/A

How to test the change?

Re-run the semver-check workflow on #1624.

@yannham @claude
fix(ci): handle new crate addition in semver-check
624a6d4 
When a crate doesn't exist in the baseline, cargo-semver-checks exits
with code 1 and "package not found". Treat this as a minor change
instead of an error.

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
@yannham yannham requested a review from a team as a code owner March 20, 2026 12:22
@pr-commenter
Copy link
Copy Markdown

pr-commenter bot commented Mar 20, 2026 •
edited
Loading

Benchmarks

Comparison

Benchmark execution time: 2026-03-20 12:35:44

Comparing candidate commit 624a6d4 in PR branch yannham/fix-semver-check-new-crate with baseline commit d88e70e in branch main.

Found 0 performance improvements and 2 performance regressions! Performance is the same for 57 metrics, 2 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:credit_card/is_card_number/378282246310005

  • 🟥 execution_time [+14.587µs; +14.677µs] or [+22.518%; +22.656%]
  • 🟥 throughput [-2852105.252op/s; -2836299.725op/s] or [-18.476%; -18.374%]

Candidate

Candidate benchmark details

Group 1

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 624a6d4 1774009052 yannham/fix-semver-check-new-crate
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sql/obfuscate_sql_string execution_time 85.958µs 86.252µs ± 0.170µs 86.224µs ± 0.058µs 86.301µs 86.430µs 86.784µs 87.768µs 1.79% 4.944 36.795 0.20% 0.012µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sql/obfuscate_sql_string execution_time [86.229µs; 86.276µs] or [-0.027%; +0.027%] None None None

Group 2

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 624a6d4 1774009052 yannham/fix-semver-check-new-crate
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
credit_card/is_card_number/ execution_time 3.892µs 3.913µs ± 0.007µs 3.913µs ± 0.002µs 3.915µs 3.917µs 3.919µs 3.984µs 1.83% 7.710 70.671 0.18% 0.001µs 1 200
credit_card/is_card_number/ throughput 250993417.413op/s 255532297.212op/s ± 464392.131op/s 255585929.996op/s ± 119071.158op/s 255694798.861op/s 255784210.673op/s 255846107.325op/s 256950377.011op/s 0.53% -7.637 69.871 0.18% 32837.482op/s 1 200
credit_card/is_card_number/ 3782-8224-6310-005 execution_time 74.756µs 76.676µs ± 0.877µs 76.640µs ± 0.621µs 77.295µs 78.078µs 78.627µs 79.869µs 4.21% 0.339 -0.024 1.14% 0.062µs 1 200
credit_card/is_card_number/ 3782-8224-6310-005 throughput 12520441.293op/s 13043504.939op/s ± 148580.525op/s 13048034.919op/s ± 104866.641op/s 13141197.394op/s 13274837.681op/s 13298309.440op/s 13376855.426op/s 2.52% -0.277 -0.139 1.14% 10506.230op/s 1 200
credit_card/is_card_number/ 378282246310005 execution_time 68.324µs 68.523µs ± 0.269µs 68.506µs ± 0.055µs 68.560µs 68.642µs 68.670µs 72.150µs 5.32% 12.353 164.251 0.39% 0.019µs 1 200
credit_card/is_card_number/ 378282246310005 throughput 13860058.596op/s 14593894.587op/s ± 54639.438op/s 14597292.933op/s ± 11653.586op/s 14608946.519op/s 14624728.663op/s 14631733.486op/s 14636176.237op/s 0.27% -12.190 161.345 0.37% 3863.592op/s 1 200
credit_card/is_card_number/37828224631 execution_time 3.890µs 3.913µs ± 0.003µs 3.913µs ± 0.001µs 3.914µs 3.916µs 3.918µs 3.919µs 0.17% -2.774 22.754 0.07% 0.000µs 1 200
credit_card/is_card_number/37828224631 throughput 255149845.902op/s 255590761.744op/s ± 175576.212op/s 255582036.128op/s ± 91124.275op/s 255673678.395op/s 255821943.925op/s 255921774.906op/s 257060363.443op/s 0.58% 2.809 23.110 0.07% 12415.113op/s 1 200
credit_card/is_card_number/378282246310005 execution_time 78.861µs 79.412µs ± 0.290µs 79.454µs ± 0.159µs 79.583µs 79.677µs 80.050µs 81.121µs 2.10% 1.473 8.870 0.36% 0.020µs 1 200
credit_card/is_card_number/378282246310005 throughput 12327313.269op/s 12592670.585op/s ± 45670.546op/s 12585920.590op/s ± 25251.292op/s 12618770.767op/s 12667204.481op/s 12677913.264op/s 12680520.185op/s 0.75% -1.380 8.294 0.36% 3229.395op/s 1 200
credit_card/is_card_number/37828224631000521389798 execution_time 45.363µs 45.696µs ± 0.130µs 45.709µs ± 0.096µs 45.790µs 45.891µs 45.929µs 46.145µs 0.96% -0.052 -0.212 0.28% 0.009µs 1 200
credit_card/is_card_number/37828224631000521389798 throughput 21670610.091op/s 21883915.583op/s ± 62396.000op/s 21877633.408op/s ± 46055.032op/s 21931261.384op/s 21987615.812op/s 22024348.680op/s 22044337.504op/s 0.76% 0.067 -0.225 0.28% 4412.063op/s 1 200
credit_card/is_card_number/x371413321323331 execution_time 6.428µs 6.436µs ± 0.004µs 6.435µs ± 0.003µs 6.438µs 6.444µs 6.448µs 6.451µs 0.25% 0.751 0.395 0.07% 0.000µs 1 200
credit_card/is_card_number/x371413321323331 throughput 155012112.758op/s 155382198.837op/s ± 107856.705op/s 155397413.832op/s ± 73062.182op/s 155466072.457op/s 155521875.830op/s 155564899.250op/s 155579042.244op/s 0.12% -0.748 0.386 0.07% 7626.621op/s 1 200
credit_card/is_card_number_no_luhn/ execution_time 3.891µs 3.910µs ± 0.003µs 3.910µs ± 0.002µs 3.912µs 3.914µs 3.916µs 3.919µs 0.21% -1.644 14.212 0.07% 0.000µs 1 200
credit_card/is_card_number_no_luhn/ throughput 255194805.519op/s 255724268.864op/s ± 171428.794op/s 255732882.907op/s ± 100871.483op/s 255829068.578op/s 255934058.261op/s 255981531.635op/s 257004838.684op/s 0.50% 1.671 14.436 0.07% 12121.846op/s 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time 61.163µs 62.640µs ± 0.484µs 62.579µs ± 0.335µs 62.973µs 63.513µs 63.813µs 63.944µs 2.18% 0.285 -0.019 0.77% 0.034µs 1 200
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput 15638609.415op/s 15965084.133op/s ± 123142.297op/s 15979752.364op/s ± 85531.290op/s 16051359.181op/s 16137443.242op/s 16199585.436op/s 16349767.219op/s 2.32% -0.241 -0.023 0.77% 8707.475op/s 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time 54.019µs 54.225µs ± 0.075µs 54.233µs ± 0.047µs 54.272µs 54.357µs 54.384µs 54.438µs 0.38% -0.052 0.164 0.14% 0.005µs 1 200
credit_card/is_card_number_no_luhn/ 378282246310005 throughput 18369683.627op/s 18441750.230op/s ± 25488.710op/s 18439093.829op/s ± 15934.281op/s 18457350.602op/s 18486621.974op/s 18501612.642op/s 18512153.229op/s 0.40% 0.061 0.164 0.14% 1802.324op/s 1 200
credit_card/is_card_number_no_luhn/37828224631 execution_time 3.891µs 3.911µs ± 0.003µs 3.911µs ± 0.002µs 3.913µs 3.917µs 3.921µs 3.922µs 0.28% -0.313 6.566 0.09% 0.000µs 1 200
credit_card/is_card_number_no_luhn/37828224631 throughput 254976732.999op/s 255665226.340op/s ± 223095.589op/s 255692047.977op/s ± 112627.318op/s 255803485.317op/s 255923340.460op/s 256012047.205op/s 257025532.074op/s 0.52% 0.336 6.682 0.09% 15775.240op/s 1 200
credit_card/is_card_number_no_luhn/378282246310005 execution_time 50.242µs 50.491µs ± 0.105µs 50.476µs ± 0.060µs 50.550µs 50.671µs 50.741µs 51.045µs 1.13% 1.375 5.421 0.21% 0.007µs 1 200
credit_card/is_card_number_no_luhn/378282246310005 throughput 19590599.780op/s 19805699.981op/s ± 41231.393op/s 19811202.230op/s ± 23669.968op/s 19830560.813op/s 19859705.359op/s 19892465.447op/s 19903586.626op/s 0.47% -1.341 5.243 0.21% 2915.500op/s 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time 45.378µs 45.692µs ± 0.127µs 45.690µs ± 0.098µs 45.787µs 45.885µs 45.946µs 45.998µs 0.67% -0.179 -0.522 0.28% 0.009µs 1 200
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput 21740284.996op/s 21885791.354op/s ± 61074.889op/s 21886771.645op/s ± 46931.683op/s 21933684.656op/s 21985831.410op/s 22028399.037op/s 22037148.192op/s 0.69% 0.192 -0.516 0.28% 4318.647op/s 1 200
credit_card/is_card_number_no_luhn/x371413321323331 execution_time 6.427µs 6.436µs ± 0.004µs 6.435µs ± 0.003µs 6.438µs 6.442µs 6.446µs 6.448µs 0.19% 0.590 0.436 0.06% 0.000µs 1 200
credit_card/is_card_number_no_luhn/x371413321323331 throughput 155094517.509op/s 155386952.565op/s ± 90662.773op/s 155390634.202op/s ± 60397.299op/s 155448822.016op/s 155508325.852op/s 155553935.155op/s 155586578.854op/s 0.13% -0.586 0.429 0.06% 6410.826op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
credit_card/is_card_number/ execution_time [3.912µs; 3.914µs] or [-0.026%; +0.026%] None None None
credit_card/is_card_number/ throughput [255467936.929op/s; 255596657.495op/s] or [-0.025%; +0.025%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 execution_time [76.555µs; 76.798µs] or [-0.158%; +0.158%] None None None
credit_card/is_card_number/ 3782-8224-6310-005 throughput [13022913.107op/s; 13064096.771op/s] or [-0.158%; +0.158%] None None None
credit_card/is_card_number/ 378282246310005 execution_time [68.486µs; 68.560µs] or [-0.054%; +0.054%] None None None
credit_card/is_card_number/ 378282246310005 throughput [14586322.087op/s; 14601467.088op/s] or [-0.052%; +0.052%] None None None
credit_card/is_card_number/37828224631 execution_time [3.912µs; 3.913µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/37828224631 throughput [255566428.569op/s; 255615094.918op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/378282246310005 execution_time [79.372µs; 79.452µs] or [-0.051%; +0.051%] None None None
credit_card/is_card_number/378282246310005 throughput [12586341.086op/s; 12599000.083op/s] or [-0.050%; +0.050%] None None None
credit_card/is_card_number/37828224631000521389798 execution_time [45.678µs; 45.714µs] or [-0.040%; +0.040%] None None None
credit_card/is_card_number/37828224631000521389798 throughput [21875268.098op/s; 21892563.069op/s] or [-0.040%; +0.040%] None None None
credit_card/is_card_number/x371413321323331 execution_time [6.435µs; 6.436µs] or [-0.010%; +0.010%] None None None
credit_card/is_card_number/x371413321323331 throughput [155367250.935op/s; 155397146.739op/s] or [-0.010%; +0.010%] None None None
credit_card/is_card_number_no_luhn/ execution_time [3.910µs; 3.911µs] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/ throughput [255700510.481op/s; 255748027.246op/s] or [-0.009%; +0.009%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 execution_time [62.573µs; 62.708µs] or [-0.107%; +0.107%] None None None
credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 throughput [15948017.795op/s; 15982150.471op/s] or [-0.107%; +0.107%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 execution_time [54.215µs; 54.235µs] or [-0.019%; +0.019%] None None None
credit_card/is_card_number_no_luhn/ 378282246310005 throughput [18438217.740op/s; 18445282.720op/s] or [-0.019%; +0.019%] None None None
credit_card/is_card_number_no_luhn/37828224631 execution_time [3.911µs; 3.912µs] or [-0.012%; +0.012%] None None None
credit_card/is_card_number_no_luhn/37828224631 throughput [255634307.437op/s; 255696145.243op/s] or [-0.012%; +0.012%] None None None
credit_card/is_card_number_no_luhn/378282246310005 execution_time [50.476µs; 50.505µs] or [-0.029%; +0.029%] None None None
credit_card/is_card_number_no_luhn/378282246310005 throughput [19799985.707op/s; 19811414.256op/s] or [-0.029%; +0.029%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 execution_time [45.674µs; 45.710µs] or [-0.039%; +0.039%] None None None
credit_card/is_card_number_no_luhn/37828224631000521389798 throughput [21877326.962op/s; 21894255.746op/s] or [-0.039%; +0.039%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 execution_time [6.435µs; 6.436µs] or [-0.008%; +0.008%] None None None
credit_card/is_card_number_no_luhn/x371413321323331 throughput [155374387.576op/s; 155399517.553op/s] or [-0.008%; +0.008%] None None None

Group 3

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 624a6d4 1774009052 yannham/fix-semver-check-new-crate
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
profile_add_sample2_frames_x1000 execution_time 726.534µs 727.474µs ± 0.457µs 727.463µs ± 0.305µs 727.769µs 728.199µs 728.719µs 729.265µs 0.25% 0.510 0.873 0.06% 0.032µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
profile_add_sample2_frames_x1000 execution_time [727.411µs; 727.537µs] or [-0.009%; +0.009%] None None None

Group 4

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 624a6d4 1774009052 yannham/fix-semver-check-new-crate
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
sdk_test_data/rules-based execution_time 144.617µs 146.389µs ± 1.597µs 146.094µs ± 0.491µs 146.629µs 148.201µs 152.307µs 161.915µs 10.83% 5.928 48.617 1.09% 0.113µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
sdk_test_data/rules-based execution_time [146.167µs; 146.610µs] or [-0.151%; +0.151%] None None None

Group 5

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 624a6d4 1774009052 yannham/fix-semver-check-new-crate
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
single_flag_killswitch/rules-based execution_time 190.788ns 193.147ns ± 2.150ns 192.780ns ± 1.349ns 194.196ns 196.339ns 200.947ns 206.298ns 7.01% 1.993 7.469 1.11% 0.152ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
single_flag_killswitch/rules-based execution_time [192.849ns; 193.445ns] or [-0.154%; +0.154%] None None None

Group 6

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 624a6d4 1774009052 yannham/fix-semver-check-new-crate
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
redis/obfuscate_redis_string execution_time 34.521µs 35.001µs ± 0.754µs 34.645µs ± 0.072µs 34.841µs 36.555µs 36.590µs 38.317µs 10.60% 1.797 1.941 2.15% 0.053µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
redis/obfuscate_redis_string execution_time [34.896µs; 35.105µs] or [-0.299%; +0.299%] None None None

Group 7

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 624a6d4 1774009052 yannham/fix-semver-check-new-crate
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
receiver_entry_point/report/2598 execution_time 3.576ms 3.605ms ± 0.016ms 3.601ms ± 0.007ms 3.609ms 3.637ms 3.672ms 3.682ms 2.25% 1.931 5.385 0.45% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
receiver_entry_point/report/2598 execution_time [3.603ms; 3.607ms] or [-0.062%; +0.062%] None None None

Group 8

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 624a6d4 1774009052 yannham/fix-semver-check-new-crate
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
concentrator/add_spans_to_concentrator execution_time 12.997ms 13.035ms ± 0.018ms 13.032ms ± 0.011ms 13.045ms 13.061ms 13.087ms 13.168ms 1.05% 2.419 14.140 0.14% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
concentrator/add_spans_to_concentrator execution_time [13.033ms; 13.038ms] or [-0.019%; +0.019%] None None None

Group 9

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 624a6d4 1774009052 yannham/fix-semver-check-new-crate
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
ip_address/quantize_peer_ip_address_benchmark execution_time 4.974µs 5.045µs ± 0.053µs 5.028µs ± 0.035µs 5.093µs 5.128µs 5.132µs 5.151µs 2.44% 0.407 -1.407 1.04% 0.004µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
ip_address/quantize_peer_ip_address_benchmark execution_time [5.038µs; 5.053µs] or [-0.145%; +0.145%] None None None

Group 10

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 624a6d4 1774009052 yannham/fix-semver-check-new-crate
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching deserializing traces from msgpack to their internal representation execution_time 48.755ms 49.008ms ± 0.983ms 48.879ms ± 0.029ms 48.908ms 49.002ms 51.477ms 59.063ms 20.83% 9.098 84.782 2.00% 0.070ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching deserializing traces from msgpack to their internal representation execution_time [48.872ms; 49.145ms] or [-0.278%; +0.278%] None None None

Group 11

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 624a6d4 1774009052 yannham/fix-semver-check-new-crate
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching serializing traces from their internal representation to msgpack execution_time 13.910ms 13.969ms ± 0.031ms 13.962ms ± 0.013ms 13.978ms 14.027ms 14.067ms 14.137ms 1.25% 1.810 5.203 0.22% 0.002ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching serializing traces from their internal representation to msgpack execution_time [13.964ms; 13.973ms] or [-0.031%; +0.031%] None None None

Group 12

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 624a6d4 1774009052 yannham/fix-semver-check-new-crate
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
profile_add_sample_timestamped_x1000 execution_time 4.226ms 4.236ms ± 0.008ms 4.235ms ± 0.002ms 4.237ms 4.239ms 4.240ms 4.340ms 2.48% 12.195 160.746 0.18% 0.001ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
profile_add_sample_timestamped_x1000 execution_time [4.235ms; 4.237ms] or [-0.025%; +0.025%] None None None

Group 13

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 624a6d4 1774009052 yannham/fix-semver-check-new-crate
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
write only interface execution_time 1.205µs 3.172µs ± 1.383µs 2.967µs ± 0.024µs 2.990µs 3.645µs 13.558µs 14.534µs 389.84% 7.316 54.842 43.50% 0.098µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
write only interface execution_time [2.980µs; 3.364µs] or [-6.044%; +6.044%] None None None

Group 14

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 624a6d4 1774009052 yannham/fix-semver-check-new-crate
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
profile_add_sample_frames_x1000 execution_time 4.274ms 4.280ms ± 0.004ms 4.279ms ± 0.002ms 4.280ms 4.286ms 4.295ms 4.299ms 0.47% 2.556 8.892 0.09% 0.000ms 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
profile_add_sample_frames_x1000 execution_time [4.279ms; 4.280ms] or [-0.012%; +0.012%] None None None

Group 15

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 624a6d4 1774009052 yannham/fix-semver-check-new-crate
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
two way interface execution_time 17.441µs 25.046µs ± 9.418µs 17.903µs ± 0.375µs 33.697µs 40.562µs 45.516µs 67.865µs 279.07% 0.994 0.764 37.51% 0.666µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
two way interface execution_time [23.741µs; 26.352µs] or [-5.211%; +5.211%] None None None

Group 16

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 624a6d4 1774009052 yannham/fix-semver-check-new-crate
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time 495.231µs 496.041µs ± 0.400µs 496.013µs ± 0.274µs 496.281µs 496.735µs 496.948µs 497.473µs 0.29% 0.566 0.144 0.08% 0.028µs 1 200
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput 2010157.998op/s 2015962.000op/s ± 1625.794op/s 2016075.627op/s ± 1113.853op/s 2017184.224op/s 2018177.987op/s 2018904.071op/s 2019259.019op/s 0.16% -0.562 0.135 0.08% 114.961op/s 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time 377.772µs 378.804µs ± 0.496µs 378.789µs ± 0.332µs 379.110µs 379.740µs 380.023µs 380.233µs 0.38% 0.369 -0.022 0.13% 0.035µs 1 200
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput 2629966.663op/s 2639889.851op/s ± 3453.844op/s 2639993.095op/s ± 2315.045op/s 2642480.920op/s 2645246.351op/s 2646539.607op/s 2647096.871op/s 0.27% -0.362 -0.031 0.13% 244.224op/s 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time 168.627µs 168.989µs ± 0.123µs 168.995µs ± 0.083µs 169.068µs 169.169µs 169.366µs 169.400µs 0.24% 0.281 0.856 0.07% 0.009µs 1 200
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput 5903177.186op/s 5917561.025op/s ± 4289.674op/s 5917349.490op/s ± 2891.597op/s 5920500.891op/s 5924115.445op/s 5927009.152op/s 5930251.690op/s 0.22% -0.275 0.849 0.07% 303.326op/s 1 200
normalization/normalize_service/normalize_service/[empty string] execution_time 36.857µs 37.057µs ± 0.085µs 37.059µs ± 0.058µs 37.112µs 37.195µs 37.221µs 37.252µs 0.52% -0.123 -0.565 0.23% 0.006µs 1 200
normalization/normalize_service/normalize_service/[empty string] throughput 26844373.202op/s 26985810.212op/s ± 61778.055op/s 26984308.073op/s ± 42502.448op/s 27029614.503op/s 27092942.156op/s 27114218.717op/s 27131804.356op/s 0.55% 0.133 -0.562 0.23% 4368.368op/s 1 200
normalization/normalize_service/normalize_service/test_ASCII execution_time 46.183µs 46.301µs ± 0.112µs 46.289µs ± 0.031µs 46.322µs 46.400µs 46.487µs 47.662µs 2.97% 9.211 109.003 0.24% 0.008µs 1 200
normalization/normalize_service/normalize_service/test_ASCII throughput 20980901.957op/s 21597891.249op/s ± 50948.250op/s 21603457.138op/s ± 14593.809op/s 21617534.445op/s 21637649.910op/s 21643280.736op/s 21652825.888op/s 0.23% -9.023 105.895 0.24% 3602.585op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... execution_time [495.986µs; 496.097µs] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... throughput [2015736.680op/s; 2016187.319op/s] or [-0.011%; +0.011%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて execution_time [378.736µs; 378.873µs] or [-0.018%; +0.018%] None None None
normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて throughput [2639411.181op/s; 2640368.521op/s] or [-0.018%; +0.018%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters execution_time [168.972µs; 169.006µs] or [-0.010%; +0.010%] None None None
normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters throughput [5916966.517op/s; 5918155.532op/s] or [-0.010%; +0.010%] None None None
normalization/normalize_service/normalize_service/[empty string] execution_time [37.045µs; 37.068µs] or [-0.032%; +0.032%] None None None
normalization/normalize_service/normalize_service/[empty string] throughput [26977248.368op/s; 26994372.057op/s] or [-0.032%; +0.032%] None None None
normalization/normalize_service/normalize_service/test_ASCII execution_time [46.286µs; 46.317µs] or [-0.033%; +0.033%] None None None
normalization/normalize_service/normalize_service/test_ASCII throughput [21590830.312op/s; 21604952.187op/s] or [-0.033%; +0.033%] None None None

Group 17

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 624a6d4 1774009052 yannham/fix-semver-check-new-crate
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
benching string interning on wordpress profile execution_time 161.404µs 162.643µs ± 0.357µs 162.608µs ± 0.201µs 162.840µs 163.200µs 163.874µs 164.560µs 1.20% 1.018 5.615 0.22% 0.025µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
benching string interning on wordpress profile execution_time [162.593µs; 162.692µs] or [-0.030%; +0.030%] None None None

Group 18

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 624a6d4 1774009052 yannham/fix-semver-check-new-crate
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_trace/test_trace execution_time 238.661ns 252.251ns ± 15.094ns 244.439ns ± 4.236ns 260.315ns 284.446ns 286.689ns 287.662ns 17.68% 1.094 -0.296 5.97% 1.067ns 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_trace/test_trace execution_time [250.159ns; 254.343ns] or [-0.829%; +0.829%] None None None

Group 19

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 624a6d4 1774009052 yannham/fix-semver-check-new-crate
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time 186.007µs 186.327µs ± 0.165µs 186.303µs ± 0.106µs 186.427µs 186.638µs 186.751µs 186.865µs 0.30% 0.589 -0.049 0.09% 0.012µs 1 200
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput 5351456.218op/s 5366913.262op/s ± 4757.400op/s 5367587.127op/s ± 3052.229op/s 5370334.400op/s 5373594.019op/s 5374919.228op/s 5376132.365op/s 0.16% -0.585 -0.056 0.09% 336.399op/s 1 200
normalization/normalize_name/normalize_name/bad-name execution_time 17.810µs 17.950µs ± 0.049µs 17.957µs ± 0.027µs 17.981µs 18.015µs 18.044µs 18.128µs 0.95% -0.299 0.626 0.27% 0.003µs 1 200
normalization/normalize_name/normalize_name/bad-name throughput 55164587.487op/s 55709353.620op/s ± 150809.675op/s 55687742.449op/s ± 82916.132op/s 55804984.708op/s 55988163.447op/s 56049516.314op/s 56148840.668op/s 0.83% 0.320 0.606 0.27% 10663.854op/s 1 200
normalization/normalize_name/normalize_name/good execution_time 10.334µs 10.467µs ± 0.050µs 10.470µs ± 0.038µs 10.502µs 10.546µs 10.564µs 10.579µs 1.04% -0.165 -0.471 0.48% 0.004µs 1 200
normalization/normalize_name/normalize_name/good throughput 94527975.984op/s 95541391.655op/s ± 459029.509op/s 95508911.078op/s ± 342771.648op/s 95891085.433op/s 96309156.399op/s 96629710.890op/s 96770433.407op/s 1.32% 0.187 -0.458 0.48% 32458.288op/s 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... execution_time [186.304µs; 186.350µs] or [-0.012%; +0.012%] None None None
normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... throughput [5366253.932op/s; 5367572.592op/s] or [-0.012%; +0.012%] None None None
normalization/normalize_name/normalize_name/bad-name execution_time [17.944µs; 17.957µs] or [-0.037%; +0.037%] None None None
normalization/normalize_name/normalize_name/bad-name throughput [55688452.849op/s; 55730254.390op/s] or [-0.038%; +0.038%] None None None
normalization/normalize_name/normalize_name/good execution_time [10.460µs; 10.474µs] or [-0.067%; +0.067%] None None None
normalization/normalize_name/normalize_name/good throughput [95477774.580op/s; 95605008.731op/s] or [-0.067%; +0.067%] None None None

Group 20

cpu_model git_commit_sha git_commit_date git_branch
Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz 624a6d4 1774009052 yannham/fix-semver-check-new-crate
scenario metric min mean ± sd median ± mad p75 p95 p99 max peak_to_median_ratio skewness kurtosis cv sem runs sample_size
tags/replace_trace_tags execution_time 2.311µs 2.344µs ± 0.010µs 2.344µs ± 0.005µs 2.351µs 2.358µs 2.364µs 2.380µs 1.55% -0.697 2.257 0.43% 0.001µs 1 200
scenario metric 95% CI mean Shapiro-Wilk pvalue Ljung-Box pvalue (lag=1) Dip test pvalue
tags/replace_trace_tags execution_time [2.343µs; 2.346µs] or [-0.060%; +0.060%] None None None

Baseline

Omitted due to size.

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Mar 20, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.29%. Comparing base (d88e70e) to head (624a6d4).

Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #1769      +/-   ##
==========================================
+ Coverage   71.28%   71.29%   +0.01%     
==========================================
  Files         431      431              
  Lines       64585    64585              
==========================================
+ Hits        46038    46048      +10     
+ Misses      18547    18537      -10
Components Coverage Δ
libdd-crashtracker 64.95% <ø> (+0.01%) ⬆️
libdd-crashtracker-ffi 34.86% <ø> (ø)
libdd-alloc 98.77% <ø> (ø)
libdd-data-pipeline 87.99% <ø> (ø)
libdd-data-pipeline-ffi 75.20% <ø> (ø)
libdd-common 79.87% <ø> (ø)
libdd-common-ffi 73.87% <ø> (ø)
libdd-telemetry 62.48% <ø> (ø)
libdd-telemetry-ffi 16.75% <ø> (ø)
libdd-dogstatsd-client 82.64% <ø> (ø)
datadog-ipc 80.29% <ø> (ø)
libdd-profiling 81.61% <ø> (ø)
libdd-profiling-ffi 64.94% <ø> (ø)
datadog-sidecar 31.60% <ø> (ø)
datdog-sidecar-ffi 8.71% <ø> (ø)
spawn-worker 54.69% <ø> (ø)
libdd-tinybytes 93.16% <ø> (ø)
libdd-trace-normalization 81.71% <ø> (ø)
libdd-trace-obfuscation 91.80% <ø> (ø)
libdd-trace-protobuf 68.25% <ø> (ø)
libdd-trace-utils 89.05% <ø> (ø)
datadog-tracer-flare 86.88% <ø> (ø)
libdd-log 74.69% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@datadog-official
Copy link
Copy Markdown

datadog-official bot commented Mar 20, 2026 •
edited
Loading

✅ Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 71.30% (+0.02%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 624a6d4 | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback!

@dd-octo-sts
Copy link
Copy Markdown
Contributor

dd-octo-sts bot commented Mar 20, 2026

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 100.37 MB 100.37 MB 0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 8.70 MB 8.70 MB 0% (0 B) 👌
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 11.28 MB 11.28 MB 0% (0 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 117.04 MB 117.04 MB 0% (0 B) 👌
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 27.18 MB 27.18 MB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 77.50 KB 77.50 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 186.11 MB 186.12 MB +0% (+8.00 KB) 👌
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 917.27 MB 917.27 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 9.94 MB 9.94 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 77.50 KB 77.50 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 24.78 MB 24.78 MB 0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 51.47 MB 51.47 MB 0% (0 B) 👌
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 22.97 MB 22.97 MB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 78.71 KB 78.71 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 190.29 MB 190.30 MB +0% (+8.00 KB) 👌
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 900.94 MB 900.94 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 7.54 MB 7.54 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 78.71 KB 78.71 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 26.52 MB 26.52 MB 0% (0 B) 👌
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 47.09 MB 47.09 MB 0% (0 B) 👌
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 87.59 MB 87.59 MB 0% (0 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 10.22 MB 10.22 MB 0% (0 B) 👌
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 109.92 MB 109.92 MB 0% (0 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 11.97 MB 11.97 MB 0% (0 B) 👌

@yannham
Copy link
Copy Markdown
Contributor Author

yannham commented Mar 20, 2026

Tested in https://github.com/DataDog/libdatadog/actions/runs/23344761141/job/67907104730?pr=1624.

@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d bot merged commit 5bdbcc9 into main Mar 20, 2026
86 checks passed
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d bot deleted the yannham/fix-semver-check-new-crate branch March 20, 2026 14:09
bwoebi added a commit that referenced this pull request Mar 20, 2026
@bwoebi
Merge branch 'main' of github.com:DataDog/libdatadog into bob/prevent…
d75b301 
…-unprocessed

* 'main' of github.com:DataDog/libdatadog:
  chore(build): ekump/APMSP-2718 update aws-lc dependencies (#1751)
  chore(crashtracking): add integration test for errno preservation (#1768)
  chore(crashtracking): preserve errno for crashtracker (#1767)
  chore(examples): add compilation flags to prevent logic errors (#1766)
  feat(agents)!: retrieve container tags hash from /info endpoint (#1700)
  fix(ci): handle new crate addition in semver-check (#1769)
  fix(obfuscation/redis): fuzzer fixes [APMSP-2670] (#1694)
  fix(obfuscation/http)!: fuzzer fixes [APMSP-2670] (#1684)
  ci: add Datadog code coverage upload (#1718)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ekump ekump ekump approved these changes

@hoolioh hoolioh hoolioh approved these changes

Assignees

No one assigned

Labels

mergequeue-status: done

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@yannham @codecov-commenter @ekump @hoolioh