chore(ci): run crashtracking ffi example tests in CI#1687
Conversation
Clippy Allow Annotation ReportComparing clippy allow annotations between branches:
Summary by Rule
Annotation Counts by File
Annotation Stats by Crate
About This ReportThis report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality. |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #1687 +/- ##
==========================================
- Coverage 71.39% 71.31% -0.08%
==========================================
Files 427 427
Lines 62897 62954 +57
==========================================
- Hits 44903 44897 -6
- Misses 17994 18057 +63
🚀 New features to boost your workflow:
|
BenchmarksComparisonBenchmark execution time: 2026-03-25 19:32:12 Comparing candidate commit 4b412c1 in PR branch Found 7 performance improvements and 16 performance regressions! Performance is the same for 35 metrics, 1 unstable metrics.
|
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 4b412c1 | 1772834981 | gyuheon0h/enable-crashtracker-ffi-ci |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| normalization/normalize_trace/test_trace | execution_time | 241.352ns | 250.417ns ± 12.170ns | 244.559ns ± 1.607ns | 252.282ns | 280.969ns | 287.671ns | 290.229ns | 18.67% | 1.776 | 1.993 | 4.85% | 0.861ns | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| normalization/normalize_trace/test_trace | execution_time | [248.731ns; 252.104ns] or [-0.674%; +0.674%] | None | None | None |
Group 2
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 4b412c1 | 1772834981 | gyuheon0h/enable-crashtracker-ffi-ci |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| write only interface | execution_time | 1.221µs | 3.198µs ± 1.416µs | 2.982µs ± 0.030µs | 3.008µs | 3.656µs | 13.953µs | 14.661µs | 391.72% | 7.330 | 54.967 | 44.16% | 0.100µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| write only interface | execution_time | [3.002µs; 3.394µs] or [-6.135%; +6.135%] | None | None | None |
Group 3
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 4b412c1 | 1772834981 | gyuheon0h/enable-crashtracker-ffi-ci |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... | execution_time | 534.099µs | 535.572µs ± 1.616µs | 535.096µs ± 0.379µs | 535.557µs | 537.734µs | 544.624µs | 545.201µs | 1.89% | 3.525 | 15.709 | 0.30% | 0.114µs | 1 | 200 |
| normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... | throughput | 1834184.737op/s | 1867178.262op/s ± 5574.970op/s | 1868824.052op/s ± 1325.304op/s | 1870033.240op/s | 1871426.648op/s | 1871963.605op/s | 1872311.650op/s | 0.19% | -3.478 | 15.321 | 0.30% | 394.210op/s | 1 | 200 |
| normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて | execution_time | 380.213µs | 380.847µs ± 0.303µs | 380.828µs ± 0.194µs | 381.030µs | 381.342µs | 381.524µs | 381.767µs | 0.25% | 0.295 | -0.090 | 0.08% | 0.021µs | 1 | 200 |
| normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて | throughput | 2619395.760op/s | 2625729.754op/s ± 2087.964op/s | 2625854.944op/s ± 1337.134op/s | 2627106.554op/s | 2629103.289op/s | 2629760.282op/s | 2630107.411op/s | 0.16% | -0.291 | -0.094 | 0.08% | 147.641op/s | 1 | 200 |
| normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters | execution_time | 189.667µs | 190.131µs ± 0.159µs | 190.140µs ± 0.100µs | 190.232µs | 190.374µs | 190.516µs | 190.614µs | 0.25% | 0.013 | 0.090 | 0.08% | 0.011µs | 1 | 200 |
| normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters | throughput | 5246210.723op/s | 5259538.418op/s ± 4394.987op/s | 5259282.989op/s ± 2777.021op/s | 5262147.009op/s | 5266403.520op/s | 5269840.217op/s | 5272389.914op/s | 0.25% | -0.008 | 0.090 | 0.08% | 310.773op/s | 1 | 200 |
| normalization/normalize_service/normalize_service/[empty string] | execution_time | 37.226µs | 37.465µs ± 0.066µs | 37.463µs ± 0.045µs | 37.511µs | 37.560µs | 37.613µs | 37.651µs | 0.50% | -0.244 | 0.569 | 0.18% | 0.005µs | 1 | 200 |
| normalization/normalize_service/normalize_service/[empty string] | throughput | 26559845.498op/s | 26691958.107op/s ± 46887.937op/s | 26692823.519op/s ± 32061.956op/s | 26720874.092op/s | 26772641.280op/s | 26803107.215op/s | 26863001.093op/s | 0.64% | 0.257 | 0.582 | 0.18% | 3315.478op/s | 1 | 200 |
| normalization/normalize_service/normalize_service/test_ASCII | execution_time | 45.783µs | 45.897µs ± 0.101µs | 45.885µs ± 0.035µs | 45.922µs | 45.979µs | 46.028µs | 47.128µs | 2.71% | 9.222 | 110.062 | 0.22% | 0.007µs | 1 | 200 |
| normalization/normalize_service/normalize_service/test_ASCII | throughput | 21218708.299op/s | 21788153.640op/s ± 46875.851op/s | 21793390.186op/s ± 16448.377op/s | 21808257.099op/s | 21826736.841op/s | 21834851.320op/s | 21842216.199op/s | 0.22% | -9.045 | 107.175 | 0.21% | 3314.623op/s | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... | execution_time | [535.348µs; 535.796µs] or [-0.042%; +0.042%] | None | None | None |
| normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000... | throughput | [1866405.625op/s; 1867950.900op/s] or [-0.041%; +0.041%] | None | None | None |
| normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて | execution_time | [380.805µs; 380.889µs] or [-0.011%; +0.011%] | None | None | None |
| normalization/normalize_service/normalize_service/Data🐨dog🐶 繋がっ⛰てて | throughput | [2625440.382op/s; 2626019.125op/s] or [-0.011%; +0.011%] | None | None | None |
| normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters | execution_time | [190.109µs; 190.153µs] or [-0.012%; +0.012%] | None | None | None |
| normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters | throughput | [5258929.315op/s; 5260147.521op/s] or [-0.012%; +0.012%] | None | None | None |
| normalization/normalize_service/normalize_service/[empty string] | execution_time | [37.455µs; 37.474µs] or [-0.024%; +0.024%] | None | None | None |
| normalization/normalize_service/normalize_service/[empty string] | throughput | [26685459.890op/s; 26698456.324op/s] or [-0.024%; +0.024%] | None | None | None |
| normalization/normalize_service/normalize_service/test_ASCII | execution_time | [45.883µs; 45.911µs] or [-0.030%; +0.030%] | None | None | None |
| normalization/normalize_service/normalize_service/test_ASCII | throughput | [21781657.097op/s; 21794650.182op/s] or [-0.030%; +0.030%] | None | None | None |
Group 4
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 4b412c1 | 1772834981 | gyuheon0h/enable-crashtracker-ffi-ci |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| concentrator/add_spans_to_concentrator | execution_time | 10.613ms | 10.646ms ± 0.016ms | 10.645ms ± 0.008ms | 10.652ms | 10.669ms | 10.701ms | 10.739ms | 0.88% | 1.589 | 6.776 | 0.15% | 0.001ms | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| concentrator/add_spans_to_concentrator | execution_time | [10.644ms; 10.648ms] or [-0.021%; +0.021%] | None | None | None |
Group 5
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 4b412c1 | 1772834981 | gyuheon0h/enable-crashtracker-ffi-ci |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| two way interface | execution_time | 17.895µs | 25.567µs ± 9.430µs | 18.044µs ± 0.114µs | 33.981µs | 43.280µs | 44.442µs | 65.469µs | 262.83% | 0.907 | 0.305 | 36.79% | 0.667µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| two way interface | execution_time | [24.261µs; 26.874µs] or [-5.111%; +5.111%] | None | None | None |
Group 6
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 4b412c1 | 1772834981 | gyuheon0h/enable-crashtracker-ffi-ci |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ip_address/quantize_peer_ip_address_benchmark | execution_time | 4.945µs | 5.027µs ± 0.060µs | 4.996µs ± 0.015µs | 5.061µs | 5.133µs | 5.136µs | 5.139µs | 2.87% | 0.860 | -0.902 | 1.19% | 0.004µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| ip_address/quantize_peer_ip_address_benchmark | execution_time | [5.018µs; 5.035µs] or [-0.166%; +0.166%] | None | None | None |
Group 7
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 4b412c1 | 1772834981 | gyuheon0h/enable-crashtracker-ffi-ci |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| tags/replace_trace_tags | execution_time | 2.384µs | 2.437µs ± 0.021µs | 2.437µs ± 0.014µs | 2.451µs | 2.479µs | 2.484µs | 2.496µs | 2.45% | 0.203 | 0.079 | 0.88% | 0.002µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| tags/replace_trace_tags | execution_time | [2.434µs; 2.440µs] or [-0.122%; +0.122%] | None | None | None |
Group 8
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 4b412c1 | 1772834981 | gyuheon0h/enable-crashtracker-ffi-ci |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| profile_add_sample_frames_x1000 | execution_time | 4.218ms | 4.223ms ± 0.008ms | 4.221ms ± 0.001ms | 4.223ms | 4.229ms | 4.235ms | 4.324ms | 2.42% | 11.162 | 141.539 | 0.18% | 0.001ms | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| profile_add_sample_frames_x1000 | execution_time | [4.222ms; 4.224ms] or [-0.025%; +0.025%] | None | None | None |
Group 9
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 4b412c1 | 1772834981 | gyuheon0h/enable-crashtracker-ffi-ci |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| profile_add_sample2_frames_x1000 | execution_time | 750.705µs | 751.647µs ± 0.550µs | 751.555µs ± 0.284µs | 751.870µs | 752.585µs | 753.191µs | 755.214µs | 0.49% | 1.946 | 8.453 | 0.07% | 0.039µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| profile_add_sample2_frames_x1000 | execution_time | [751.571µs; 751.723µs] or [-0.010%; +0.010%] | None | None | None |
Group 10
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 4b412c1 | 1772834981 | gyuheon0h/enable-crashtracker-ffi-ci |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... | execution_time | 185.310µs | 185.695µs ± 0.208µs | 185.672µs ± 0.150µs | 185.849µs | 186.061µs | 186.182µs | 186.225µs | 0.30% | 0.415 | -0.538 | 0.11% | 0.015µs | 1 | 200 |
| normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... | throughput | 5369839.651op/s | 5385184.179op/s ± 6043.529op/s | 5385832.635op/s ± 4340.164op/s | 5389792.125op/s | 5394091.881op/s | 5395346.824op/s | 5396357.027op/s | 0.20% | -0.411 | -0.543 | 0.11% | 427.342op/s | 1 | 200 |
| normalization/normalize_name/normalize_name/bad-name | execution_time | 17.888µs | 17.986µs ± 0.120µs | 17.977µs ± 0.023µs | 17.998µs | 18.045µs | 18.075µs | 19.592µs | 8.98% | 12.194 | 161.393 | 0.66% | 0.008µs | 1 | 200 |
| normalization/normalize_name/normalize_name/bad-name | throughput | 51041854.321op/s | 55602020.223op/s ± 342319.266op/s | 55625840.987op/s ± 70820.844op/s | 55699471.863op/s | 55797807.128op/s | 55874368.730op/s | 55904580.746op/s | 0.50% | -11.894 | 156.081 | 0.61% | 24205.627op/s | 1 | 200 |
| normalization/normalize_name/normalize_name/good | execution_time | 10.344µs | 10.465µs ± 0.048µs | 10.462µs ± 0.036µs | 10.500µs | 10.547µs | 10.571µs | 10.589µs | 1.21% | 0.149 | -0.471 | 0.46% | 0.003µs | 1 | 200 |
| normalization/normalize_name/normalize_name/good | throughput | 94438434.726op/s | 95554429.954op/s ± 439781.271op/s | 95580108.139op/s ± 334196.843op/s | 95906134.061op/s | 96243166.250op/s | 96424238.409op/s | 96671073.012op/s | 1.14% | -0.128 | -0.474 | 0.46% | 31097.232op/s | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... | execution_time | [185.666µs; 185.724µs] or [-0.016%; +0.016%] | None | None | None |
| normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo... | throughput | [5384346.604op/s; 5386021.754op/s] or [-0.016%; +0.016%] | None | None | None |
| normalization/normalize_name/normalize_name/bad-name | execution_time | [17.969µs; 18.002µs] or [-0.092%; +0.092%] | None | None | None |
| normalization/normalize_name/normalize_name/bad-name | throughput | [55554578.065op/s; 55649462.381op/s] or [-0.085%; +0.085%] | None | None | None |
| normalization/normalize_name/normalize_name/good | execution_time | [10.459µs; 10.472µs] or [-0.064%; +0.064%] | None | None | None |
| normalization/normalize_name/normalize_name/good | throughput | [95493480.499op/s; 95615379.408op/s] or [-0.064%; +0.064%] | None | None | None |
Group 11
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 4b412c1 | 1772834981 | gyuheon0h/enable-crashtracker-ffi-ci |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| redis/obfuscate_redis_string | execution_time | 32.811µs | 33.750µs ± 1.270µs | 32.948µs ± 0.054µs | 35.410µs | 35.896µs | 35.939µs | 36.081µs | 9.51% | 0.938 | -1.058 | 3.75% | 0.090µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| redis/obfuscate_redis_string | execution_time | [33.574µs; 33.926µs] or [-0.522%; +0.522%] | None | None | None |
Group 12
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 4b412c1 | 1772834981 | gyuheon0h/enable-crashtracker-ffi-ci |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| receiver_entry_point/report/2597 | execution_time | 3.144ms | 3.172ms ± 0.020ms | 3.165ms ± 0.010ms | 3.180ms | 3.209ms | 3.230ms | 3.265ms | 3.16% | 1.321 | 2.136 | 0.63% | 0.001ms | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| receiver_entry_point/report/2597 | execution_time | [3.169ms; 3.174ms] or [-0.088%; +0.088%] | None | None | None |
Group 13
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 4b412c1 | 1772834981 | gyuheon0h/enable-crashtracker-ffi-ci |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| sql/obfuscate_sql_string | execution_time | 85.691µs | 86.026µs ± 0.200µs | 86.015µs ± 0.058µs | 86.073µs | 86.143µs | 86.306µs | 88.433µs | 2.81% | 8.983 | 104.681 | 0.23% | 0.014µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| sql/obfuscate_sql_string | execution_time | [85.998µs; 86.053µs] or [-0.032%; +0.032%] | None | None | None |
Group 14
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 4b412c1 | 1772834981 | gyuheon0h/enable-crashtracker-ffi-ci |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| benching string interning on wordpress profile | execution_time | 160.072µs | 160.798µs ± 0.412µs | 160.721µs ± 0.143µs | 160.873µs | 161.276µs | 162.429µs | 164.014µs | 2.05% | 4.547 | 29.828 | 0.26% | 0.029µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| benching string interning on wordpress profile | execution_time | [160.741µs; 160.855µs] or [-0.035%; +0.035%] | None | None | None |
Group 15
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 4b412c1 | 1772834981 | gyuheon0h/enable-crashtracker-ffi-ci |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| benching deserializing traces from msgpack to their internal representation | execution_time | 48.427ms | 48.726ms ± 0.919ms | 48.579ms ± 0.059ms | 48.660ms | 48.820ms | 55.414ms | 56.489ms | 16.28% | 7.485 | 56.226 | 1.88% | 0.065ms | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| benching deserializing traces from msgpack to their internal representation | execution_time | [48.599ms; 48.854ms] or [-0.261%; +0.261%] | None | None | None |
Group 16
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 4b412c1 | 1772834981 | gyuheon0h/enable-crashtracker-ffi-ci |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| benching serializing traces from their internal representation to msgpack | execution_time | 13.933ms | 13.992ms ± 0.033ms | 13.988ms ± 0.016ms | 14.008ms | 14.044ms | 14.118ms | 14.176ms | 1.34% | 1.842 | 6.138 | 0.24% | 0.002ms | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| benching serializing traces from their internal representation to msgpack | execution_time | [13.988ms; 13.997ms] or [-0.033%; +0.033%] | None | None | None |
Group 17
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 4b412c1 | 1772834981 | gyuheon0h/enable-crashtracker-ffi-ci |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| credit_card/is_card_number/ | execution_time | 3.895µs | 3.913µs ± 0.007µs | 3.912µs ± 0.002µs | 3.914µs | 3.917µs | 3.922µs | 3.977µs | 1.67% | 7.624 | 68.782 | 0.17% | 0.000µs | 1 | 200 |
| credit_card/is_card_number/ | throughput | 251420338.017op/s | 255544656.328op/s ± 425210.692op/s | 255608128.343op/s ± 105274.910op/s | 255685072.818op/s | 255785260.154op/s | 255867559.976op/s | 256711689.973op/s | 0.43% | -7.560 | 68.051 | 0.17% | 30066.936op/s | 1 | 200 |
| credit_card/is_card_number/ 3782-8224-6310-005 | execution_time | 78.980µs | 79.787µs ± 0.418µs | 79.780µs ± 0.326µs | 80.097µs | 80.590µs | 80.759µs | 81.226µs | 1.81% | 0.474 | -0.119 | 0.52% | 0.030µs | 1 | 200 |
| credit_card/is_card_number/ 3782-8224-6310-005 | throughput | 12311391.437op/s | 12533706.778op/s ± 65581.160op/s | 12534458.629op/s ± 51258.644op/s | 12586517.009op/s | 12621011.045op/s | 12658429.297op/s | 12661447.270op/s | 1.01% | -0.449 | -0.167 | 0.52% | 4637.288op/s | 1 | 200 |
| credit_card/is_card_number/ 378282246310005 | execution_time | 72.281µs | 73.006µs ± 0.356µs | 72.982µs ± 0.230µs | 73.196µs | 73.666µs | 73.883µs | 74.257µs | 1.75% | 0.597 | 0.153 | 0.49% | 0.025µs | 1 | 200 |
| credit_card/is_card_number/ 378282246310005 | throughput | 13466829.297op/s | 13697810.645op/s ± 66581.482op/s | 13701974.721op/s ± 42993.444op/s | 13756134.096op/s | 13791102.744op/s | 13809788.829op/s | 13834831.956op/s | 0.97% | -0.571 | 0.101 | 0.48% | 4708.022op/s | 1 | 200 |
| credit_card/is_card_number/37828224631 | execution_time | 3.896µs | 3.913µs ± 0.003µs | 3.913µs ± 0.002µs | 3.915µs | 3.919µs | 3.920µs | 3.922µs | 0.23% | -0.458 | 4.985 | 0.08% | 0.000µs | 1 | 200 |
| credit_card/is_card_number/37828224631 | throughput | 254957501.428op/s | 255532785.888op/s ± 195905.673op/s | 255556401.330op/s ± 122660.842op/s | 255671133.474op/s | 255777525.022op/s | 255844481.045op/s | 256678823.163op/s | 0.44% | 0.474 | 5.067 | 0.08% | 13852.623op/s | 1 | 200 |
| credit_card/is_card_number/378282246310005 | execution_time | 69.203µs | 69.783µs ± 0.365µs | 69.719µs ± 0.260µs | 70.027µs | 70.417µs | 70.692µs | 70.915µs | 1.72% | 0.681 | -0.243 | 0.52% | 0.026µs | 1 | 200 |
| credit_card/is_card_number/378282246310005 | throughput | 14101377.940op/s | 14330507.014op/s ± 74621.304op/s | 14343233.785op/s ± 53411.580op/s | 14391385.677op/s | 14425335.445op/s | 14439095.122op/s | 14450260.897op/s | 0.75% | -0.661 | -0.284 | 0.52% | 5276.523op/s | 1 | 200 |
| credit_card/is_card_number/37828224631000521389798 | execution_time | 52.160µs | 52.228µs ± 0.037µs | 52.226µs ± 0.026µs | 52.250µs | 52.289µs | 52.334µs | 52.371µs | 0.28% | 0.736 | 0.824 | 0.07% | 0.003µs | 1 | 200 |
| credit_card/is_card_number/37828224631000521389798 | throughput | 19094373.743op/s | 19146654.043op/s ± 13400.850op/s | 19147452.689op/s ± 9495.975op/s | 19157418.327op/s | 19165396.302op/s | 19168034.275op/s | 19171756.169op/s | 0.13% | -0.732 | 0.810 | 0.07% | 947.583op/s | 1 | 200 |
| credit_card/is_card_number/x371413321323331 | execution_time | 6.030µs | 6.038µs ± 0.011µs | 6.036µs ± 0.003µs | 6.039µs | 6.071µs | 6.081µs | 6.118µs | 1.37% | 4.008 | 18.753 | 0.18% | 0.001µs | 1 | 200 |
| credit_card/is_card_number/x371413321323331 | throughput | 163452431.002op/s | 165612006.438op/s ± 297741.497op/s | 165683829.263op/s ± 70160.519op/s | 165743457.936op/s | 165798369.003op/s | 165820139.023op/s | 165846625.001op/s | 0.10% | -3.983 | 18.472 | 0.18% | 21053.503op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/ | execution_time | 3.897µs | 3.913µs ± 0.003µs | 3.913µs ± 0.002µs | 3.914µs | 3.917µs | 3.920µs | 3.928µs | 0.39% | 0.565 | 7.600 | 0.08% | 0.000µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/ | throughput | 254575765.323op/s | 255564755.224op/s ± 199492.097op/s | 255573254.259op/s ± 101605.583op/s | 255670648.192op/s | 255816528.228op/s | 255916618.382op/s | 256610789.901op/s | 0.41% | -0.543 | 7.595 | 0.08% | 14106.221op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 | execution_time | 64.164µs | 64.488µs ± 0.141µs | 64.474µs ± 0.091µs | 64.580µs | 64.727µs | 64.817µs | 64.912µs | 0.68% | 0.353 | -0.152 | 0.22% | 0.010µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 | throughput | 15405447.682op/s | 15506883.190op/s ± 33944.390op/s | 15510114.236op/s ± 21919.912op/s | 15529712.917op/s | 15556409.549op/s | 15572496.193op/s | 15584949.614op/s | 0.48% | -0.341 | -0.164 | 0.22% | 2400.231op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/ 378282246310005 | execution_time | 58.107µs | 58.380µs ± 0.155µs | 58.347µs ± 0.080µs | 58.454µs | 58.680µs | 58.887µs | 59.076µs | 1.25% | 1.392 | 2.802 | 0.26% | 0.011µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/ 378282246310005 | throughput | 16927387.805op/s | 17129239.943op/s ± 45319.170op/s | 17138820.415op/s ± 23462.399op/s | 17160114.317op/s | 17186178.121op/s | 17195150.435op/s | 17209549.579op/s | 0.41% | -1.370 | 2.708 | 0.26% | 3204.549op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/37828224631 | execution_time | 3.892µs | 3.913µs ± 0.003µs | 3.913µs ± 0.002µs | 3.914µs | 3.917µs | 3.917µs | 3.919µs | 0.16% | -2.290 | 17.511 | 0.07% | 0.000µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/37828224631 | throughput | 255165529.653op/s | 255570747.944op/s ± 171658.120op/s | 255574276.453op/s ± 100162.292op/s | 255670967.870op/s | 255782993.570op/s | 255891450.992op/s | 256923283.371op/s | 0.53% | 2.319 | 17.784 | 0.07% | 12138.062op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/378282246310005 | execution_time | 54.612µs | 55.114µs ± 0.314µs | 55.093µs ± 0.215µs | 55.300µs | 55.652µs | 55.958µs | 56.430µs | 2.43% | 0.672 | 0.802 | 0.57% | 0.022µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/378282246310005 | throughput | 17720970.276op/s | 18144930.133op/s ± 103027.469op/s | 18151068.465op/s ± 70950.121op/s | 18222706.416op/s | 18294054.587op/s | 18301152.417op/s | 18310883.385op/s | 0.88% | -0.632 | 0.675 | 0.57% | 7285.142op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/37828224631000521389798 | execution_time | 52.156µs | 52.219µs ± 0.036µs | 52.214µs ± 0.020µs | 52.235µs | 52.280µs | 52.312µs | 52.434µs | 0.42% | 1.625 | 5.968 | 0.07% | 0.003µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/37828224631000521389798 | throughput | 19071697.065op/s | 19149966.941op/s ± 13207.929op/s | 19151832.185op/s ± 7228.824op/s | 19158297.658op/s | 19166296.260op/s | 19170426.493op/s | 19173343.301op/s | 0.11% | -1.614 | 5.894 | 0.07% | 933.942op/s | 1 | 200 |
| credit_card/is_card_number_no_luhn/x371413321323331 | execution_time | 6.030µs | 6.038µs ± 0.009µs | 6.036µs ± 0.003µs | 6.039µs | 6.046µs | 6.077µs | 6.118µs | 1.36% | 5.189 | 35.952 | 0.15% | 0.001µs | 1 | 200 |
| credit_card/is_card_number_no_luhn/x371413321323331 | throughput | 163443864.879op/s | 165622673.318op/s ± 245319.128op/s | 165672978.948op/s ± 74096.731op/s | 165739764.200op/s | 165794623.822op/s | 165837979.657op/s | 165845501.095op/s | 0.10% | -5.140 | 35.315 | 0.15% | 17346.682op/s | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| credit_card/is_card_number/ | execution_time | [3.912µs; 3.914µs] or [-0.023%; +0.023%] | None | None | None |
| credit_card/is_card_number/ | throughput | [255485726.216op/s; 255603586.440op/s] or [-0.023%; +0.023%] | None | None | None |
| credit_card/is_card_number/ 3782-8224-6310-005 | execution_time | [79.729µs; 79.845µs] or [-0.073%; +0.073%] | None | None | None |
| credit_card/is_card_number/ 3782-8224-6310-005 | throughput | [12524617.860op/s; 12542795.696op/s] or [-0.073%; +0.073%] | None | None | None |
| credit_card/is_card_number/ 378282246310005 | execution_time | [72.957µs; 73.055µs] or [-0.068%; +0.068%] | None | None | None |
| credit_card/is_card_number/ 378282246310005 | throughput | [13688583.092op/s; 13707038.199op/s] or [-0.067%; +0.067%] | None | None | None |
| credit_card/is_card_number/37828224631 | execution_time | [3.913µs; 3.914µs] or [-0.011%; +0.011%] | None | None | None |
| credit_card/is_card_number/37828224631 | throughput | [255505635.246op/s; 255559936.530op/s] or [-0.011%; +0.011%] | None | None | None |
| credit_card/is_card_number/378282246310005 | execution_time | [69.733µs; 69.834µs] or [-0.072%; +0.072%] | None | None | None |
| credit_card/is_card_number/378282246310005 | throughput | [14320165.219op/s; 14340848.809op/s] or [-0.072%; +0.072%] | None | None | None |
| credit_card/is_card_number/37828224631000521389798 | execution_time | [52.223µs; 52.234µs] or [-0.010%; +0.010%] | None | None | None |
| credit_card/is_card_number/37828224631000521389798 | throughput | [19144796.814op/s; 19148511.272op/s] or [-0.010%; +0.010%] | None | None | None |
| credit_card/is_card_number/x371413321323331 | execution_time | [6.037µs; 6.040µs] or [-0.025%; +0.025%] | None | None | None |
| credit_card/is_card_number/x371413321323331 | throughput | [165570742.330op/s; 165653270.546op/s] or [-0.025%; +0.025%] | None | None | None |
| credit_card/is_card_number_no_luhn/ | execution_time | [3.912µs; 3.913µs] or [-0.011%; +0.011%] | None | None | None |
| credit_card/is_card_number_no_luhn/ | throughput | [255537107.538op/s; 255592402.910op/s] or [-0.011%; +0.011%] | None | None | None |
| credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 | execution_time | [64.468µs; 64.507µs] or [-0.030%; +0.030%] | None | None | None |
| credit_card/is_card_number_no_luhn/ 3782-8224-6310-005 | throughput | [15502178.824op/s; 15511587.556op/s] or [-0.030%; +0.030%] | None | None | None |
| credit_card/is_card_number_no_luhn/ 378282246310005 | execution_time | [58.359µs; 58.402µs] or [-0.037%; +0.037%] | None | None | None |
| credit_card/is_card_number_no_luhn/ 378282246310005 | throughput | [17122959.142op/s; 17135520.745op/s] or [-0.037%; +0.037%] | None | None | None |
| credit_card/is_card_number_no_luhn/37828224631 | execution_time | [3.912µs; 3.913µs] or [-0.009%; +0.009%] | None | None | None |
| credit_card/is_card_number_no_luhn/37828224631 | throughput | [255546957.779op/s; 255594538.108op/s] or [-0.009%; +0.009%] | None | None | None |
| credit_card/is_card_number_no_luhn/378282246310005 | execution_time | [55.070µs; 55.157µs] or [-0.079%; +0.079%] | None | None | None |
| credit_card/is_card_number_no_luhn/378282246310005 | throughput | [18130651.517op/s; 18159208.749op/s] or [-0.079%; +0.079%] | None | None | None |
| credit_card/is_card_number_no_luhn/37828224631000521389798 | execution_time | [52.214µs; 52.224µs] or [-0.010%; +0.010%] | None | None | None |
| credit_card/is_card_number_no_luhn/37828224631000521389798 | throughput | [19148136.449op/s; 19151797.432op/s] or [-0.010%; +0.010%] | None | None | None |
| credit_card/is_card_number_no_luhn/x371413321323331 | execution_time | [6.037µs; 6.039µs] or [-0.021%; +0.021%] | None | None | None |
| credit_card/is_card_number_no_luhn/x371413321323331 | throughput | [165588674.446op/s; 165656672.189op/s] or [-0.021%; +0.021%] | None | None | None |
Group 18
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 4b412c1 | 1772834981 | gyuheon0h/enable-crashtracker-ffi-ci |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| sdk_test_data/rules-based | execution_time | 144.537µs | 146.399µs ± 1.919µs | 146.076µs ± 0.472µs | 146.591µs | 147.867µs | 152.072µs | 166.740µs | 14.15% | 7.094 | 65.851 | 1.31% | 0.136µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| sdk_test_data/rules-based | execution_time | [146.133µs; 146.665µs] or [-0.182%; +0.182%] | None | None | None |
Group 19
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz | 4b412c1 | 1772834981 | gyuheon0h/enable-crashtracker-ffi-ci |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| single_flag_killswitch/rules-based | execution_time | 188.246ns | 190.488ns ± 1.854ns | 190.287ns ± 1.388ns | 191.583ns | 194.428ns | 195.863ns | 196.010ns | 3.01% | 0.908 | 0.196 | 0.97% | 0.131ns | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| single_flag_killswitch/rules-based | execution_time | [190.231ns; 190.745ns] or [-0.135%; +0.135%] | None | None | None |
Baseline
Omitted due to size.
fe3215f to
23fe2b7
Compare
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
23fe2b7 to
4b412c1
Compare
# What does this PR do? Bump to 29.0.0 [feat(profiling)!: add Tracepoint sample type](#1676) [feat(obfuscation/redis): Reach feature parity on redis obfuscation [APMSP-2668]](#1632) [fix(sidecar): Handle backpressure more gracefully](#1682) [feat(trace-protobuf)!: Add two fields to ClientGroupedStats [SVLS-8627]](#1630) [chore: exclude libdatadog from ADMS auto generated PRs for dependency updates](#1688) [chore(ci): run crashtracking ffi example tests in CI](#1687) [fix(crashtracking): use libunwind to unwind frames](#1663) [feat: publish tracer metadata as OTel process ctx](#1658) [ci: run thread count test in own process](#1693) [feat(obfuscation/json): Init json obfuscation [APMSP-2665]](#1635) [chore(ci): add final_status property on junit XML [APMSP-2610]](#1681) [refactor(data-pipeline-ffi): move macro definitions for better reuse](#1699) [fix(obfuscation/memcached): fuzzing fix](#1695) [ci: replace use of cargo cross for centos7 tests](#1675) #1702 (comment) [feat(stats_exporter)!: add process tags to CSS payloads](#1709) # Motivation I mainly am creating this to use the new unwinding from ucontext for crashtracking # Additional Notes Anything else we should know when reviewing? # How to test the change? Describe here in detail how the change can be validated. [APMSP-2668]: https://datadoghq.atlassian.net/browse/APMSP-2668?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [SVLS-8627]: https://datadoghq.atlassian.net/browse/SVLS-8627?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [APMSP-2665]: https://datadoghq.atlassian.net/browse/APMSP-2665?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ Co-authored-by: gyuheon.oh <[email protected]>

What does this PR do?
Previously, we couldn't run crashtracking FFI examples in CI because the C example had paths like /tmp/libdatadog/bin/libdatadog-crashtracking-receiver and /tmp/crashreports/ baked in, which is variable in CI. Also, the test runner had no way to handle intentional crashes.
This PR
examples/ffi/crashtracking.ctools/src/bin/ffi_test.rsMotivation
Although FFI examples are not meant to increase test coverage, running the crashtracking example in CI is a QOL improvement that makes sure our public FFI surface is buildable and usable in real-world scenarios.
How to test the change?
crashtracking FFI tests now run in CI
