ci(benchmarks): Parellelize microbenchmarks, deprecate dependency on benchmarking-platform repo by ddyurchenko · Pull Request #4718 · DataDog/dd-trace-go

ddyurchenko · 2026-05-04T15:24:59Z

What does this PR do?

It restructures the benchmarks CI so that everything needed to run microbenchmarks lives inside dd-trace-go instead of being driven by an external clone of the benchmarking-platform repo, and parallelizes the microbenchmark run across multiple CI jobs.

Key changes

Self-contained benchmark configuration under .gitlab/benchmarks/
- New micro/ suite owns its own gitlab-ci.yml, bp-runner.yml, and container/ (Dockerfile + install scripts) instead of fetching a job spec from the external benchmarking-platform repo.
- macro/ and test-apps/ are split into their own subdirectories alongside micro/.
- Adds top-level pr-gate.thresholds.yml (PR regression gates) and renames the release SLOs file to pre-release-gate.slos.yml.
Parallelized microbenchmarks
- Microbenchmarks are split into multiple microbenchmarks-N jobs (groups of benchmarks), each running on a dedicated 48-core bare-metal runner (runner:apm-k8s-m8g-metal).
- bp-runner.yml parallelizes scenarios across CPU cores 4–47 (cores 0–3 reserved for kernel/background), with CPUS_PER_BENCHMARK=1 and REPETITIONS=10.
- A pr-performance-gates job aggregates results and fails the pipeline on regressions above the configured threshold.
- Adds FLAKY_BENCHMARKS_REGEX to mark known-flaky benchmarks (still run/reported, but don't fail the gate).
Deprecation of benchmarking-platform repo coupling
- Removes .gitlab/bp-runner.yml, .gitlab/child_pipeline.yml.tpl, and .gitlab/generate_config.go (the dynamic child-pipeline generator that pulled jobs from the external repo).
- Trims ~114 lines from the root .gitlab-ci.yml and .gitlab/benchmarks.yml; benchmark suites are now included directly from this repo.
- CI image is built from .gitlab/benchmarks/micro/container/ (Go base + bp-runner/bp-analyzer/bp-github via bp-install) and pinned via BENCHMARKS_CI_IMAGE.
Documentation
- New .gitlab/benchmarks/README.md explains how the micro/macro/test-apps suites fit together and how go-go-prof-app-parallel (from apm-sdks-benchmarks) plugs in.
- New .gitlab/benchmarks/micro/README.md documents how to add a benchmark, mark one as flaky, rebuild the CI Docker image, and run the pipeline locally with bp-runner.
- Adds .env.example and .gitignore for local runs.
Misc
- Updates CODEOWNERS for the new paths.

Motivation

Removes orchestration coupling to the external benchmarking-platform repo, making benchmark configuration reviewable and versioned alongside the code it benchmarks;
Cuts wall-clock time for the PR performance gate by sharding benchmarks across cores and across multiple parallel jobs;
Reduce number of metal machines needed for runs, preventing capacity errors;
Provides a documented, locally-runnable workflow (bp-runner bp-runner.yml --local) for iterating on microbenchmarks CI image before opening a PR.

Reviewer's Checklist

Changed code has unit tests for its functionality at or near 100% coverage.
System-Tests covering this feature have been added and enabled with the va.b.c-dev version tag.
There is a benchmark for any new code, or changes to existing code.
If this interacts with the agent in a new way, a system test has been added.
New code is free of linting errors. You can check this by running make lint locally.
New code doesn't break existing tests. You can check this by running make test locally.
Add an appropriate team label so this PR gets put in the right place for the release notes.
All generated files are up to date. You can check this by running make generate locally.
Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild. Make sure all nested modules are up to date by running make fix-modules locally.

Unsure? Have a question? Request a review!

datadog-prod-us1-5 · 2026-05-05T13:53:49Z

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 61.64% (+4.29%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 7f770e3 | Docs | Datadog PR Page | Give us feedback!}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 065670181e

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

codecov · 2026-05-05T15:06:58Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 62.33%. Comparing base (42d74ee) to head (7f770e3).

Additional details and impacted files

see 451 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

pr-commenter · 2026-05-06T13:15:38Z

Benchmarks

Benchmark execution time: 2026-05-07 15:09:19

Comparing candidate commit 7f770e3 in PR branch ddyurchenko/deprecate-bp-repo with baseline commit 42d74ee in branch main.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 273 metrics, 2 unstable metrics, 1 flaky benchmarks without significant changes.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

🟩 = significantly better candidate vs. baseline
🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Known flaky benchmarks

These benchmarks are marked as flaky and will not trigger a failure. Modify FLAKY_BENCHMARKS_REGEX to control which benchmarks are marked as flaky.

Known flaky benchmarks without significant changes:

scenario:BenchmarkOTLPTraceWriterFlush

ddyurchenko · 2026-05-06T15:42:38Z

Alright! After doing flaky benchmarks audit, I noticed 1 benchmark that was flaky before the change and flaky afterwards

BenchmarkOTLPTraceWriterFlush

(from time to time it reports 5-6% difference)

I marked it as known flaky benchmark.

Rest of benchmarks have less than 1% difference on reruns.

Stability Analysis Report

Total pipelines: 7

Pipelines: 111739801, 111739814, 111739822, 111739826, 111739839, 111739843, 111739890

CPU: Neoverse-V2

Stability Results

Significant impact threshold: 2.0%, t-test alpha: 0.05

Scenario	Metric	Significant impact FP	t-test FP	Max bound
BenchmarkOTLPTraceWriterFlush	execution_time	14/21 (66.7%)	19/21 (90.5%)	9.77%
BenchmarkCaptureStackTrace/10	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkCaptureStackTrace/100	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkCaptureStackTrace/20	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkCaptureStackTrace/200	execution_time	0/21 (0.0%)	5/21 (23.8%)	0.32%
BenchmarkCaptureStackTrace/50	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkConfig/scenario_WithStartSpanConfig	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkConfig/scenario_none	execution_time	0/21 (0.0%)	1/21 (4.8%)	0.00%
BenchmarkExtractW3C	execution_time	0/21 (0.0%)	2/21 (9.5%)	0.02%
BenchmarkGraphQL/version_baseline/scenario_basic	allocated_mem	0/21 (0.0%)	3/21 (14.3%)	0.00%
BenchmarkGraphQL/version_baseline/scenario_basic	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkGraphQL/version_baseline/scenario_embedded-variable	allocated_mem	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkGraphQL/version_baseline/scenario_embedded-variable	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkGraphQL/version_baseline/scenario_with-default-parameter	allocated_mem	0/21 (0.0%)	6/21 (28.6%)	0.00%
BenchmarkGraphQL/version_baseline/scenario_with-default-parameter	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkGraphQL/version_dyngo/scenario_basic	allocated_mem	0/21 (0.0%)	4/21 (19.0%)	0.00%
BenchmarkGraphQL/version_dyngo/scenario_basic	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkGraphQL/version_dyngo/scenario_embedded-variable	allocated_mem	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkGraphQL/version_dyngo/scenario_embedded-variable	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkGraphQL/version_dyngo/scenario_with-default-parameter	allocated_mem	0/21 (0.0%)	6/21 (28.6%)	0.00%
BenchmarkGraphQL/version_dyngo/scenario_with-default-parameter	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkHttpServeTrace	allocated_mem	0/21 (0.0%)	3/21 (14.3%)	0.05%
BenchmarkHttpServeTrace	execution_time	0/21 (0.0%)	2/21 (9.5%)	0.14%
BenchmarkInjectW3C	execution_time	0/21 (0.0%)	3/21 (14.3%)	0.02%
BenchmarkLogs/simple	execution_time	0/21 (0.0%)	7/21 (33.3%)	0.17%
BenchmarkLogs/with-stacktrace	execution_time	0/21 (0.0%)	10/21 (47.6%)	0.45%
BenchmarkLogs/with-tags	execution_time	0/21 (0.0%)	5/21 (23.8%)	0.31%
BenchmarkMetrics/count/get-handle	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkMetrics/count/handle-reused	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkMetrics/distribution/get-handle	execution_time	0/21 (0.0%)	4/21 (19.0%)	0.02%
BenchmarkMetrics/distribution/handle-reused	execution_time	0/21 (0.0%)	1/21 (4.8%)	0.00%
BenchmarkMetrics/gauge/get-handle	execution_time	0/21 (0.0%)	3/21 (14.3%)	0.14%
BenchmarkMetrics/gauge/handle-reused	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkMetrics/rate/get-handle	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkMetrics/rate/handle-reused	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkOTLPProtoMarshal/1000spans	execution_time	0/21 (0.0%)	3/21 (14.3%)	0.08%
BenchmarkOTLPProtoMarshal/100spans	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkOTLPProtoMarshal/10spans	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkOTLPProtoMarshal/1span	execution_time	0/21 (0.0%)	13/21 (61.9%)	0.75%
BenchmarkOTLPProtoMarshal/rich_spans	execution_time	0/21 (0.0%)	2/21 (9.5%)	0.04%
BenchmarkOTLPProtoSize/1000spans	execution_time	0/21 (0.0%)	3/21 (14.3%)	0.03%
BenchmarkOTLPProtoSize/100spans	execution_time	0/21 (0.0%)	9/21 (42.9%)	0.12%
BenchmarkOTLPProtoSize/10spans	execution_time	0/21 (0.0%)	16/21 (76.2%)	0.36%
BenchmarkOTLPProtoSize/1span	execution_time	0/21 (0.0%)	5/21 (23.8%)	1.09%
BenchmarkOTLPTraceWriterAdd/10spans	allocated_mem	0/21 (0.0%)	4/21 (19.0%)	0.04%
BenchmarkOTLPTraceWriterAdd/10spans	execution_time	0/21 (0.0%)	1/21 (4.8%)	0.10%
BenchmarkOTLPTraceWriterAdd/1span	allocated_mem	0/21 (0.0%)	4/21 (19.0%)	0.06%
BenchmarkOTLPTraceWriterAdd/1span	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkOTLPTraceWriterAdd/50spans	allocated_mem	0/21 (0.0%)	5/21 (23.8%)	0.07%
BenchmarkOTLPTraceWriterAdd/50spans	execution_time	0/21 (0.0%)	3/21 (14.3%)	0.27%
BenchmarkOTLPTraceWriterAdd/5spans	allocated_mem	0/21 (0.0%)	1/21 (4.8%)	0.03%
BenchmarkOTLPTraceWriterAdd/5spans	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkOTLPTraceWriterConcurrent/concurrency_1	allocated_mem	0/21 (0.0%)	2/21 (9.5%)	0.02%
BenchmarkOTLPTraceWriterConcurrent/concurrency_1	execution_time	0/21 (0.0%)	4/21 (19.0%)	0.52%
BenchmarkOTLPTraceWriterConcurrent/concurrency_2	allocated_mem	0/21 (0.0%)	5/21 (23.8%)	0.08%
BenchmarkOTLPTraceWriterConcurrent/concurrency_2	execution_time	0/21 (0.0%)	6/21 (28.6%)	0.30%
BenchmarkOTLPTraceWriterConcurrent/concurrency_4	allocated_mem	0/21 (0.0%)	10/21 (47.6%)	0.23%
BenchmarkOTLPTraceWriterConcurrent/concurrency_4	execution_time	0/21 (0.0%)	4/21 (19.0%)	0.27%
BenchmarkOTLPTraceWriterConcurrent/concurrency_8	allocated_mem	0/21 (0.0%)	2/21 (9.5%)	0.04%
BenchmarkOTLPTraceWriterConcurrent/concurrency_8	execution_time	0/21 (0.0%)	4/21 (19.0%)	0.27%
BenchmarkOTLPTraceWriterFlush	allocated_mem	0/21 (0.0%)	6/21 (28.6%)	0.00%
BenchmarkOTelApiWithCustomTags/datadog_otel_api	allocated_mem	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkOTelApiWithCustomTags/datadog_otel_api	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkOTelApiWithCustomTags/otel_api	allocated_mem	0/21 (0.0%)	5/21 (23.8%)	0.08%
BenchmarkOTelApiWithCustomTags/otel_api	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkParallelLogs	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkParallelMetrics/count/get-handle	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkParallelMetrics/count/handle-reused	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkParallelMetrics/distribution/get-handle	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkParallelMetrics/distribution/handle-reused	execution_time	0/21 (0.0%)	4/21 (19.0%)	0.11%
BenchmarkParallelMetrics/gauge/get-handle	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkParallelMetrics/gauge/handle-reused	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkParallelMetrics/rate/get-handle	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkParallelMetrics/rate/handle-reused	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkPartialFlushing/Disabled	allocated_mem	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkPartialFlushing/Disabled	allocations	0/21 (0.0%)	1/21 (4.8%)	0.00%
BenchmarkPartialFlushing/Disabled	avgHeapInUse(Mb)	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkPartialFlushing/Disabled	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkPartialFlushing/Enabled	allocated_mem	0/21 (0.0%)	6/21 (28.6%)	0.28%
BenchmarkPartialFlushing/Enabled	allocations	0/21 (0.0%)	1/21 (4.8%)	0.01%
BenchmarkPartialFlushing/Enabled	avgHeapInUse(Mb)	0/21 (0.0%)	3/21 (14.3%)	1.13%
BenchmarkPartialFlushing/Enabled	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkPayloadVersions/detailed_1000spans/v0.4	allocated_mem	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkPayloadVersions/detailed_1000spans/v0.4	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkPayloadVersions/detailed_1000spans/v1.0	allocated_mem	0/21 (0.0%)	8/21 (38.1%)	0.08%
BenchmarkPayloadVersions/detailed_1000spans/v1.0	execution_time	0/21 (0.0%)	2/21 (9.5%)	0.05%
BenchmarkPayloadVersions/detailed_100spans/v0.4	execution_time	0/21 (0.0%)	7/21 (33.3%)	0.25%
BenchmarkPayloadVersions/detailed_100spans/v1.0	execution_time	0/21 (0.0%)	1/21 (4.8%)	0.06%
BenchmarkPayloadVersions/detailed_10spans/v0.4	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkPayloadVersions/detailed_10spans/v1.0	execution_time	0/21 (0.0%)	7/21 (33.3%)	0.37%
BenchmarkPayloadVersions/detailed_1spans/v0.4	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkPayloadVersions/detailed_1spans/v1.0	execution_time	0/21 (0.0%)	2/21 (9.5%)	0.09%
BenchmarkPayloadVersions/metastruct_1000spans/v1.0	allocated_mem	0/21 (0.0%)	7/21 (33.3%)	0.00%
BenchmarkPayloadVersions/metastruct_1000spans/v1.0	execution_time	0/21 (0.0%)	1/21 (4.8%)	0.00%
BenchmarkPayloadVersions/metastruct_100spans/v1.0	allocated_mem	0/21 (0.0%)	3/21 (14.3%)	0.00%
BenchmarkPayloadVersions/metastruct_100spans/v1.0	execution_time	0/21 (0.0%)	1/21 (4.8%)	0.03%
BenchmarkPayloadVersions/metastruct_10spans/v1.0	allocated_mem	0/21 (0.0%)	10/21 (47.6%)	0.00%
BenchmarkPayloadVersions/metastruct_10spans/v1.0	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkPayloadVersions/metastruct_1spans/v1.0	allocated_mem	0/21 (0.0%)	1/21 (4.8%)	0.00%
BenchmarkPayloadVersions/metastruct_1spans/v1.0	execution_time	0/21 (0.0%)	3/21 (14.3%)	0.27%
BenchmarkPayloadVersions/simple_1000spans/v0.4	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkPayloadVersions/simple_1000spans/v1.0	execution_time	0/21 (0.0%)	1/21 (4.8%)	0.00%
BenchmarkPayloadVersions/simple_100spans/v0.4	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkPayloadVersions/simple_100spans/v1.0	execution_time	0/21 (0.0%)	4/21 (19.0%)	0.28%
BenchmarkPayloadVersions/simple_10spans/v0.4	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkPayloadVersions/simple_10spans/v1.0	execution_time	0/21 (0.0%)	4/21 (19.0%)	0.13%
BenchmarkPayloadVersions/simple_1spans/v0.4	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkPayloadVersions/simple_1spans/v1.0	execution_time	0/21 (0.0%)	13/21 (61.9%)	0.65%
BenchmarkSampleWAFContext	allocated_mem	0/21 (0.0%)	9/21 (42.9%)	0.00%
BenchmarkSampleWAFContext	execution_time	0/21 (0.0%)	10/21 (47.6%)	0.27%
BenchmarkSerializeSpanLinksInMeta	execution_time	0/21 (0.0%)	7/21 (33.3%)	0.18%
BenchmarkSetTagMetric	execution_time	0/21 (0.0%)	7/21 (33.3%)	0.23%
BenchmarkSetTagString	execution_time	0/21 (0.0%)	8/21 (38.1%)	0.22%
BenchmarkSetTagStringPtr	execution_time	0/21 (0.0%)	2/21 (9.5%)	0.10%
BenchmarkSetTagStringer	execution_time	0/21 (0.0%)	5/21 (23.8%)	0.12%
BenchmarkSingleSpanRetention/no-rules	allocated_mem	0/21 (0.0%)	4/21 (19.0%)	0.00%
BenchmarkSingleSpanRetention/no-rules	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkSingleSpanRetention/with-rules/match-all	allocated_mem	0/21 (0.0%)	7/21 (33.3%)	0.00%
BenchmarkSingleSpanRetention/with-rules/match-all	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkSingleSpanRetention/with-rules/match-half	allocated_mem	0/21 (0.0%)	3/21 (14.3%)	0.00%
BenchmarkSingleSpanRetention/with-rules/match-half	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkStartRequestSpan	execution_time	0/21 (0.0%)	2/21 (9.5%)	0.06%
BenchmarkStartSpan	execution_time	0/21 (0.0%)	1/21 (4.8%)	0.02%
BenchmarkStartSpanConfig/scenario_WithStartSpanConfig	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkStartSpanConfig/scenario_none	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%
BenchmarkTracerAddSpans	execution_time	0/21 (0.0%)	0/21 (0.0%)	0.00%

Significant Impact FP Rate: 0.5%

T-test FP Rate: 13.5%

Max Significant Impact Bound: 9.77%

p95 Significant Impact Bound: 0.43%

darccio

Solid CI improvement. Some nits added.

Co-authored-by: Dario Castañé <[email protected]>

ddyurchenko requested review from a team as code owners May 4, 2026 15:25

ddyurchenko marked this pull request as draft May 4, 2026 15:28

darccio requested changes May 4, 2026

View reviewed changes

Comment thread .gitlab/benchmarks/micro/container/Dockerfile Outdated

Comment thread CODEOWNERS Outdated

ddyurchenko marked this pull request as ready for review May 5, 2026 13:52

ddyurchenko added 3 commits May 5, 2026 15:56

Deprecate dependency on benchmarking-platform repo

fb898b6

Update codeowners

48fe759

Update microbenchmark runs

7720ab4

chatgpt-codex-connector Bot reviewed May 5, 2026

View reviewed changes

Comment thread .gitlab/benchmarks/micro/gitlab-ci.yml Outdated

Comment thread .gitlab/benchmarks/micro/gitlab-ci.yml Outdated

Comment thread .gitlab/benchmarks/micro/bp-runner.yml

Comment thread .gitlab/benchmarks/micro/steps/run-benchmarks-with-profiler.sh Outdated

Update microbenchmark runs

14a6bfb

ddyurchenko force-pushed the ddyurchenko/deprecate-bp-repo branch from 0656701 to 14a6bfb Compare May 5, 2026 13:58

ddyurchenko changed the title ~~Deprecate dependency on benchmarking-platform repo & parallelize execution~~ ci(benchmarks): Parellelize microbenchmarks, deprecate dependency on benchmarking-platform repo May 5, 2026

Fix Gitlab definitions

246f843

ddyurchenko added 12 commits May 5, 2026 17:09

Remove extra baseline check as it is now part of bp-runner

e39d0b0

Remove deprecated steps folder

2c502cf

Remove explicit silencing in how_to_run_benchmarks

3492f50

Replace parallel:matrix with shorhand Ci job names

7b15491

Fix path for benchmarks CI image build

eef7b6b

Update microbenchmarks image

99aa169

Separate deps install from benchmarks run

da0da19

Separate deps install from benchmarks run

7b60546

Fix pipeline_id env var

a896c84

Update input_filter_pattern and deps installation

2c8a537

Update input_filter_pattern

b5305de

Fix BASE_IMAGE -> BENCHMARKS_CI_IMAGE

8b37087

ddyurchenko added 2 commits May 6, 2026 15:16

Increase repetitions back to 10

fdbd765

Add known flaky benchmarks

cc25a08

Add documentation

fd2f91f

ddyurchenko requested a review from darccio May 7, 2026 11:49

Fix errors in .env.example

59ee973

darccio approved these changes May 7, 2026

View reviewed changes

Comment thread .gitlab/benchmarks/micro/container/install-base-deps.sh Outdated

Comment thread .gitlab/benchmarks/micro/gitlab-ci.yml

Comment thread .gitlab/benchmarks/micro/bp-runner.yml Outdated

Comment thread .gitlab/benchmarks/micro/bp-runner.yml Outdated

ddyurchenko and others added 5 commits May 7, 2026 09:22

Update .gitlab/benchmarks/micro/container/install-base-deps.sh

547b437

Co-authored-by: Dario Castañé <[email protected]>

Update .gitlab/benchmarks/micro/bp-runner.yml

90aeba7

Co-authored-by: Dario Castañé <[email protected]>

Update rules

b6c64ee

Update grep rule and bp-runner command for local runs

acd7276

Merge branch 'main' into ddyurchenko/deprecate-bp-repo

7f770e3

darccio merged commit 0ec584d into main May 7, 2026
153 of 160 checks passed

darccio deleted the ddyurchenko/deprecate-bp-repo branch May 7, 2026 14:49

Uh oh!

Conversation

ddyurchenko commented May 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Key changes

Motivation

Reviewer's Checklist

Uh oh!

Uh oh!

Uh oh!

datadog-prod-us1-5 Bot commented May 5, 2026 • edited by datadog-datadog-prod-us1-2 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codecov Bot commented May 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

pr-commenter Bot commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Explanation

More details about the CI and significant changes

Known flaky benchmarks

Uh oh!

ddyurchenko commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Stability Analysis Report

Stability Results

Uh oh!

darccio left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ddyurchenko commented May 4, 2026 •

edited

Loading

datadog-prod-us1-5 Bot commented May 5, 2026 •

edited by datadog-datadog-prod-us1-2 Bot

Loading

codecov Bot commented May 5, 2026 •

edited

Loading

pr-commenter Bot commented May 6, 2026 •

edited

Loading

ddyurchenko commented May 6, 2026 •

edited

Loading