ci(benchmarks): support BP_EXTERNAL_S3_URL override#9155
Conversation
Add optional EXTERNAL_S3_URL to redirect uploads to an orchestrator-controlled prefix, with a prefix validation guard (must stay within relenv-benchmarking-data). Write a _READY marker object after each upload so downstream pollers know the sample is complete. Default behavior is unchanged when EXTERNAL_S3_URL is unset. Co-Authored-By: Claude Opus 4.8 <[email protected]>
Overall package sizeSelf size: 6.65 MB Dependency sizes| name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.3.1 | 122.62 kB | 437.94 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |🤖 This report was automatically generated by heaviest-objects-in-the-universe |
Collapse the override to ${EXTERNAL_S3_URL:-default} with a single prefix
guard and the _READY marker, so the per-repo diff stays small when this is
replicated across dd-trace-* repos.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: 39bf127 | Docs | Datadog PR Page | Give us feedback! |
Completion is signalled by the bench pipeline reaching success, polled by the orchestrator, so a per-sample S3 marker is unnecessary. Leaves the override and its prefix guard as the only change. Co-Authored-By: Claude Opus 4.8 <[email protected]>
BenchmarksBenchmark execution time: 2026-07-08 14:59:47 Comparing candidate commit 39bf127 in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 2312 metrics, 46 unstable metrics.
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #9155 +/- ##
========================================
Coverage 96.54% 96.54%
========================================
Files 915 915
Lines 121034 121034
Branches 20808 20610 -198
========================================
+ Hits 116849 116851 +2
+ Misses 4185 4183 -2 Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Namespace the orchestrator-supplied override with the BP_ prefix, matching the other bp-tools contract variables. Co-Authored-By: Claude Opus 4.8 <[email protected]>
### What does this PR do? Adds an opt-in `BP_EXTERNAL_S3_URL` override to the microbenchmark upload path (`.gitlab/benchmarks/micro/gitlab-ci.yml`). When set, it overrides the S3 destination prefix for the artifact upload. When unset, the existing path is used unchanged. The value is used as-is when set; an empty value falls back to the existing default destination. ### Motivation [APMSP-2893 Set up flaky benchmark monitoring](https://datadoghq.atlassian.net/browse/APMSP-2893) - benchmarking-platform-tools will orchestrate dd-trace-go benchmark jobs for monitoring benchmark health and needs results uploaded under a s3 URL it controls. - The override lets the orchestrator pass that prefix in via an env var, without changing default behavior. This mirrors the same change on dd-trace-js: DataDog/dd-trace-js#9155 ### Reviewer's Checklist - [ ] Changed code has unit tests for its functionality at or near 100% coverage. - [ ] [System-Tests](https://github.com/DataDog/system-tests/) covering this feature have been added and enabled with the va.b.c-dev version tag. - [ ] There is a benchmark for any new code, or changes to existing code. - [ ] If this interacts with the agent in a new way, a system test has been added. - [ ] New code is free of linting errors. You can check this by running `make lint` locally. - [ ] New code doesn't break existing tests. You can check this by running `make test` locally. - [ ] Add an appropriate team label so this PR gets put in the right place for the release notes. - [ ] All generated files are up to date. You can check this by running `make generate` locally. - [ ] Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild. Make sure all nested modules are up to date by running `make fix-modules` locally.
* ci(benchmarks): support EXTERNAL_S3_URL override and _READY marker Add optional EXTERNAL_S3_URL to redirect uploads to an orchestrator-controlled prefix, with a prefix validation guard (must stay within relenv-benchmarking-data). Write a _READY marker object after each upload so downstream pollers know the sample is complete. Default behavior is unchanged when EXTERNAL_S3_URL is unset. Co-Authored-By: Claude Opus 4.8 <[email protected]> * ci(benchmarks): trim EXTERNAL_S3_URL override to minimal form Collapse the override to ${EXTERNAL_S3_URL:-default} with a single prefix guard and the _READY marker, so the per-repo diff stays small when this is replicated across dd-trace-* repos. Co-Authored-By: Claude Opus 4.8 <[email protected]> * ci(benchmarks): drop _READY marker, keep EXTERNAL_S3_URL only Completion is signalled by the bench pipeline reaching success, polled by the orchestrator, so a per-sample S3 marker is unnecessary. Leaves the override and its prefix guard as the only change. Co-Authored-By: Claude Opus 4.8 <[email protected]> * ci(benchmarks): rename override var to BP_EXTERNAL_S3_URL Namespace the orchestrator-supplied override with the BP_ prefix, matching the other bp-tools contract variables. Co-Authored-By: Claude Opus 4.8 <[email protected]> --------- Co-authored-by: Claude Opus 4.8 <[email protected]>
* ci(benchmarks): support EXTERNAL_S3_URL override and _READY marker Add optional EXTERNAL_S3_URL to redirect uploads to an orchestrator-controlled prefix, with a prefix validation guard (must stay within relenv-benchmarking-data). Write a _READY marker object after each upload so downstream pollers know the sample is complete. Default behavior is unchanged when EXTERNAL_S3_URL is unset. Co-Authored-By: Claude Opus 4.8 <[email protected]> * ci(benchmarks): trim EXTERNAL_S3_URL override to minimal form Collapse the override to ${EXTERNAL_S3_URL:-default} with a single prefix guard and the _READY marker, so the per-repo diff stays small when this is replicated across dd-trace-* repos. Co-Authored-By: Claude Opus 4.8 <[email protected]> * ci(benchmarks): drop _READY marker, keep EXTERNAL_S3_URL only Completion is signalled by the bench pipeline reaching success, polled by the orchestrator, so a per-sample S3 marker is unnecessary. Leaves the override and its prefix guard as the only change. Co-Authored-By: Claude Opus 4.8 <[email protected]> * ci(benchmarks): rename override var to BP_EXTERNAL_S3_URL Namespace the orchestrator-supplied override with the BP_ prefix, matching the other bp-tools contract variables. Co-Authored-By: Claude Opus 4.8 <[email protected]> --------- Co-authored-by: Claude Opus 4.8 <[email protected]>
What does this PR do?
Adds an opt-in
BP_EXTERNAL_S3_URLoverride to.gitlab/benchmarks/upload-results-to-s3.sh. When set, it overrides the S3 destination prefix for the artifact upload. When unset, the existing path is used unchanged.BP_EXTERNAL_S3_URLis validated to start withs3://relenv-benchmarking-data/before use. Any other value exits non-zero with a clear error, preventing uploads to an arbitrary bucket.Motivation
APMSP-2893 Set up flaky benchmark monitoring
This will also be used on allowing libdatadog to run benchmarks on dd-trace-js, futurely: (WIP) Libdatadog benchmarks on dd-trace-* repos