ci: preserve coverage from custom plugin jobs by BridgeAR · Pull Request #9386 · DataDog/dd-trace-js

BridgeAR · 2026-07-15T16:43:38Z

Summary

Express and Electron ran coverage-producing plugin suites but discarded their reports when the jobs ended, which left exercised code missing from Codecov. Upload both reports and make verify-exercised-tests reject any workflow job whose coverage collector cannot reach the uploader.

Express and Electron generated V8 coverage but discarded the reports when their jobs ended, so Codecov could report exercised plugin code as uncovered. Reject coverage-producing jobs that cannot reach the uploader to prevent silent regressions.

dd-octo-sts · 2026-07-15T16:44:30Z

Overall package size

Self size: 6.77 MB
Deduped: 7.43 MB
No deduping: 7.43 MB

Dependency sizes

| name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.3.1 | 122.62 kB | 438.86 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |

_{🤖 This report was automatically generated by heaviest-objects-in-the-universe}

datadog-datadog-prod-us1 · 2026-07-15T16:50:07Z

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🔄 Datadog auto-retried 1 job - 1 passed on retry

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 18b29c3 | Docs | Datadog PR Page | Give us feedback!}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 18b29c3935

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-07-15T16:54:48Z

+          typeof step.uses === 'string' &&
+          localActionUploadsCoverage(repoRoot, step.uses, new Set())
+        ) {
+          coverageUploadJobs.add(`${wf}#${jobId}`)


Check uploader reachability in step order

When a job contains one uploader anywhere, this job-level set makes every coverage-producing run in that job look covered. In ordered multi-step jobs like .github/workflows/apm-integrations.yml's dns job, which runs several yarn test:plugins:ci steps and uploads once at the end, a later edit that adds or moves a scripts/c8-ci.js/:coverage step after the uploader would still pass verify-exercised-tests, even though .github/actions/upload-coverage-artifact/action.yml only uploads files present when that step runs. Track uploader positions, or scan steps in order, so each coverage producer must be followed by an uploader.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-07-15T16:54:48Z

+  visiting.add(actionFile)
+
+  const doc = parseYamlFile(repoRoot, actionFile)
+  const steps = doc.runs.steps


Guard non-composite local actions

If any workflow starts using a valid local JavaScript or Docker action, this dereference throws because those actions have no runs.steps, so verify-exercised-tests crashes instead of continuing with its CI checks. The existing composite expansion above guards doc, runs, and steps; this path should do the same before scanning nested uses steps.

Useful? React with 👍 / 👎.

pr-commenter · 2026-07-15T16:58:40Z

Benchmarks

Benchmark execution time: 2026-07-15 16:58:36

Comparing candidate commit 18b29c3 in PR branch BridgeAR/2026-07-15-enforce-coverage-upload with baseline commit e45de38 in branch master.

📊 Benchmarking dashboard

Found 0 performance improvements and 0 performance regressions! Performance is the same for 2310 metrics, 48 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

🟩 = significantly better candidate vs. baseline
🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:appsec-appsec-enabled-24

unstable execution_time [-217.867ms; +220.765ms] or [-8.210%; +8.319%]

scenario:appsec-appsec-enabled-26

unstable execution_time [-245126.889µs; +246911.522µs] or [-9.537%; +9.606%]

scenario:appsec-appsec-enabled-with-attacks-24

unstable execution_time [-164.159ms; +155.502ms] or [-5.317%; +5.037%]

scenario:appsec-appsec-enabled-with-attacks-26

unstable execution_time [-190.356ms; +188.135ms] or [-6.540%; +6.464%]

scenario:appsec-control-20

unstable execution_time [-121.232ms; +123.696ms] or [-7.377%; +7.527%]

scenario:appsec-control-24

unstable execution_time [-113.997ms; +117.349ms] or [-9.151%; +9.420%]

scenario:appsec-control-26

unstable execution_time [-106.671ms; +137.188ms] or [-8.731%; +11.229%]

scenario:appsec-iast-no-vulnerability-iast-enabled-always-active-20

unstable execution_time [-10.095ms; +17.662ms] or [-4.027%; +7.045%]

scenario:appsec-iast-no-vulnerability-iast-enabled-default-config-20

unstable execution_time [-10.013ms; +24.486ms] or [-3.822%; +9.345%]

scenario:appsec-iast-with-vulnerability-iast-enabled-always-active-20

unstable execution_time [-30622.305µs; +30779.705µs] or [-5.560%; +5.588%]

scenario:appsec-iast-with-vulnerability-iast-enabled-default-config-20

unstable execution_time [-26.256ms; +29.214ms] or [-4.799%; +5.339%]

scenario:debugger-line-probe-with-snapshot-default-24

unstable cpu_user_time [-1718.658ms; +565.195ms] or [-20.707%; +6.810%]
unstable execution_time [-1715.111ms; +581.989ms] or [-19.026%; +6.456%]
unstable instructions [-14.7G instructions; +4.7G instructions] or [-21.744%; +7.005%]
unstable throughput [-161.815op/s; +458.742op/s] or [-4.433%; +12.566%]

scenario:debugger-line-probe-with-snapshot-default-26

unstable cpu_user_time [-2705.164ms; +4205.855ms] or [-28.310%; +44.015%]
unstable execution_time [-2819.653ms; +4308.066ms] or [-27.358%; +41.800%]
unstable instructions [-23.2G instructions; +36.8G instructions] or [-29.115%; +46.151%]
unstable max_rss_usage [-8.097MB; +13.307MB] or [-5.088%; +8.361%]
unstable throughput [-822.124op/s; +556.636op/s] or [-25.511%; +17.273%]

scenario:debugger-line-probe-with-snapshot-minimal-24

unstable cpu_user_time [-1728.506ms; +575.172ms] or [-20.940%; +6.968%]
unstable execution_time [-1732.507ms; +568.520ms] or [-19.356%; +6.352%]
unstable instructions [-14.6G instructions; +4.8G instructions] or [-21.564%; +7.178%]
unstable throughput [-155.112op/s; +466.765op/s] or [-4.217%; +12.689%]

scenario:debugger-line-probe-with-snapshot-minimal-26

unstable cpu_user_time [-2280.238ms; +740.403ms] or [-23.876%; +7.753%]
unstable execution_time [-2286.272ms; +738.005ms] or [-22.253%; +7.183%]
unstable instructions [-20.3G instructions; +6.6G instructions] or [-25.445%; +8.275%]
unstable throughput [-146.646op/s; +451.610op/s] or [-4.548%; +14.006%]

scenario:debugger-line-probe-without-snapshot-24

unstable cpu_user_time [-1750.467ms; +567.431ms] or [-21.166%; +6.861%]
unstable execution_time [-1764.412ms; +583.828ms] or [-19.656%; +6.504%]
unstable instructions [-14.6G instructions; +4.8G instructions] or [-21.681%; +7.053%]
unstable throughput [-160.508op/s; +473.583op/s] or [-4.373%; +12.903%]

scenario:debugger-line-probe-without-snapshot-26

unstable cpu_user_time [-4.663s; +0.005s] or [-41.992%; +0.043%]
unstable execution_time [-4.727s; +0.018s] or [-39.919%; +0.150%]
unstable instructions [-41.1G instructions; -0.1G instructions] or [-44.041%; -0.081%]
unstable throughput [-7.738op/s; +918.324op/s] or [-0.264%; +31.381%]

scenario:dogstatsd-with-tags-20

unstable cpu_user_time [-341.139ms; +347.768ms] or [-7.101%; +7.239%]
unstable execution_time [-337.193ms; +349.369ms] or [-6.902%; +7.151%]
unstable throughput [-125553.444op/s; +117228.026op/s] or [-7.301%; +6.817%]

scenario:plugin-claude-agent-sdk-compact-stream-scan-24

unstable cpu_usage_percentage [-5.758%; +4.669%]

scenario:plugin-graphql-long-with-depth-and-collapse-off-20

unstable max_rss_usage [-23.558MB; +39.554MB] or [-6.254%; +10.501%]

scenario:plugin-graphql-long-with-depth-off-20

unstable max_rss_usage [-6362.264KB; +6464.550KB] or [-4.970%; +5.050%]

scenario:plugin-graphql-long-with-depth-on-max-20

unstable cpu_user_time [-721.054ms; +740.588ms] or [-5.624%; +5.777%]
unstable execution_time [-726.159ms; +750.448ms] or [-5.546%; +5.732%]
unstable throughput [-3.443op/s; +3.363op/s] or [-5.601%; +5.470%]

scenario:plugin-pg-service-20

unstable cpu_usage_percentage [-7.711%; +4.838%]
unstable execution_time [-113.243ms; +174.030ms] or [-6.948%; +10.677%]
unstable throughput [-292110.308op/s; +190909.509op/s] or [-7.853%; +5.132%]

Express and Electron generated V8 coverage but discarded the reports when their jobs ended, so Codecov could report exercised plugin code as uncovered. Reject coverage-producing jobs that cannot reach the uploader to prevent silent regressions.

github-actions Bot added the semver-patch label Jul 15, 2026

BridgeAR marked this pull request as ready for review July 15, 2026 16:45

BridgeAR requested review from a team as code owners July 15, 2026 16:45

BridgeAR requested review from khanayan123 and tlhunter and removed request for a team July 15, 2026 16:45

tlhunter approved these changes Jul 15, 2026

View reviewed changes

chatgpt-codex-connector Bot reviewed Jul 15, 2026

View reviewed changes

tlhunter merged commit 4f01d4e into master Jul 15, 2026
689 of 691 checks passed

tlhunter deleted the BridgeAR/2026-07-15-enforce-coverage-upload branch July 15, 2026 17:06

dd-octo-sts Bot mentioned this pull request Jul 16, 2026

v6.4.0 proposal #9368

Merged

dd-octo-sts Bot mentioned this pull request Jul 16, 2026

v5.115.0 proposal #9369

Merged

Uh oh!

Conversation

BridgeAR commented Jul 15, 2026

Summary

Uh oh!

dd-octo-sts Bot commented Jul 15, 2026

Overall package size

Uh oh!

datadog-datadog-prod-us1 Bot commented Jul 15, 2026 • edited by datadog-prod-us1-3 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

pr-commenter Bot commented Jul 15, 2026

Benchmarks

Explanation

More details about the CI and significant changes

Unstable benchmarks

scenario:appsec-appsec-enabled-24

scenario:appsec-appsec-enabled-26

scenario:appsec-appsec-enabled-with-attacks-24

scenario:appsec-appsec-enabled-with-attacks-26

scenario:appsec-control-20

scenario:appsec-control-24

scenario:appsec-control-26

scenario:appsec-iast-no-vulnerability-iast-enabled-always-active-20

scenario:appsec-iast-no-vulnerability-iast-enabled-default-config-20

scenario:appsec-iast-with-vulnerability-iast-enabled-always-active-20

scenario:appsec-iast-with-vulnerability-iast-enabled-default-config-20

scenario:debugger-line-probe-with-snapshot-default-24

scenario:debugger-line-probe-with-snapshot-default-26

scenario:debugger-line-probe-with-snapshot-minimal-24

scenario:debugger-line-probe-with-snapshot-minimal-26

scenario:debugger-line-probe-without-snapshot-24

scenario:debugger-line-probe-without-snapshot-26

scenario:dogstatsd-with-tags-20

scenario:plugin-claude-agent-sdk-compact-stream-scan-24

scenario:plugin-graphql-long-with-depth-and-collapse-off-20

scenario:plugin-graphql-long-with-depth-off-20

scenario:plugin-graphql-long-with-depth-on-max-20

scenario:plugin-pg-service-20

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

datadog-datadog-prod-us1 Bot commented Jul 15, 2026 •

edited by datadog-prod-us1-3 Bot

Loading