Skip to content

test(child_process): fix Bluebird flake that cascades to every later spec#9078

Merged
BridgeAR merged 1 commit into
masterfrom
BridgeAR/2026-06-26-fix-child-process-bluebird-flake
Jul 2, 2026
Merged

test(child_process): fix Bluebird flake that cascades to every later spec#9078
BridgeAR merged 1 commit into
masterfrom
BridgeAR/2026-06-26-fix-child-process-bluebird-flake

Conversation

@BridgeAR

Copy link
Copy Markdown
Member

Summary

The Bluebird specs set global.Promise = Bluebird in beforeEach and restored
it only in afterEach, holding the mutation across the awaited span round-trip.
Under load the subprocess span arrives late, and the shared assertion helper only
rejects once a payload has already errored, so the spec hangs to mocha's 5 s
timeout. Meanwhile leftover async keeps global.Promise flipped, corrupting
context for every later spec in the file — one slow span became 47 failures.

The instrumentation only reads global.Promise synchronously, when the wrapped
method runs. The fix scopes the swap to that window via withBluebird() and
restores it in a finally before any await.

Drive-by

  • Drop the dead delete require.cache[require.resolve('util')] (util is a builtin).
  • Replace the try/catch rejection check with assert.rejects.

@dd-octo-sts

dd-octo-sts Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

Overall package size

Self size: 6.44 MB
Deduped: 7.5 MB
No deduping: 7.5 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.2.0 | 104.26 kB | 843.44 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

@codecov

codecov Bot commented Jun 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.73%. Comparing base (a4a3beb) to head (9e59a8f).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #9078   +/-   ##
=======================================
  Coverage   93.73%   93.73%           
=======================================
  Files         895      895           
  Lines       51398    51399    +1     
  Branches    11980    11980           
=======================================
+ Hits        48178    48180    +2     
+ Misses       3220     3219    -1     
Flag Coverage Δ
aiguard 34.87% <ø> (-0.08%) ⬇️
aiguard-integration 41.79% <ø> (ø)
apm-bucket-0 34.84% <ø> (-0.08%) ⬇️
apm-bucket-1 40.33% <ø> (-0.07%) ⬇️
apm-bucket-2 37.34% <ø> (-0.07%) ⬇️
apm-capabilities-tracing 48.69% <ø> (+<0.01%) ⬆️
apm-integrations-aerospike 33.12% <ø> (-0.08%) ⬇️
apm-integrations-confluentinc-kafka-javascript 40.02% <ø> (-0.08%) ⬇️
apm-integrations-couchbase 33.24% <ø> (-0.08%) ⬇️
apm-integrations-http 41.95% <ø> (-0.07%) ⬇️
apm-integrations-kafkajs 40.30% <ø> (-0.04%) ⬇️
apm-integrations-next 29.46% <ø> (-0.11%) ⬇️
apm-integrations-prisma 35.03% <ø> (-0.07%) ⬇️
apm-integrations-tedious 33.89% <ø> (-0.08%) ⬇️
appsec 57.62% <ø> (-0.15%) ⬇️
appsec-express_fastify_graphql 53.66% <ø> (-0.07%) ⬇️
appsec-integration 36.44% <ø> (ø)
appsec-kafka_ldapjs_lodash 43.58% <ø> (-0.07%) ⬇️
appsec-mongodb-core_mongoose_mysql 48.76% <ø> (-0.04%) ⬇️
appsec-next 28.01% <ø> (-0.06%) ⬇️
appsec-node-serialize_passport_postgres 47.91% <ø> (-0.06%) ⬇️
appsec-sourcing_stripe_template 45.47% <ø> (-0.06%) ⬇️
debugger 44.37% <ø> (ø)
instrumentations-bucket-0 28.15% <ø> (-0.08%) ⬇️
instrumentations-bucket-1 37.40% <ø> (-0.07%) ⬇️
instrumentations-bucket-10 40.35% <ø> (-0.07%) ⬇️
instrumentations-bucket-11 27.95% <ø> (-0.08%) ⬇️
instrumentations-bucket-12 28.67% <ø> (-0.08%) ⬇️
instrumentations-bucket-13 27.78% <ø> (-0.08%) ⬇️
instrumentations-bucket-2 30.23% <ø> (-0.07%) ⬇️
instrumentations-bucket-3 35.90% <ø> (-0.07%) ⬇️
instrumentations-bucket-4 28.56% <ø> (-0.08%) ⬇️
instrumentations-bucket-5 36.28% <ø> (-0.07%) ⬇️
instrumentations-bucket-6 38.25% <ø> (-0.08%) ⬇️
instrumentations-bucket-7 36.01% <ø> (-0.07%) ⬇️
instrumentations-bucket-8 36.95% <ø> (-0.07%) ⬇️
instrumentations-bucket-9 39.45% <ø> (-0.07%) ⬇️
instrumentations-instrumentation-couchbase 46.48% <ø> (ø)
instrumentations-integration-esbuild 25.05% <ø> (+<0.01%) ⬆️
llmobs-ai_anthropic_bedrock 39.53% <ø> (-0.07%) ⬇️
llmobs-google-genai_langchain_vertex-ai 36.97% <ø> (-0.06%) ⬇️
llmobs-openai 39.58% <ø> (-0.07%) ⬇️
llmobs-sdk 43.59% <ø> (-0.08%) ⬇️
master-coverage 93.73% <ø> (?)
openfeature 37.76% <ø> (ø)
openfeature-unit 50.39% <ø> (ø)
platform-core_esbuild_instrumentations-misc 23.50% <ø> (ø)
platform-integration 47.42% <ø> (ø)
platform-shimmer_unit-guardrails_webpack 19.02% <ø> (ø)
plugins-bucket-0 36.38% <ø> (-0.06%) ⬇️
plugins-bucket-1 39.60% <ø> (ø)
plugins-bucket-11 38.46% <ø> (-0.08%) ⬇️
plugins-bucket-17 39.15% <ø> (-0.07%) ⬇️
plugins-bucket-18 42.01% <ø> (-0.07%) ⬇️
plugins-bucket-19 39.56% <ø> (-0.07%) ⬇️
plugins-bucket-20 43.25% <ø> (-0.07%) ⬇️
plugins-bucket-4 37.71% <ø> (-0.07%) ⬇️
plugins-bullmq_cassandra_cookie 39.76% <ø> (-0.07%) ⬇️
plugins-cookie-parser_crypto_dd-trace-api 33.22% <ø> (-0.08%) ⬇️
plugins-fetch_fs_generic-pool 36.05% <ø> (+0.08%) ⬆️
plugins-google-cloud-pubsub_grpc_handlebars 43.10% <ø> (-0.07%) ⬇️
plugins-hapi_hono_ioredis 37.78% <ø> (-0.07%) ⬇️
plugins-jest_knex_langgraph 32.60% <ø> (-0.08%) ⬇️
plugins-ldapjs_light-my-request_limitd-client 27.91% <ø> (-0.08%) ⬇️
plugins-lodash_mariadb_memcached 35.17% <ø> (-0.08%) ⬇️
plugins-mongodb_mongodb-core_mongoose 36.31% <ø> (-0.09%) ⬇️
plugins-multer_mysql_mysql2 35.14% <ø> (-0.08%) ⬇️
plugins-nats_node-serialize_opensearch 37.14% <ø> (-0.08%) ⬇️
plugins-passport-http_pino_postgres 35.53% <ø> (-0.08%) ⬇️
plugins-process_pug_redis 34.26% <ø> (-0.08%) ⬇️
plugins-undici_url_valkey 35.88% <ø> (-0.07%) ⬇️
plugins-vm_winston_ws 37.55% <ø> (-0.08%) ⬇️
profiling 43.62% <ø> (-0.07%) ⬇️
serverless-aws-sdk-aws-sdk 33.29% <ø> (-0.07%) ⬇️
serverless-aws-sdk-bedrockruntime 32.16% <ø> (-0.07%) ⬇️
serverless-aws-sdk-client 37.16% <ø> (ø)
serverless-aws-sdk-dynamodb 34.14% <ø> (-0.07%) ⬇️
serverless-aws-sdk-eventbridge 27.25% <ø> (-0.07%) ⬇️
serverless-aws-sdk-kinesis 37.40% <ø> (-0.07%) ⬇️
serverless-aws-sdk-lambda 34.59% <ø> (-0.07%) ⬇️
serverless-aws-sdk-s3 32.59% <ø> (-0.07%) ⬇️
serverless-aws-sdk-serverless-peer-service 39.47% <ø> (-0.08%) ⬇️
serverless-aws-sdk-sns 38.25% <ø> (-0.19%) ⬇️
serverless-aws-sdk-sqs 38.00% <ø> (-0.07%) ⬇️
serverless-aws-sdk-stepfunctions 33.18% <ø> (-0.07%) ⬇️
serverless-aws-sdk-util 47.95% <ø> (ø)
serverless-bucket-0 39.46% <ø> (ø)
serverless-lambda 34.33% <ø> (-0.09%) ⬇️
test-optimization-cucumber 52.30% <ø> (+0.11%) ⬆️
test-optimization-cypress 49.52% <ø> (+0.06%) ⬆️
test-optimization-jest 55.37% <ø> (+0.17%) ⬆️
test-optimization-mocha 53.41% <ø> (+0.11%) ⬆️
test-optimization-playwright-playwright-atr 43.56% <ø> (+0.31%) ⬆️
test-optimization-playwright-playwright-efd 43.61% <ø> (+0.08%) ⬆️
test-optimization-playwright-playwright-final-status 43.73% <ø> (+0.10%) ⬆️
test-optimization-playwright-playwright-impacted-tests 43.16% <ø> (ø)
test-optimization-playwright-playwright-reporting 43.26% <ø> (+0.08%) ⬆️
test-optimization-playwright-playwright-test-management 44.72% <ø> (+0.08%) ⬆️
test-optimization-playwright-playwright-test-span 44.51% <ø> (+0.07%) ⬆️
test-optimization-selenium 45.26% <ø> (+0.06%) ⬆️
test-optimization-testopt 48.01% <ø> (+0.08%) ⬆️
test-optimization-vitest 51.02% <ø> (+0.12%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pr-commenter

pr-commenter Bot commented Jun 26, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-07-01 15:50:01

Comparing candidate commit 9e59a8f in PR branch BridgeAR/2026-06-26-fix-child-process-bluebird-flake with baseline commit a4a3beb in branch master.

📊 Benchmarking dashboard

Found 0 performance improvements and 0 performance regressions! Performance is the same for 2255 metrics, 31 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:appsec-appsec-enabled-24

  • unstable execution_time [-181866.572µs; +183282.572µs] or [-6.906%; +6.960%]

scenario:appsec-appsec-enabled-26

  • unstable execution_time [-206.057ms; +208.293ms] or [-8.208%; +8.297%]

scenario:appsec-appsec-enabled-with-attacks-26

  • unstable execution_time [-153338.799µs; +154099.799µs] or [-5.333%; +5.359%]

scenario:appsec-control-20

  • unstable execution_time [-157.748ms; +162.692ms] or [-8.931%; +9.211%]

scenario:appsec-control-24

  • unstable execution_time [-103405.453µs; +104685.220µs] or [-8.500%; +8.605%]

scenario:appsec-control-26

  • unstable execution_time [-112866.874µs; +113231.074µs] or [-9.339%; +9.369%]

scenario:debugger-line-probe-with-snapshot-default-24

  • unstable cpu_user_time [-1884.929ms; +2981.324ms] or [-23.820%; +37.676%]
  • unstable execution_time [-1907.313ms; +3000.374ms] or [-22.129%; +34.811%]
  • unstable instructions [-15.7G instructions; +25.2G instructions] or [-24.471%; +39.114%]
  • unstable max_rss_usage [-6.361MB; +10.744MB] or [-4.024%; +6.797%]
  • unstable throughput [-881.715op/s; +565.319op/s] or [-23.037%; +14.770%]

scenario:debugger-line-probe-with-snapshot-default-26

  • unstable cpu_user_time [-2966189.986µs; +2965801.386µs] or [-29.852%; +29.848%]
  • unstable execution_time [-3041.171ms; +3018.521ms] or [-28.576%; +28.363%]
  • unstable instructions [-26170.6M instructions; +26098.4M instructions] or [-31.508%; +31.421%]
  • unstable max_rss_usage [-9140.841KB; +8700.041KB] or [-5.674%; +5.401%]
  • unstable throughput [-656.177op/s; +672.395op/s] or [-20.842%; +21.357%]

scenario:debugger-line-probe-with-snapshot-minimal-26

  • unstable cpu_user_time [-2137.339ms; +3397.612ms] or [-23.017%; +36.589%]
  • unstable execution_time [-2174.184ms; +3436.197ms] or [-21.827%; +34.497%]
  • unstable instructions [-18.7G instructions; +30.1G instructions] or [-24.276%; +39.023%]
  • unstable max_rss_usage [-6.572MB; +9.499MB] or [-4.119%; +5.954%]
  • unstable throughput [-757.467op/s; +479.946op/s] or [-22.915%; +14.520%]

scenario:debugger-line-probe-without-snapshot-24

  • unstable cpu_user_time [-1898.887ms; +3023.051ms] or [-24.043%; +38.276%]
  • unstable execution_time [-1903.848ms; +3055.065ms] or [-22.153%; +35.548%]
  • unstable instructions [-16.2G instructions; +25.8G instructions] or [-25.187%; +40.075%]
  • unstable max_rss_usage [-6.845MB; +10.725MB] or [-4.329%; +6.783%]
  • unstable throughput [-901.070op/s; +558.885op/s] or [-23.457%; +14.549%]

scenario:dogstatsd-with-tags-20

  • unstable cpu_user_time [-481.873ms; +100.004ms] or [-10.188%; +2.114%]
  • unstable execution_time [-478.218ms; +103.617ms] or [-9.950%; +2.156%]
  • unstable throughput [-37430.867op/s; +165070.991op/s] or [-2.143%; +9.449%]

scenario:plugin-graphql-long-with-depth-and-collapse-off-20

  • unstable max_rss_usage [-28.301MB; +19.823MB] or [-7.414%; +5.193%]

scenario:spans-finish-later-26

  • unstable max_rss_usage [-6.595MB; +13.572MB] or [-4.417%; +9.091%]

@BridgeAR
BridgeAR marked this pull request as ready for review June 26, 2026 09:10
@BridgeAR
BridgeAR requested a review from a team as a code owner June 26, 2026 09:10
@BridgeAR
BridgeAR requested review from crysmags and removed request for a team June 26, 2026 09:10

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f90d86f4dd

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread packages/datadog-plugin-child_process/test/index.spec.js
juan-fernandez
juan-fernandez previously approved these changes Jul 1, 2026
@BridgeAR
BridgeAR force-pushed the BridgeAR/2026-06-26-fix-child-process-bluebird-flake branch from f90d86f to 4174e9d Compare July 1, 2026 14:57
@datadog-prod-us1-3

datadog-prod-us1-3 Bot commented Jul 1, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 88.02% (-0.00%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 9e59a8f | Docs | Datadog PR Page | Give us feedback!

@BridgeAR
BridgeAR force-pushed the BridgeAR/2026-06-26-fix-child-process-bluebird-flake branch from 4174e9d to 2fe78d6 Compare July 1, 2026 15:34
…spec

The Bluebird specs set global.Promise = Bluebird in beforeEach and restored it
only in afterEach, holding the mutation across the awaited span round-trip. Under
load the subprocess span arrives late, and the shared assertion helper only
rejects once a payload has already errored, so the spec hangs to mocha's 5 s
timeout while leftover async keeps global.Promise flipped, corrupting context for
every later spec in the file.

1. The instrumentation only reads global.Promise synchronously, when the wrapped
   method runs. Scope the swap to that window via withBluebird() and restore it in
   a finally before any await.
2. assert.strictEqual(promise.constructor, Bluebird) asserts a property that
   depends on module-load order and the call-time global, not on any contract the
   instrumentation makes; it held locally and on Node 18 but saw the native
   constructor on Node 20. Assert the real contract instead: the promisified call
   resolves with the expected stdout and still produces the span.
3. Await the span expectation and the subprocess completion together in one
   Promise.all per spec, so a late or failed span cannot float onto the shared
   agent and match a later spec's expectation.

Drive-by fix:

* Drop the dead `delete require.cache[require.resolve('util')]` (util is a builtin).
* Replace the try/catch rejection check with assert.rejects.
@BridgeAR
BridgeAR force-pushed the BridgeAR/2026-06-26-fix-child-process-bluebird-flake branch from 2fe78d6 to 9e59a8f Compare July 1, 2026 15:39
@BridgeAR
BridgeAR merged commit 9e5845e into master Jul 2, 2026
862 of 863 checks passed
@BridgeAR
BridgeAR deleted the BridgeAR/2026-06-26-fix-child-process-bluebird-flake branch July 2, 2026 13:28
@dd-octo-sts dd-octo-sts Bot mentioned this pull request Jul 2, 2026
dd-octo-sts Bot pushed a commit that referenced this pull request Jul 2, 2026
…spec (#9078)

The Bluebird specs set global.Promise = Bluebird in beforeEach and restored it
only in afterEach, holding the mutation across the awaited span round-trip. Under
load the subprocess span arrives late, and the shared assertion helper only
rejects once a payload has already errored, so the spec hangs to mocha's 5 s
timeout while leftover async keeps global.Promise flipped, corrupting context for
every later spec in the file.

1. The instrumentation only reads global.Promise synchronously, when the wrapped
   method runs. Scope the swap to that window via withBluebird() and restore it in
   a finally before any await.
2. assert.strictEqual(promise.constructor, Bluebird) asserts a property that
   depends on module-load order and the call-time global, not on any contract the
   instrumentation makes; it held locally and on Node 18 but saw the native
   constructor on Node 20. Assert the real contract instead: the promisified call
   resolves with the expected stdout and still produces the span.
3. Await the span expectation and the subprocess completion together in one
   Promise.all per spec, so a late or failed span cannot float onto the shared
   agent and match a later spec's expectation.

Drive-by fix:

* Drop the dead `delete require.cache[require.resolve('util')]` (util is a builtin).
* Replace the try/catch rejection check with assert.rejects.
@dd-octo-sts dd-octo-sts Bot mentioned this pull request Jul 2, 2026
rochdev pushed a commit that referenced this pull request Jul 2, 2026
…spec (#9078)

The Bluebird specs set global.Promise = Bluebird in beforeEach and restored it
only in afterEach, holding the mutation across the awaited span round-trip. Under
load the subprocess span arrives late, and the shared assertion helper only
rejects once a payload has already errored, so the spec hangs to mocha's 5 s
timeout while leftover async keeps global.Promise flipped, corrupting context for
every later spec in the file.

1. The instrumentation only reads global.Promise synchronously, when the wrapped
   method runs. Scope the swap to that window via withBluebird() and restore it in
   a finally before any await.
2. assert.strictEqual(promise.constructor, Bluebird) asserts a property that
   depends on module-load order and the call-time global, not on any contract the
   instrumentation makes; it held locally and on Node 18 but saw the native
   constructor on Node 20. Assert the real contract instead: the promisified call
   resolves with the expected stdout and still produces the span.
3. Await the span expectation and the subprocess completion together in one
   Promise.all per spec, so a late or failed span cannot float onto the shared
   agent and match a later spec's expectation.

Drive-by fix:

* Drop the dead `delete require.cache[require.resolve('util')]` (util is a builtin).
* Replace the try/catch rejection check with assert.rejects.
rochdev pushed a commit that referenced this pull request Jul 2, 2026
…spec (#9078)

The Bluebird specs set global.Promise = Bluebird in beforeEach and restored it
only in afterEach, holding the mutation across the awaited span round-trip. Under
load the subprocess span arrives late, and the shared assertion helper only
rejects once a payload has already errored, so the spec hangs to mocha's 5 s
timeout while leftover async keeps global.Promise flipped, corrupting context for
every later spec in the file.

1. The instrumentation only reads global.Promise synchronously, when the wrapped
   method runs. Scope the swap to that window via withBluebird() and restore it in
   a finally before any await.
2. assert.strictEqual(promise.constructor, Bluebird) asserts a property that
   depends on module-load order and the call-time global, not on any contract the
   instrumentation makes; it held locally and on Node 18 but saw the native
   constructor on Node 20. Assert the real contract instead: the promisified call
   resolves with the expected stdout and still produces the span.
3. Await the span expectation and the subprocess completion together in one
   Promise.all per spec, so a late or failed span cannot float onto the shared
   agent and match a later spec's expectation.

Drive-by fix:

* Drop the dead `delete require.cache[require.resolve('util')]` (util is a builtin).
* Replace the try/catch rejection check with assert.rejects.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants