Skip to content

fix(child_process): stabilize promisified exec spans#9180

Merged
BridgeAR merged 3 commits into
masterfrom
apm-ai-toolkit/execute/child-process/20260701-143026
Jul 6, 2026
Merged

fix(child_process): stabilize promisified exec spans#9180
BridgeAR merged 3 commits into
masterfrom
apm-ai-toolkit/execute/child-process/20260701-143026

Conversation

@crysmags

@crysmags crysmags commented Jul 1, 2026

Copy link
Copy Markdown
Collaborator

What

Stabilizes the child_process plugin tests and fixes issues found in the promisified execFile path.

Root Cause / Reproduction Evidence

This came from the execute workflow run for child_process on branch apm-ai-toolkit/execute/child-process/20260701-143026.

Source signal: https://github.com/DataDog/dd-trace-js/actions/runs/28529738893/job/84575757777

The workflow reproduced with the repo-defined command shape:

unset OTEL_TRACES_EXPORTER OTEL_LOGS_EXPORTER OTEL_METRICS_EXPORTER
PLUGINS=child_process npm run test:plugins:ci

The first reproduced failure was environmental: _DD_APM_TRACING_AGENTLESS_ENABLED leaked from the instrumented shell into plugin tests. That forced spans through the agentless exporter instead of the mock test agent, so expectSomeSpan() tests timed out even though spans were created.

After that was fixed, one real order-dependent failure remained: the Bluebird concurrent test created 5 echo spans but did not drain them. The following rejection test could consume one of those stale success spans instead of its own failing node -invalidFlag span.

The run also found two production-code issues:

  • wrapChildProcessCustomPromisifyMethod shadowed the diagnostic error channel with catch (error), masking synchronous throws.
  • asyncEnd() stringified the promisified { stdout, stderr } result as cmd.exit_code: "[object Object]".

Changes

  • Scrub _DD_APM_TRACING_AGENTLESS_ENABLED in the plugin test agent harness.
  • Avoid shadowing the child_process diagnostic error channel.
  • Set cmd.exit_code correctly for promisified execFile success/error paths.
  • Drain all spans produced by the concurrent Bluebird test.
  • Add regression coverage for the synchronous custom-promisify throw path.

Validation

Execute workflow final result:

  • 88 passing, 0 failing
  • Post-simplify test passed
  • Final Review approved

Generated with APM Instrumentation Toolkit.

@crysmags crysmags added semver-patch ai-generated PR created with AI assistance apm-integration-toolkit PR Generated by APM AI Integration Toolkit labels Jul 1, 2026
@dd-octo-sts

dd-octo-sts Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Overall package size

Self size: 6.52 MB
Deduped: 7.59 MB
No deduping: 7.59 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.2.0 | 104.26 kB | 843.44 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

@datadog-prod-us1-6

datadog-prod-us1-6 Bot commented Jul 1, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 87.86% (+0.00%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 9905af5 | Docs | Datadog PR Page | Give us feedback!

@codecov

codecov Bot commented Jul 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.64%. Comparing base (9e5845e) to head (9905af5).
⚠️ Report is 3 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #9180      +/-   ##
==========================================
+ Coverage   93.63%   93.64%   +0.01%     
==========================================
  Files         896      898       +2     
  Lines       52220    52371     +151     
  Branches    12265    12329      +64     
==========================================
+ Hits        48895    49044     +149     
- Misses       3325     3327       +2     
Flag Coverage Δ
aiguard 34.86% <0.00%> (-0.08%) ⬇️
aiguard-integration 41.95% <0.00%> (-0.01%) ⬇️
apm-bucket-0 34.89% <100.00%> (-0.02%) ⬇️
apm-bucket-1 40.49% <0.00%> (-0.09%) ⬇️
apm-bucket-2 37.49% <0.00%> (-0.08%) ⬇️
apm-capabilities-tracing 48.88% <0.00%> (+0.09%) ⬆️
apm-integrations-aerospike 33.11% <0.00%> (-0.08%) ⬇️
apm-integrations-confluentinc-kafka-javascript 40.02% <0.00%> (-0.08%) ⬇️
apm-integrations-couchbase 33.40% <0.00%> (+0.08%) ⬆️
apm-integrations-http 42.04% <0.00%> (-0.08%) ⬇️
apm-integrations-kafkajs 40.25% <0.00%> (-0.12%) ⬇️
apm-integrations-next 29.62% <0.00%> (-0.07%) ⬇️
apm-integrations-prisma 35.17% <0.00%> (-0.08%) ⬇️
apm-integrations-tedious 34.03% <0.00%> (-0.08%) ⬇️
appsec 57.81% <0.00%> (-0.16%) ⬇️
appsec-express_fastify_graphql 53.87% <100.00%> (-0.05%) ⬇️
appsec-integration 35.68% <0.00%> (+<0.01%) ⬆️
appsec-kafka_ldapjs_lodash 43.74% <0.00%> (-0.08%) ⬇️
appsec-mongodb-core_mongoose_mysql 48.90% <0.00%> (-0.08%) ⬇️
appsec-next 28.10% <0.00%> (-0.07%) ⬇️
appsec-node-serialize_passport_postgres 48.07% <0.00%> (-0.08%) ⬇️
appsec-sourcing_stripe_template 45.62% <0.00%> (-0.08%) ⬇️
debugger 44.62% <0.00%> (+0.28%) ⬆️
instrumentations-bucket-0 28.15% <0.00%> (-0.07%) ⬇️
instrumentations-bucket-1 37.59% <90.90%> (-0.04%) ⬇️
instrumentations-bucket-10 40.50% <0.00%> (-0.08%) ⬇️
instrumentations-bucket-11 27.95% <0.00%> (-0.08%) ⬇️
instrumentations-bucket-12 28.78% <0.00%> (-0.07%) ⬇️
instrumentations-bucket-13 27.78% <0.00%> (-0.08%) ⬇️
instrumentations-bucket-2 30.35% <0.00%> (-0.07%) ⬇️
instrumentations-bucket-3 36.05% <0.00%> (-0.08%) ⬇️
instrumentations-bucket-4 28.56% <0.00%> (-0.08%) ⬇️
instrumentations-bucket-5 36.41% <0.00%> (-0.07%) ⬇️
instrumentations-bucket-6 38.41% <0.00%> (-0.09%) ⬇️
instrumentations-bucket-7 36.15% <0.00%> (-0.08%) ⬇️
instrumentations-bucket-8 37.10% <0.00%> (-0.08%) ⬇️
instrumentations-bucket-9 39.60% <0.00%> (-0.08%) ⬇️
instrumentations-instrumentation-couchbase 46.48% <ø> (ø)
instrumentations-integration-esbuild 24.50% <0.00%> (+<0.01%) ⬆️
llmobs-ai_anthropic_bedrock 39.67% <0.00%> (-0.07%) ⬇️
llmobs-google-genai_langchain_vertex-ai 37.09% <0.00%> (-0.07%) ⬇️
llmobs-openai 39.73% <0.00%> (-0.08%) ⬇️
llmobs-sdk 43.57% <0.00%> (-0.09%) ⬇️
master-coverage 93.64% <100.00%> (?)
openfeature 37.92% <0.00%> (+<0.01%) ⬆️
openfeature-unit 50.47% <ø> (+0.07%) ⬆️
platform-core_esbuild_instrumentations-misc 22.96% <0.00%> (+<0.01%) ⬆️
platform-integration 47.57% <0.00%> (-0.05%) ⬇️
platform-shimmer_unit-guardrails_webpack 18.48% <0.00%> (-0.01%) ⬇️
plugins-bucket-0 36.51% <0.00%> (-0.07%) ⬇️
plugins-bucket-1 39.77% <0.00%> (+<0.01%) ⬆️
plugins-bucket-11 38.61% <0.00%> (-0.08%) ⬇️
plugins-bucket-17 39.21% <0.00%> (-0.19%) ⬇️
plugins-bucket-18 42.16% <0.00%> (-0.09%) ⬇️
plugins-bucket-19 39.72% <0.00%> (-0.08%) ⬇️
plugins-bucket-20 43.42% <0.00%> (-0.09%) ⬇️
plugins-bucket-4 37.86% <0.00%> (-0.08%) ⬇️
plugins-bullmq_cassandra_cookie 39.92% <0.00%> (-0.09%) ⬇️
plugins-cookie-parser_crypto_dd-trace-api 33.21% <0.00%> (-0.08%) ⬇️
plugins-fetch_fs_generic-pool 36.20% <0.00%> (-0.08%) ⬇️
plugins-google-cloud-pubsub_grpc_handlebars 43.08% <0.00%> (-0.09%) ⬇️
plugins-hapi_hono_ioredis 37.93% <0.00%> (-0.08%) ⬇️
plugins-jest_knex_langgraph 32.60% <0.00%> (-0.08%) ⬇️
plugins-ldapjs_light-my-request_limitd-client 27.85% <0.00%> (-0.08%) ⬇️
plugins-lodash_mariadb_memcached 35.16% <0.00%> (-0.08%) ⬇️
plugins-mongodb_mongodb-core_mongoose 36.45% <0.00%> (-0.08%) ⬇️
plugins-multer_mysql_mysql2 35.14% <0.00%> (-0.08%) ⬇️
plugins-nats_node-serialize_opensearch 37.30% <0.00%> (-0.08%) ⬇️
plugins-passport-http_pino_postgres 35.52% <0.00%> (-0.08%) ⬇️
plugins-process_pug_redis 34.25% <0.00%> (-0.08%) ⬇️
plugins-undici_url_valkey 36.02% <0.00%> (-0.08%) ⬇️
plugins-vm_winston_ws 37.71% <0.00%> (-0.08%) ⬇️
profiling 43.80% <0.00%> (-0.07%) ⬇️
serverless-aws-sdk-aws-sdk 33.31% <0.00%> (-0.17%) ⬇️
serverless-aws-sdk-bedrockruntime 32.18% <0.00%> (-0.17%) ⬇️
serverless-aws-sdk-client 37.16% <ø> (ø)
serverless-aws-sdk-dynamodb 34.12% <0.00%> (-0.17%) ⬇️
serverless-aws-sdk-eventbridge 27.35% <0.00%> (-0.06%) ⬇️
serverless-aws-sdk-kinesis 37.54% <0.00%> (-0.08%) ⬇️
serverless-aws-sdk-lambda 34.63% <0.00%> (-0.17%) ⬇️
serverless-aws-sdk-s3 32.62% <0.00%> (-0.16%) ⬇️
serverless-aws-sdk-serverless-peer-service 39.64% <0.00%> (-0.10%) ⬇️
serverless-aws-sdk-sns 38.39% <0.00%> (-0.08%) ⬇️
serverless-aws-sdk-sqs 38.13% <0.00%> (-0.25%) ⬇️
serverless-aws-sdk-stepfunctions 33.21% <0.00%> (-0.17%) ⬇️
serverless-aws-sdk-util 47.95% <ø> (ø)
serverless-bucket-0 39.62% <0.00%> (+<0.01%) ⬆️
serverless-lambda 34.33% <0.00%> (-0.10%) ⬇️
test-optimization-cucumber 52.52% <0.00%> (+0.10%) ⬆️
test-optimization-cypress 49.61% <0.00%> (+0.06%) ⬆️
test-optimization-jest 55.61% <54.54%> (+0.18%) ⬆️
test-optimization-mocha 53.60% <0.00%> (+0.06%) ⬆️
test-optimization-playwright-playwright-atr 43.51% <0.00%> (+0.08%) ⬆️
test-optimization-playwright-playwright-efd 43.80% <0.00%> (+0.08%) ⬆️
test-optimization-playwright-playwright-final-status 43.92% <0.00%> (+0.08%) ⬆️
test-optimization-playwright-playwright-impacted-tests 43.34% <0.00%> (-0.01%) ⬇️
test-optimization-playwright-playwright-reporting 43.45% <0.00%> (+0.08%) ⬆️
test-optimization-playwright-playwright-test-management 44.92% <0.00%> (+0.08%) ⬆️
test-optimization-playwright-playwright-test-span 44.70% <0.00%> (+0.07%) ⬆️
test-optimization-selenium 45.43% <0.00%> (+0.06%) ⬆️
test-optimization-testopt 46.81% <0.00%> (+0.08%) ⬆️
test-optimization-vitest 52.69% <0.00%> (+0.08%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pr-commenter

pr-commenter Bot commented Jul 1, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-07-02 17:56:18

Comparing candidate commit 9905af5 in PR branch apm-ai-toolkit/execute/child-process/20260701-143026 with baseline commit 9e5845e in branch master.

📊 Benchmarking dashboard

Found 0 performance improvements and 0 performance regressions! Performance is the same for 2272 metrics, 14 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:appsec-appsec-enabled-24

  • unstable execution_time [-181.596ms; +184.632ms] or [-6.926%; +7.041%]

scenario:appsec-appsec-enabled-26

  • unstable execution_time [-201.152ms; +208.421ms] or [-8.011%; +8.300%]

scenario:appsec-appsec-enabled-with-attacks-26

  • unstable execution_time [-156.134ms; +152.216ms] or [-5.434%; +5.297%]

scenario:appsec-control-20

  • unstable execution_time [-159.980ms; +157.974ms] or [-9.157%; +9.042%]

scenario:appsec-control-24

  • unstable execution_time [-111.487ms; +94.039ms] or [-9.238%; +7.793%]

scenario:appsec-control-26

  • unstable execution_time [-116253.358µs; +117422.491µs] or [-9.574%; +9.670%]

scenario:debugger-line-probe-with-snapshot-minimal-26

  • unstable cpu_user_time [-2.902s; +0.345s] or [-29.202%; +3.475%]
  • unstable execution_time [-2.920s; +0.354s] or [-27.488%; +3.333%]
  • unstable instructions [-26.1G instructions; +3.0G instructions] or [-31.392%; +3.607%]
  • unstable throughput [-74.754op/s; +653.141op/s] or [-2.373%; +20.733%]

scenario:dogstatsd-with-tags-20

  • unstable cpu_user_time [-433.662ms; +259.967ms] or [-9.029%; +5.413%]
  • unstable execution_time [-430.372ms; +261.565ms] or [-8.818%; +5.359%]
  • unstable throughput [-89727.152op/s; +147158.876op/s] or [-5.214%; +8.551%]

scenario:plugin-graphql-long-with-depth-and-collapse-off-20

  • unstable max_rss_usage [-32.177MB; +22.689MB] or [-8.459%; +5.964%]

@crysmags
crysmags marked this pull request as ready for review July 1, 2026 20:32
@crysmags
crysmags requested a review from a team as a code owner July 1, 2026 20:32
@crysmags
crysmags requested review from wconti27 and removed request for a team July 1, 2026 20:32
@crysmags
crysmags force-pushed the apm-ai-toolkit/execute/child-process/20260701-143026 branch from d69cf0d to e58552e Compare July 2, 2026 13:45

@BridgeAR BridgeAR left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great!

This definitely fixes small issues. Mind adding new tests for the asyncEnd changes with the exit code? I believe we are missing a few for that.

context.error = err
error.publish(context)
throw error
throw err

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice this fixes a real error!

Could we just rename the error channel to errorChannel and keep error here? That way it reads easier AFAIC

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

much cleaner, updated in ba1dff7. kept catch (error) and renamed the channel binding to errorChannel.

Comment thread packages/dd-trace/test/plugins/agent.js Outdated
// `_DD_APM_TRACING_AGENTLESS_ENABLED`, which switches the tracer to the
// agentless intake exporter, so spans never reach this agent and every
// span assertion times out. Force the agent exporter for plugin tests.
delete process.env._DD_APM_TRACING_AGENTLESS_ENABLED

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this would never match all theoretical parts that could leak. If we find one, we have to fix it in place :)

So removing this again seems best

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very true, removed this again in ba1dff7.

// success, where the exit code is 0.
exitCode = result.status ?? 0
} else if (result === undefined && error !== undefined) {
exitCode = error.status ?? error.code

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
exitCode = error.status ?? error.code
exitCode = error.status ?? error.code ?? 0

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, updated to error.status ?? error.code ?? 0 in ba1dff7 and added the asyncEnd unit coverage for status/code/no-code.

@crysmags

crysmags commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator Author

@BridgeAR added the asyncEnd unit coverage in ba1dff7: promisified success object, rejection status/code/no-code. Full child_process spec passes locally for me: 81 passing.

@crysmags
crysmags requested a review from a team as a code owner July 2, 2026 17:45

@BridgeAR BridgeAR left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@BridgeAR
BridgeAR merged commit 678ddd9 into master Jul 6, 2026
695 of 696 checks passed
@BridgeAR
BridgeAR deleted the apm-ai-toolkit/execute/child-process/20260701-143026 branch July 6, 2026 18:29
@dd-octo-sts dd-octo-sts Bot mentioned this pull request Jul 7, 2026
@dd-octo-sts dd-octo-sts Bot mentioned this pull request Jul 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai-generated PR created with AI assistance apm-integration-toolkit PR Generated by APM AI Integration Toolkit semver-patch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants