Skip to content

perf(http): load the GCP pubsub push plugin lazily#9178

Merged
BridgeAR merged 1 commit into
masterfrom
BridgeAR/2026-07-01-http-lazy-pubsub-require
Jul 2, 2026
Merged

perf(http): load the GCP pubsub push plugin lazily#9178
BridgeAR merged 1 commit into
masterfrom
BridgeAR/2026-07-01-http-lazy-pubsub-require

Conversation

@BridgeAR

@BridgeAR BridgeAR commented Jul 1, 2026

Copy link
Copy Markdown
Member

Summary

The http plugin required datadog-plugin-google-cloud-pubsub's pubsub-push-subscription module at the top of packages/datadog-plugin-http/src/index.js. Because http is instrumented in almost every service, that pulled the pubsub plugin graph into the startup path of every process — even though the plugin is only used on GCP Cloud Run (K_SERVICE set and DD_TRACE_GCP_PUBSUB_PUSH_ENABLED not opted out).

Moving the require inside the existing enableGCPPubSubPushSubscription() gate keeps the module and its transitive graph out of startup for every non-GCP-Cloud-Run deployment.

Test plan

  • packages/datadog-plugin-http/test/index.spec.js — new: asserts the pubsub module is absent from require.cache after reading HttpPlugin.plugins with the GCP push gate off (fails on the eager-require shape), and that it loads first when the gate is on.
  • 100% line/branch coverage on the changed file; the sole defensive catch log is istanbul ignored.
  • Picked up by the existing http plugin CI job (PLUGINS: http, globs test/**/*.spec.js).

@dd-octo-sts

dd-octo-sts Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Overall package size

Self size: 6.44 MB
Deduped: 7.5 MB
No deduping: 7.5 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.2.0 | 104.26 kB | 843.44 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

@datadog-official

datadog-official Bot commented Jul 1, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 88.02% (-0.00%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 32db337 | Docs | Datadog PR Page | Give us feedback!

@pr-commenter

pr-commenter Bot commented Jul 1, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-07-01 16:42:22

Comparing candidate commit 32db337 in PR branch BridgeAR/2026-07-01-http-lazy-pubsub-require with baseline commit e035b64 in branch master.

📊 Benchmarking dashboard

Found 0 performance improvements and 0 performance regressions! Performance is the same for 2254 metrics, 32 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:appsec-appsec-enabled-20

  • unstable execution_time [-189757.723µs; +187854.356µs] or [-5.121%; +5.070%]

scenario:appsec-appsec-enabled-24

  • unstable execution_time [-183145.537µs; +181267.104µs] or [-6.971%; +6.899%]

scenario:appsec-appsec-enabled-26

  • unstable execution_time [-205264.992µs; +203399.558µs] or [-8.160%; +8.086%]

scenario:appsec-appsec-enabled-with-attacks-26

  • unstable execution_time [-166.403ms; +155.964ms] or [-5.780%; +5.417%]

scenario:appsec-control-20

  • unstable execution_time [-161.093ms; +167.854ms] or [-9.260%; +9.649%]

scenario:appsec-control-24

  • unstable execution_time [-110.964ms; +89.912ms] or [-9.214%; +7.466%]

scenario:appsec-control-26

  • unstable execution_time [-114.869ms; +111.242ms] or [-9.508%; +9.208%]

scenario:debugger-line-probe-with-snapshot-default-24

  • unstable cpu_user_time [-1972.198ms; +3127.661ms] or [-24.707%; +39.182%]
  • unstable execution_time [-2006.235ms; +3185.942ms] or [-23.180%; +36.811%]
  • unstable instructions [-16.9G instructions; +27.1G instructions] or [-26.182%; +41.878%]
  • unstable max_rss_usage [-6.274MB; +11.552MB] or [-3.952%; +7.276%]
  • unstable throughput [-910.471op/s; +574.619op/s] or [-23.825%; +15.036%]

scenario:debugger-line-probe-with-snapshot-default-26

  • unstable cpu_user_time [-1896.219ms; +541.259ms] or [-20.284%; +5.790%]
  • unstable execution_time [-1918.111ms; +551.431ms] or [-19.067%; +5.482%]
  • unstable instructions [-17.1G instructions; +4.8G instructions] or [-21.975%; +6.231%]
  • unstable throughput [-113.323op/s; +431.387op/s] or [-3.468%; +13.202%]

scenario:debugger-line-probe-with-snapshot-minimal-24

  • unstable cpu_user_time [-1730.954ms; +584.955ms] or [-21.777%; +7.359%]
  • unstable execution_time [-1748.853ms; +589.298ms] or [-20.291%; +6.837%]
  • unstable instructions [-14.6G instructions; +4.8G instructions] or [-22.652%; +7.450%]
  • unstable throughput [-173.664op/s; +498.336op/s] or [-4.523%; +12.978%]

scenario:debugger-line-probe-with-snapshot-minimal-26

  • unstable cpu_user_time [-2142.653ms; +3353.714ms] or [-23.056%; +36.087%]
  • unstable execution_time [-2172.123ms; +3372.188ms] or [-21.784%; +33.819%]
  • unstable instructions [-19.4G instructions; +29.7G instructions] or [-25.144%; +38.467%]
  • unstable throughput [-743.449op/s; +489.524op/s] or [-22.533%; +14.837%]

scenario:debugger-line-probe-without-snapshot-24

  • unstable cpu_user_time [-1738.772ms; +557.258ms] or [-21.830%; +6.996%]
  • unstable execution_time [-1749.883ms; +570.452ms] or [-20.249%; +6.601%]
  • unstable instructions [-14.7G instructions; +4.8G instructions] or [-22.798%; +7.377%]
  • unstable throughput [-165.415op/s; +503.290op/s] or [-4.322%; +13.151%]

scenario:dogstatsd-with-tags-20

  • unstable cpu_user_time [-259.286ms; +420.279ms] or [-5.344%; +8.662%]
  • unstable execution_time [-257.757ms; +423.158ms] or [-5.235%; +8.595%]
  • unstable throughput [-146003.894op/s; +90338.072op/s] or [-8.565%; +5.299%]

scenario:test-optimization-large-suite-20

  • unstable max_rss_usage [-3.395MB; +6.650MB] or [-4.275%; +8.374%]

@codecov

codecov Bot commented Jul 1, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.73%. Comparing base (e035b64) to head (32db337).

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #9178      +/-   ##
==========================================
- Coverage   93.74%   93.73%   -0.01%     
==========================================
  Files         895      895              
  Lines       51399    51396       -3     
  Branches    11980    11980              
==========================================
- Hits        48182    48178       -4     
- Misses       3217     3218       +1     
Flag Coverage Δ
aiguard 34.87% <ø> (-0.08%) ⬇️
aiguard-integration 41.96% <0.00%> (+0.16%) ⬆️
apm-bucket-0 34.84% <ø> (-0.08%) ⬇️
apm-bucket-1 40.51% <0.00%> (+0.10%) ⬆️
apm-bucket-2 37.50% <0.00%> (+0.09%) ⬆️
apm-capabilities-tracing 48.79% <0.00%> (+0.09%) ⬆️
apm-integrations-aerospike 33.12% <ø> (-0.08%) ⬇️
apm-integrations-confluentinc-kafka-javascript 40.02% <ø> (-0.08%) ⬇️
apm-integrations-couchbase 33.51% <ø> (+0.03%) ⬆️
apm-integrations-http 42.05% <100.00%> (+0.02%) ⬆️
apm-integrations-kafkajs 40.30% <ø> (+0.02%) ⬆️
apm-integrations-next 29.58% <0.00%> (+<0.01%) ⬆️
apm-integrations-prisma 35.17% <0.00%> (+0.07%) ⬆️
apm-integrations-tedious 34.04% <0.00%> (+0.07%) ⬆️
appsec 57.84% <0.00%> (+0.08%) ⬆️
appsec-express_fastify_graphql 53.87% <0.00%> (+0.17%) ⬆️
appsec-integration 36.52% <0.00%> (+0.07%) ⬆️
appsec-kafka_ldapjs_lodash 43.84% <0.00%> (+0.19%) ⬆️
appsec-mongodb-core_mongoose_mysql 48.92% <0.00%> (+0.12%) ⬆️
appsec-next 28.10% <0.00%> (+0.03%) ⬆️
appsec-node-serialize_passport_postgres 48.09% <0.00%> (+0.12%) ⬆️
appsec-sourcing_stripe_template 45.64% <0.00%> (+0.11%) ⬆️
debugger 44.32% <0.00%> (-0.06%) ⬇️
instrumentations-bucket-0 28.15% <ø> (-0.08%) ⬇️
instrumentations-bucket-1 37.56% <0.00%> (+0.09%) ⬆️
instrumentations-bucket-10 40.51% <0.00%> (+0.09%) ⬆️
instrumentations-bucket-11 27.95% <ø> (-0.08%) ⬇️
instrumentations-bucket-12 28.78% <0.00%> (+0.04%) ⬆️
instrumentations-bucket-13 27.78% <ø> (-0.08%) ⬇️
instrumentations-bucket-2 30.35% <0.00%> (+0.05%) ⬆️
instrumentations-bucket-3 36.06% <0.00%> (+0.08%) ⬆️
instrumentations-bucket-4 28.56% <ø> (-0.08%) ⬇️
instrumentations-bucket-5 36.42% <0.00%> (+0.07%) ⬆️
instrumentations-bucket-6 38.42% <0.00%> (+0.09%) ⬆️
instrumentations-bucket-7 36.15% <0.00%> (+0.16%) ⬆️
instrumentations-bucket-8 37.11% <0.00%> (+0.08%) ⬆️
instrumentations-bucket-9 39.61% <0.00%> (+0.09%) ⬆️
instrumentations-instrumentation-couchbase 46.48% <ø> (ø)
instrumentations-integration-esbuild 25.10% <0.00%> (+0.04%) ⬆️
llmobs-ai_anthropic_bedrock 39.68% <0.00%> (+0.09%) ⬆️
llmobs-google-genai_langchain_vertex-ai 37.10% <0.00%> (+0.07%) ⬆️
llmobs-openai 39.74% <0.00%> (+0.09%) ⬆️
llmobs-sdk 43.59% <ø> (-0.08%) ⬇️
master-coverage 93.73% <100.00%> (?)
openfeature 37.92% <0.00%> (+0.15%) ⬆️
openfeature-unit 50.39% <ø> (ø)
platform-core_esbuild_instrumentations-misc 23.54% <0.00%> (+0.04%) ⬆️
platform-integration 47.62% <0.00%> (+0.19%) ⬆️
platform-shimmer_unit-guardrails_webpack 19.06% <0.00%> (+0.03%) ⬆️
plugins-bucket-0 36.52% <0.00%> (+0.07%) ⬆️
plugins-bucket-1 39.77% <0.00%> (+0.17%) ⬆️
plugins-bucket-11 38.63% <0.00%> (+0.09%) ⬆️
plugins-bucket-17 39.32% <0.00%> (+0.10%) ⬆️
plugins-bucket-18 42.18% <0.00%> (+0.10%) ⬆️
plugins-bucket-19 39.73% <0.00%> (+0.10%) ⬆️
plugins-bucket-20 43.45% <0.00%> (+0.18%) ⬆️
plugins-bucket-4 37.87% <0.00%> (+0.09%) ⬆️
plugins-bullmq_cassandra_cookie 39.94% <0.00%> (+0.10%) ⬆️
plugins-cookie-parser_crypto_dd-trace-api 33.22% <ø> (-0.08%) ⬇️
plugins-fetch_fs_generic-pool 35.90% <0.00%> (-0.22%) ⬇️
plugins-google-cloud-pubsub_grpc_handlebars 43.09% <100.00%> (-0.08%) ⬇️
plugins-hapi_hono_ioredis 37.94% <0.00%> (+0.09%) ⬆️
plugins-jest_knex_langgraph 32.60% <ø> (-0.08%) ⬇️
plugins-ldapjs_light-my-request_limitd-client 27.85% <ø> (-0.14%) ⬇️
plugins-lodash_mariadb_memcached 35.17% <ø> (-0.08%) ⬇️
plugins-mongodb_mongodb-core_mongoose 36.46% <0.00%> (+0.08%) ⬆️
plugins-multer_mysql_mysql2 35.14% <ø> (-0.08%) ⬇️
plugins-nats_node-serialize_opensearch 37.31% <0.00%> (+0.09%) ⬆️
plugins-passport-http_pino_postgres 35.53% <ø> (-0.08%) ⬇️
plugins-process_pug_redis 34.26% <ø> (-0.08%) ⬇️
plugins-undici_url_valkey 36.03% <0.00%> (+0.08%) ⬆️
plugins-vm_winston_ws 37.72% <0.00%> (+0.09%) ⬆️
profiling 43.80% <0.00%> (+0.11%) ⬆️
serverless-aws-sdk-aws-sdk 33.41% <0.00%> (+0.05%) ⬆️
serverless-aws-sdk-bedrockruntime 32.28% <0.00%> (+0.05%) ⬆️
serverless-aws-sdk-client 37.16% <ø> (ø)
serverless-aws-sdk-dynamodb 34.28% <0.00%> (+0.06%) ⬆️
serverless-aws-sdk-eventbridge 27.34% <0.00%> (+0.03%) ⬆️
serverless-aws-sdk-kinesis 37.55% <0.00%> (+0.08%) ⬆️
serverless-aws-sdk-lambda 34.73% <0.00%> (+0.06%) ⬆️
serverless-aws-sdk-s3 32.72% <0.00%> (+0.05%) ⬆️
serverless-aws-sdk-serverless-peer-service 39.66% <0.00%> (+0.11%) ⬆️
serverless-aws-sdk-sns 38.41% <0.00%> (+0.08%) ⬆️
serverless-aws-sdk-sqs 38.15% <0.00%> (+0.08%) ⬆️
serverless-aws-sdk-stepfunctions 33.31% <0.00%> (+0.06%) ⬆️
serverless-aws-sdk-util 47.95% <ø> (ø)
serverless-bucket-0 39.63% <0.00%> (+0.16%) ⬆️
serverless-lambda 34.33% <ø> (-0.09%) ⬇️
test-optimization-cucumber 52.52% <0.00%> (+0.33%) ⬆️
test-optimization-cypress 49.73% <0.00%> (+0.38%) ⬆️
test-optimization-jest 55.59% <0.00%> (+0.29%) ⬆️
test-optimization-mocha 53.63% <0.00%> (+0.32%) ⬆️
test-optimization-playwright-playwright-atr 43.50% <0.00%> (+0.26%) ⬆️
test-optimization-playwright-playwright-efd 43.79% <0.00%> (+0.26%) ⬆️
test-optimization-playwright-playwright-final-status 43.91% <0.00%> (+0.26%) ⬆️
test-optimization-playwright-playwright-impacted-tests 43.33% <0.00%> (+0.17%) ⬆️
test-optimization-playwright-playwright-reporting 43.44% <0.00%> (+0.26%) ⬆️
test-optimization-playwright-playwright-test-management 44.90% <0.00%> (+0.27%) ⬆️
test-optimization-playwright-playwright-test-span 44.69% <0.00%> (+0.25%) ⬆️
test-optimization-selenium 45.42% <0.00%> (+0.22%) ⬆️
test-optimization-testopt 48.16% <0.00%> (+0.23%) ⬆️
test-optimization-vitest 51.21% <0.00%> (+0.31%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

The http plugin requires datadog-plugin-google-cloud-pubsub's
pubsub-push-subscription module at the top of its own file, so every process
that instruments http pulls in that plugin graph even though it is only used
on GCP Cloud Run (K_SERVICE set and DD_TRACE_GCP_PUBSUB_PUSH_ENABLED not
opted out). Moving the require inside the existing gate keeps the module — and
its transitive graph — out of the startup path for every other deployment.

Drive-by fix:

* Drop the try/catch around the require: the pubsub plugin ships with the
  tracer, so this require cannot fail independently of the tracer's own load.
@BridgeAR
BridgeAR force-pushed the BridgeAR/2026-07-01-http-lazy-pubsub-require branch from 71b92f1 to 32db337 Compare July 1, 2026 16:30
@BridgeAR
BridgeAR marked this pull request as ready for review July 1, 2026 17:01
@BridgeAR
BridgeAR requested a review from a team as a code owner July 1, 2026 17:01
@BridgeAR
BridgeAR requested review from crysmags and removed request for a team July 1, 2026 17:01
@juan-fernandez

Copy link
Copy Markdown
Collaborator

As a side effect the surrounding try/catch now wraps a require that can actually throw, instead of a bare property assignment that never could.

but there is no try catch now, is there?

@BridgeAR

BridgeAR commented Jul 2, 2026

Copy link
Copy Markdown
Member Author

@juan-fernandez outdated description :)

@BridgeAR
BridgeAR merged commit c693e22 into master Jul 2, 2026
919 of 920 checks passed
@BridgeAR
BridgeAR deleted the BridgeAR/2026-07-01-http-lazy-pubsub-require branch July 2, 2026 13:28
@dd-octo-sts dd-octo-sts Bot mentioned this pull request Jul 2, 2026
dd-octo-sts Bot pushed a commit that referenced this pull request Jul 2, 2026
The http plugin requires datadog-plugin-google-cloud-pubsub's
pubsub-push-subscription module at the top of its own file, so every process
that instruments http pulls in that plugin graph even though it is only used
on GCP Cloud Run (K_SERVICE set and DD_TRACE_GCP_PUBSUB_PUSH_ENABLED not
opted out). Moving the require inside the existing gate keeps the module — and
its transitive graph — out of the startup path for every other deployment.

Drive-by fix:

* Drop the try/catch around the require: the pubsub plugin ships with the
  tracer, so this require cannot fail independently of the tracer's own load.
@dd-octo-sts dd-octo-sts Bot mentioned this pull request Jul 2, 2026
rochdev pushed a commit that referenced this pull request Jul 2, 2026
The http plugin requires datadog-plugin-google-cloud-pubsub's
pubsub-push-subscription module at the top of its own file, so every process
that instruments http pulls in that plugin graph even though it is only used
on GCP Cloud Run (K_SERVICE set and DD_TRACE_GCP_PUBSUB_PUSH_ENABLED not
opted out). Moving the require inside the existing gate keeps the module — and
its transitive graph — out of the startup path for every other deployment.

Drive-by fix:

* Drop the try/catch around the require: the pubsub plugin ships with the
  tracer, so this require cannot fail independently of the tracer's own load.
rochdev pushed a commit that referenced this pull request Jul 2, 2026
The http plugin requires datadog-plugin-google-cloud-pubsub's
pubsub-push-subscription module at the top of its own file, so every process
that instruments http pulls in that plugin graph even though it is only used
on GCP Cloud Run (K_SERVICE set and DD_TRACE_GCP_PUBSUB_PUSH_ENABLED not
opted out). Moving the require inside the existing gate keeps the module — and
its transitive graph — out of the startup path for every other deployment.

Drive-by fix:

* Drop the try/catch around the require: the pubsub plugin ships with the
  tracer, so this require cannot fail independently of the tracer's own load.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants