Skip to content

test: reject non-string version ranges in withVersions#9187

Merged
BridgeAR merged 1 commit into
masterfrom
BridgeAR/2026-07-02-withversions-range-validation
Jul 3, 2026
Merged

test: reject non-string version ranges in withVersions#9187
BridgeAR merged 1 commit into
masterfrom
BridgeAR/2026-07-02-withversions-range-validation

Conversation

@BridgeAR

@BridgeAR BridgeAR commented Jul 2, 2026

Copy link
Copy Markdown
Member

Summary

A Node-version gate written as withVersions('grpc', pkg, NODE_MAJOR >= 25 && '>=1.3.0', cb) collapses to false on older Node. The version filter's !range guard treated false (and every other nullish/empty value) identically to "no range", so the intended restriction silently vanished and the suite ran every installed version instead of the gated subset.

withVersions now throws a TypeError when a range argument is present but is not a non-empty string, so this class of misuse fails loudly at collection time instead of running the wrong set of versions. The undefined produced by the internal typeof range === 'function' reshuffle (the legitimate "no range" form) still passes through.

Callers that relied on the silent pass-through are updated to pass a real range:

  1. The grpc ESM integration spec used the && form; switched to a ternary yielding '*' on older Node, matching its sibling unit spec.
  2. The aws-sdk version helpers forwarded an omitted range as undefined straight into withVersions; they now forward '*'. semver.satisfies(v, '*') selects the same set the !range short-circuit did, so the tested versions are unchanged.

Test plan

  • packages/dd-trace/test/plugins/with-versions.spec.js pins both sides: the helper throws on false / '' / null / undefined explicit ranges, and lets the function form and an explicit '*' through to module resolution.

A Node-version gate written as `withVersions('grpc', pkg, NODE_MAJOR >= 25 &&
'>=1.3.0', cb)` collapses to `false` on older Node. The old `!range` guard in
the version filter treated `false` (and every other nullish/empty value)
identically to "no range", so the intended restriction silently vanished and
the suite ran every installed version instead of the gated subset.

withVersions now throws a TypeError when a range argument is present but not a
non-empty string, so the misuse fails loudly at collection time. Callers that
relied on the silent pass-through are updated to pass a real range:

1. The grpc ESM integration spec used the `&&` form; switched to a ternary
   yielding '*' on older Node, matching its sibling unit spec.
2. The aws-sdk version helpers forwarded an omitted range as `undefined`
   straight into withVersions; they now forward '*'. `semver.satisfies(v, '*')`
   selects the same set the `!range` short-circuit did, so the tested versions
   are unchanged.
@dd-octo-sts

dd-octo-sts Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Overall package size

Self size: 6.45 MB
Deduped: 7.52 MB
No deduping: 7.52 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.2.0 | 104.26 kB | 843.44 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Jul 2, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 88.05% (+0.01%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 2df6d45 | Docs | Datadog PR Page | Give us feedback!

@pr-commenter

pr-commenter Bot commented Jul 2, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-07-02 12:22:15

Comparing candidate commit 2df6d45 in PR branch BridgeAR/2026-07-02-withversions-range-validation with baseline commit 1bdd6fc in branch master.

📊 Benchmarking dashboard

Found 0 performance improvements and 0 performance regressions! Performance is the same for 2254 metrics, 32 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:appsec-appsec-enabled-20

  • unstable execution_time [-193.012ms; +187.400ms] or [-5.191%; +5.040%]

scenario:appsec-appsec-enabled-24

  • unstable execution_time [-173.670ms; +183.171ms] or [-6.580%; +6.940%]

scenario:appsec-appsec-enabled-26

  • unstable execution_time [-204513.052µs; +204966.386µs] or [-8.112%; +8.130%]

scenario:appsec-appsec-enabled-with-attacks-26

  • unstable execution_time [-154.028ms; +156.246ms] or [-5.354%; +5.431%]

scenario:appsec-control-20

  • unstable execution_time [-160132.528µs; +160088.695µs] or [-9.169%; +9.167%]

scenario:appsec-control-24

  • unstable execution_time [-99.342ms; +103.898ms] or [-8.166%; +8.540%]

scenario:appsec-control-26

  • unstable execution_time [-115376.731µs; +117112.031µs] or [-9.526%; +9.670%]

scenario:debugger-line-probe-with-snapshot-default-24

  • unstable cpu_user_time [-1799.542ms; +2973.085ms] or [-22.571%; +37.290%]
  • unstable execution_time [-1815.915ms; +3007.710ms] or [-20.986%; +34.759%]
  • unstable instructions [-15.2G instructions; +25.2G instructions] or [-23.565%; +38.988%]
  • unstable max_rss_usage [-7.168MB; +11.283MB] or [-4.534%; +7.136%]
  • unstable throughput [-888.857op/s; +521.482op/s] or [-23.330%; +13.687%]

scenario:debugger-line-probe-with-snapshot-minimal-26

  • unstable cpu_user_time [-3664.979ms; +2483.695ms] or [-34.760%; +23.556%]
  • unstable execution_time [-3702.328ms; +2479.169ms] or [-32.954%; +22.067%]
  • unstable instructions [-32.8G instructions; +21.9G instructions] or [-37.088%; +24.718%]
  • unstable max_rss_usage [-10.805MB; +6.790MB] or [-6.635%; +4.170%]
  • unstable throughput [-546.119op/s; +815.200op/s] or [-18.102%; +27.021%]

scenario:debugger-line-probe-without-snapshot-24

  • unstable cpu_user_time [-1723.445ms; +568.889ms] or [-21.669%; +7.153%]
  • unstable execution_time [-1757.201ms; +597.290ms] or [-20.339%; +6.913%]
  • unstable instructions [-14.7G instructions; +4.8G instructions] or [-22.805%; +7.374%]
  • unstable throughput [-175.825op/s; +498.298op/s] or [-4.590%; +13.008%]

scenario:debugger-line-probe-without-snapshot-26

  • unstable cpu_user_time [-2103.065ms; +3304.272ms] or [-22.368%; +35.143%]
  • unstable execution_time [-2099.760ms; +3339.505ms] or [-20.757%; +33.013%]
  • unstable instructions [-18.4G instructions; +29.4G instructions] or [-23.593%; +37.637%]
  • unstable max_rss_usage [-6.522MB; +11.332MB] or [-4.118%; +7.156%]
  • unstable throughput [-721.848op/s; +454.280op/s] or [-22.246%; +14.000%]

scenario:dogstatsd-with-tags-20

  • unstable cpu_user_time [-384.389ms; +311.836ms] or [-7.928%; +6.431%]
  • unstable execution_time [-383.173ms; +309.064ms] or [-7.781%; +6.276%]
  • unstable throughput [-106261.353op/s; +129767.388op/s] or [-6.231%; +7.609%]

scenario:llmobs-span-processor-agent-26

  • unstable cpu_user_time [-254.604ms; +187.165ms] or [-5.938%; +4.365%]
  • unstable execution_time [-264.668ms; +196.212ms] or [-5.951%; +4.411%]
  • unstable throughput [-10253.553op/s; +14412.494op/s] or [-4.763%; +6.695%]

@BridgeAR
BridgeAR marked this pull request as ready for review July 2, 2026 12:22
@BridgeAR
BridgeAR requested review from a team as code owners July 2, 2026 12:22
@BridgeAR
BridgeAR requested review from crysmags and removed request for a team July 2, 2026 12:22
@codecov

codecov Bot commented Jul 2, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.77%. Comparing base (1bdd6fc) to head (2df6d45).

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #9187   +/-   ##
=======================================
  Coverage   93.77%   93.77%           
=======================================
  Files         895      895           
  Lines       51538    51538           
  Branches    12020    12020           
=======================================
  Hits        48329    48329           
  Misses       3209     3209           
Flag Coverage Δ
aiguard 34.87% <ø> (-0.08%) ⬇️
aiguard-integration 41.79% <ø> (ø)
apm-bucket-0 34.84% <ø> (-0.08%) ⬇️
apm-bucket-1 40.33% <ø> (-0.07%) ⬇️
apm-bucket-2 37.34% <ø> (-0.07%) ⬇️
apm-capabilities-tracing 48.69% <ø> (ø)
apm-integrations-aerospike 33.12% <ø> (-0.08%) ⬇️
apm-integrations-confluentinc-kafka-javascript 40.02% <ø> (-0.08%) ⬇️
apm-integrations-couchbase 33.51% <ø> (-0.08%) ⬇️
apm-integrations-http 41.95% <ø> (-0.07%) ⬇️
apm-integrations-kafkajs 40.26% <ø> (-0.08%) ⬇️
apm-integrations-next 29.46% <ø> (-0.11%) ⬇️
apm-integrations-prisma 35.03% <ø> (-0.07%) ⬇️
apm-integrations-tedious 33.89% <ø> (-0.08%) ⬇️
appsec 57.69% <ø> (-0.05%) ⬇️
appsec-express_fastify_graphql 53.68% <ø> (-0.04%) ⬇️
appsec-integration 36.29% <ø> (ø)
appsec-kafka_ldapjs_lodash 43.58% <ø> (-0.07%) ⬇️
appsec-mongodb-core_mongoose_mysql 48.74% <ø> (-0.06%) ⬇️
appsec-next 28.00% <ø> (-0.07%) ⬇️
appsec-node-serialize_passport_postgres 47.96% <ø> (-0.02%) ⬇️
appsec-sourcing_stripe_template 45.47% <ø> (-0.06%) ⬇️
debugger 44.45% <ø> (-0.05%) ⬇️
instrumentations-bucket-0 28.15% <ø> (-0.08%) ⬇️
instrumentations-bucket-1 37.40% <ø> (-0.07%) ⬇️
instrumentations-bucket-10 40.35% <ø> (-0.07%) ⬇️
instrumentations-bucket-11 27.95% <ø> (-0.08%) ⬇️
instrumentations-bucket-12 28.67% <ø> (-0.08%) ⬇️
instrumentations-bucket-13 27.78% <ø> (-0.08%) ⬇️
instrumentations-bucket-2 30.23% <ø> (-0.07%) ⬇️
instrumentations-bucket-3 35.90% <ø> (-0.07%) ⬇️
instrumentations-bucket-4 28.56% <ø> (-0.08%) ⬇️
instrumentations-bucket-5 36.28% <ø> (-0.07%) ⬇️
instrumentations-bucket-6 38.25% <ø> (-0.08%) ⬇️
instrumentations-bucket-7 36.01% <ø> (-0.07%) ⬇️
instrumentations-bucket-8 36.95% <ø> (-0.07%) ⬇️
instrumentations-bucket-9 39.45% <ø> (-0.07%) ⬇️
instrumentations-instrumentation-couchbase 46.48% <ø> (ø)
instrumentations-integration-esbuild 24.94% <ø> (ø)
llmobs-ai_anthropic_bedrock 39.53% <ø> (-0.07%) ⬇️
llmobs-google-genai_langchain_vertex-ai 36.97% <ø> (-0.06%) ⬇️
llmobs-openai 39.58% <ø> (-0.07%) ⬇️
llmobs-sdk 43.59% <ø> (-0.08%) ⬇️
master-coverage 93.77% <ø> (?)
openfeature 37.76% <ø> (ø)
openfeature-unit 50.39% <ø> (ø)
platform-core_esbuild_instrumentations-misc 23.38% <ø> (ø)
platform-integration 47.44% <ø> (+0.01%) ⬆️
platform-shimmer_unit-guardrails_webpack 18.91% <ø> (ø)
plugins-bucket-0 36.38% <ø> (-0.07%) ⬇️
plugins-bucket-1 39.60% <ø> (ø)
plugins-bucket-11 38.46% <ø> (-0.08%) ⬇️
plugins-bucket-17 39.00% <ø> (-0.07%) ⬇️
plugins-bucket-18 42.01% <ø> (-0.07%) ⬇️
plugins-bucket-19 39.56% <ø> (-0.07%) ⬇️
plugins-bucket-20 43.20% <ø> (-0.15%) ⬇️
plugins-bucket-4 37.71% <ø> (-0.07%) ⬇️
plugins-bullmq_cassandra_cookie 39.76% <ø> (-0.07%) ⬇️
plugins-cookie-parser_crypto_dd-trace-api 33.22% <ø> (-0.08%) ⬇️
plugins-fetch_fs_generic-pool 36.05% <ø> (-0.07%) ⬇️
plugins-google-cloud-pubsub_grpc_handlebars 43.10% <ø> (-0.07%) ⬇️
plugins-hapi_hono_ioredis 37.78% <ø> (-0.07%) ⬇️
plugins-jest_knex_langgraph 32.60% <ø> (-0.08%) ⬇️
plugins-ldapjs_light-my-request_limitd-client 27.85% <ø> (-0.08%) ⬇️
plugins-lodash_mariadb_memcached 35.17% <ø> (-0.08%) ⬇️
plugins-mongodb_mongodb-core_mongoose 36.31% <ø> (-0.07%) ⬇️
plugins-multer_mysql_mysql2 35.14% <ø> (-0.08%) ⬇️
plugins-nats_node-serialize_opensearch 37.14% <ø> (-0.08%) ⬇️
plugins-passport-http_pino_postgres 35.53% <ø> (-0.08%) ⬇️
plugins-process_pug_redis 34.26% <ø> (-0.08%) ⬇️
plugins-undici_url_valkey 35.88% <ø> (-0.07%) ⬇️
plugins-vm_winston_ws 37.55% <ø> (-0.08%) ⬇️
profiling 43.62% <ø> (-0.07%) ⬇️
serverless-aws-sdk-aws-sdk 33.29% <ø> (-0.07%) ⬇️
serverless-aws-sdk-bedrockruntime 32.16% <ø> (-0.07%) ⬇️
serverless-aws-sdk-client 37.16% <ø> (ø)
serverless-aws-sdk-dynamodb 34.14% <ø> (-0.07%) ⬇️
serverless-aws-sdk-eventbridge 27.25% <ø> (-0.07%) ⬇️
serverless-aws-sdk-kinesis 37.40% <ø> (-0.07%) ⬇️
serverless-aws-sdk-lambda 34.59% <ø> (-0.07%) ⬇️
serverless-aws-sdk-s3 32.59% <ø> (-0.07%) ⬇️
serverless-aws-sdk-serverless-peer-service 39.47% <ø> (-0.08%) ⬇️
serverless-aws-sdk-sns 38.25% <ø> (-0.07%) ⬇️
serverless-aws-sdk-sqs 38.00% <ø> (-0.07%) ⬇️
serverless-aws-sdk-stepfunctions 33.18% <ø> (-0.07%) ⬇️
serverless-aws-sdk-util 47.95% <ø> (ø)
serverless-bucket-0 39.46% <ø> (ø)
serverless-lambda 34.33% <ø> (-0.09%) ⬇️
test-optimization-cucumber 52.30% <ø> (+0.11%) ⬆️
test-optimization-cypress 49.41% <ø> (+0.06%) ⬆️
test-optimization-jest 55.34% <ø> (+0.07%) ⬆️
test-optimization-mocha 53.38% <ø> (+0.08%) ⬆️
test-optimization-playwright-playwright-atr 43.33% <ø> (+0.08%) ⬆️
test-optimization-playwright-playwright-efd 43.61% <ø> (+0.08%) ⬆️
test-optimization-playwright-playwright-final-status 43.73% <ø> (+0.08%) ⬆️
test-optimization-playwright-playwright-impacted-tests 43.16% <ø> (ø)
test-optimization-playwright-playwright-reporting 43.26% <ø> (+0.08%) ⬆️
test-optimization-playwright-playwright-test-management 44.73% <ø> (+0.09%) ⬆️
test-optimization-playwright-playwright-test-span 44.51% <ø> (+0.07%) ⬆️
test-optimization-selenium 45.26% <ø> (+0.06%) ⬆️
test-optimization-testopt 47.78% <ø> (+0.08%) ⬆️
test-optimization-vitest 51.37% <ø> (+0.07%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

let variants

withVersions('grpc', '@grpc/grpc-js', NODE_MAJOR >= 25 && '>=1.3.0', version => {
withVersions('grpc', '@grpc/grpc-js', NODE_MAJOR >= 25 ? '>=1.3.0' : '*', version => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: For consistency with this change, we should apply the ternary form also here

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While I agree, I will keep it for less churn for now, since it does not change behavior.

@BridgeAR
BridgeAR merged commit 9919e40 into master Jul 3, 2026
651 checks passed
@BridgeAR
BridgeAR deleted the BridgeAR/2026-07-02-withversions-range-validation branch July 3, 2026 12:30
dd-octo-sts Bot pushed a commit that referenced this pull request Jul 6, 2026
A Node-version gate written as `withVersions('grpc', pkg, NODE_MAJOR >= 25 &&
'>=1.3.0', cb)` collapses to `false` on older Node. The old `!range` guard in
the version filter treated `false` (and every other nullish/empty value)
identically to "no range", so the intended restriction silently vanished and
the suite ran every installed version instead of the gated subset.

withVersions now throws a TypeError when a range argument is present but not a
non-empty string, so the misuse fails loudly at collection time. Callers that
relied on the silent pass-through are updated to pass a real range:

1. The grpc ESM integration spec used the `&&` form; switched to a ternary
   yielding '*' on older Node, matching its sibling unit spec.
2. The aws-sdk version helpers forwarded an omitted range as `undefined`
   straight into withVersions; they now forward '*'. `semver.satisfies(v, '*')`
   selects the same set the `!range` short-circuit did, so the tested versions
   are unchanged.
@dd-octo-sts dd-octo-sts Bot mentioned this pull request Jul 6, 2026
dd-octo-sts Bot pushed a commit that referenced this pull request Jul 6, 2026
A Node-version gate written as `withVersions('grpc', pkg, NODE_MAJOR >= 25 &&
'>=1.3.0', cb)` collapses to `false` on older Node. The old `!range` guard in
the version filter treated `false` (and every other nullish/empty value)
identically to "no range", so the intended restriction silently vanished and
the suite ran every installed version instead of the gated subset.

withVersions now throws a TypeError when a range argument is present but not a
non-empty string, so the misuse fails loudly at collection time. Callers that
relied on the silent pass-through are updated to pass a real range:

1. The grpc ESM integration spec used the `&&` form; switched to a ternary
   yielding '*' on older Node, matching its sibling unit spec.
2. The aws-sdk version helpers forwarded an omitted range as `undefined`
   straight into withVersions; they now forward '*'. `semver.satisfies(v, '*')`
   selects the same set the `!range` short-circuit did, so the tested versions
   are unchanged.
This was referenced Jul 6, 2026
juan-fernandez pushed a commit that referenced this pull request Jul 8, 2026
A Node-version gate written as `withVersions('grpc', pkg, NODE_MAJOR >= 25 &&
'>=1.3.0', cb)` collapses to `false` on older Node. The old `!range` guard in
the version filter treated `false` (and every other nullish/empty value)
identically to "no range", so the intended restriction silently vanished and
the suite ran every installed version instead of the gated subset.

withVersions now throws a TypeError when a range argument is present but not a
non-empty string, so the misuse fails loudly at collection time. Callers that
relied on the silent pass-through are updated to pass a real range:

1. The grpc ESM integration spec used the `&&` form; switched to a ternary
   yielding '*' on older Node, matching its sibling unit spec.
2. The aws-sdk version helpers forwarded an omitted range as `undefined`
   straight into withVersions; they now forward '*'. `semver.satisfies(v, '*')`
   selects the same set the `!range` short-circuit did, so the tested versions
   are unchanged.
juan-fernandez pushed a commit that referenced this pull request Jul 8, 2026
A Node-version gate written as `withVersions('grpc', pkg, NODE_MAJOR >= 25 &&
'>=1.3.0', cb)` collapses to `false` on older Node. The old `!range` guard in
the version filter treated `false` (and every other nullish/empty value)
identically to "no range", so the intended restriction silently vanished and
the suite ran every installed version instead of the gated subset.

withVersions now throws a TypeError when a range argument is present but not a
non-empty string, so the misuse fails loudly at collection time. Callers that
relied on the silent pass-through are updated to pass a real range:

1. The grpc ESM integration spec used the `&&` form; switched to a ternary
   yielding '*' on older Node, matching its sibling unit spec.
2. The aws-sdk version helpers forwarded an omitted range as `undefined`
   straight into withVersions; they now forward '*'. `semver.satisfies(v, '*')`
   selects the same set the `!range` short-circuit did, so the tested versions
   are unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants