fix(test-optimization): stabilize failed test replay waits by juan-fernandez · Pull Request #9193 · DataDog/dd-trace-js

juan-fernandez · 2026-07-02T15:24:58Z

What does this PR do?

Stabilizes Failed Test Replay across Jest, Vitest, Mocha, and Cucumber by replacing fixed grace-period waits and busy waits with DI-aware async waits. The shared CI plugin now prepares a bounded wait for the next breakpoint hit, can drain in-flight DI worker hits, and cancels unused waits when tests finish.

For Mocha, the retry hook wrapper is installed lazily only when Failed Test Replay and test dynamic instrumentation are enabled, so non-FTR runs avoid the extra wrapper path.

For Cucumber, Failed Test Replay DI waits are supported only when the runner exposes an awaited retry boundary. Older runners that only expose the envelope fallback intentionally skip DI probe setup instead of reintroducing a synchronous wait; this shortens FTR coverage for that Cucumber path and is an accepted tradeoff.

Motivation

The old implementation relied on fixed delays after failed attempts and, in some frameworks, synchronous busy waits before retries. That made replay timing race-prone: a breakpoint hit could arrive after the retried test span had already finished, losing the snapshot.

dd-octo-sts · 2026-07-02T15:25:55Z

Overall package size

Self size: 6.59 MB
Deduped: 7.24 MB
No deduping: 7.24 MB

Dependency sizes

| name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.3.0 | 117.14 kB | 432.47 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |

_{🤖 This report was automatically generated by heaviest-objects-in-the-universe}

datadog-datadog-prod-us1-2 · 2026-07-02T15:32:52Z

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🔄 Datadog auto-retried 2 jobs - 0 passed on retry

🎯 Code Coverage (details)
• Patch Coverage: 78.39%
• Overall Coverage: 87.60% (-0.11%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: ef99102 | Docs | Datadog PR Page | Give us feedback!}

pr-commenter · 2026-07-02T15:34:13Z

Benchmarks

Benchmark execution time: 2026-07-07 08:05:11

Comparing candidate commit ef99102 in PR branch juan-fernandez/ftr-di-replay-waits with baseline commit 37a5069 in branch master.

📊 Benchmarking dashboard

Found 0 performance improvements and 0 performance regressions! Performance is the same for 2243 metrics, 43 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

🟩 = significantly better candidate vs. baseline
🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:appsec-appsec-enabled-24

unstable execution_time [-180.683ms; +184.502ms] or [-6.907%; +7.053%]

scenario:appsec-appsec-enabled-26

unstable execution_time [-206.011ms; +214.105ms] or [-8.150%; +8.470%]

scenario:appsec-appsec-enabled-with-attacks-26

unstable execution_time [-161.635ms; +156.591ms] or [-5.611%; +5.435%]

scenario:appsec-control-20

unstable execution_time [-94.878ms; +117.659ms] or [-5.827%; +7.227%]

scenario:appsec-control-24

unstable execution_time [-91.853ms; +109.368ms] or [-7.634%; +9.090%]

scenario:appsec-control-26

unstable execution_time [-114.665ms; +118.358ms] or [-9.468%; +9.773%]

scenario:debugger-line-probe-with-snapshot-default-24

unstable cpu_user_time [-2513.868ms; +2052.650ms] or [-27.812%; +22.709%]
unstable execution_time [-2524.270ms; +2064.688ms] or [-25.939%; +21.216%]
unstable instructions [-22.9G instructions; +17.8G instructions] or [-30.852%; +23.920%]
unstable max_rss_usage [-9845.250KB; +9290.050KB] or [-6.180%; +5.832%]
unstable throughput [-557.523op/s; +755.473op/s] or [-16.409%; +22.235%]

scenario:debugger-line-probe-with-snapshot-default-26

unstable cpu_user_time [-3726.946ms; +2431.712ms] or [-35.261%; +23.007%]
unstable execution_time [-3733.023ms; +2425.235ms] or [-33.111%; +21.511%]
unstable instructions [-33.2G instructions; +21.5G instructions] or [-37.417%; +24.232%]
unstable max_rss_usage [-11.212MB; +7.986MB] or [-6.884%; +4.903%]
unstable throughput [-528.987op/s; +828.602op/s] or [-17.617%; +27.595%]

scenario:debugger-line-probe-with-snapshot-minimal-24

unstable cpu_user_time [-2548.240ms; +3024.409ms] or [-26.361%; +31.287%]
unstable execution_time [-2580.002ms; +3080.790ms] or [-24.900%; +29.733%]
unstable instructions [-22.1G instructions; +26.0G instructions] or [-27.817%; +32.646%]
unstable max_rss_usage [-10.434MB; +13.769MB] or [-6.469%; +8.537%]
unstable throughput [-876.983op/s; +669.807op/s] or [-26.841%; +20.500%]

scenario:debugger-line-probe-with-snapshot-minimal-26

unstable cpu_user_time [-1874.526ms; +619.580ms] or [-20.269%; +6.700%]
unstable execution_time [-1893.143ms; +633.112ms] or [-19.064%; +6.375%]
unstable instructions [-16.8G instructions; +5.2G instructions] or [-21.811%; +6.684%]
unstable throughput [-148.469op/s; +429.237op/s] or [-4.479%; +12.949%]

scenario:debugger-line-probe-without-snapshot-24

unstable cpu_user_time [-1376.769ms; +2924.186ms] or [-16.930%; +35.957%]
unstable execution_time [-1465.470ms; +3024.970ms] or [-16.612%; +34.290%]
unstable instructions [-11.6G instructions; +24.9G instructions] or [-17.474%; +37.519%]
unstable max_rss_usage [-6.891MB; +9.285MB] or [-4.378%; +5.899%]
unstable throughput [-919.603op/s; +383.213op/s] or [-24.639%; +10.267%]

scenario:debugger-line-probe-without-snapshot-26

unstable cpu_user_time [-3930.936ms; +2457.799ms] or [-36.856%; +23.044%]
unstable execution_time [-4004.272ms; +2535.926ms] or [-35.222%; +22.306%]
unstable instructions [-34.8G instructions; +21.6G instructions] or [-38.795%; +24.040%]
unstable max_rss_usage [-11.688MB; +8.037MB] or [-7.158%; +4.922%]
unstable throughput [-556.544op/s; +859.278op/s] or [-18.516%; +28.587%]

scenario:dogstatsd-with-tags-20

unstable cpu_user_time [-338.273ms; +381.348ms] or [-6.786%; +7.650%]
unstable execution_time [-336.011ms; +378.617ms] or [-6.640%; +7.482%]
unstable throughput [-117935.921op/s; +118482.904op/s] or [-7.080%; +7.113%]

scenario:log-with-debug-20

unstable max_rss_usage [-8.042MB; +5.389MB] or [-6.970%; +4.670%]

scenario:plugin-graphql-long-with-depth-off-20

unstable max_rss_usage [-6583.917KB; +7694.203KB] or [-5.131%; +5.996%]

scenario:plugin-graphql-long-with-depth-on-max-20

unstable cpu_user_time [-666.017ms; +635.569ms] or [-5.260%; +5.020%]
unstable execution_time [-667.813ms; +636.053ms] or [-5.162%; +4.917%]

scenario:test-optimization-large-suite-20

unstable max_rss_usage [-6.217MB; +2.979MB] or [-7.745%; +3.711%]

codecov · 2026-07-02T15:38:55Z

Codecov Report

❌ Patch coverage is 88.97243% with 44 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.48%. Comparing base (37a5069) to head (ef99102).

Files with missing lines	Patch %	Lines
...visibility/dynamic-instrumentation/worker/index.js	79.31%	12 Missing ⚠️
packages/datadog-plugin-vitest/src/index.js	84.84%	5 Missing ⚠️
packages/datadog-instrumentations/src/cucumber.js	93.22%	4 Missing ⚠️
...kages/datadog-instrumentations/src/mocha/worker.js	50.00%	4 Missing ⚠️
...ages/datadog-instrumentations/src/vitest-worker.js	63.63%	4 Missing ⚠️
packages/datadog-plugin-mocha/src/index.js	83.33%	4 Missing ⚠️
packages/dd-trace/src/plugins/ci_plugin.js	92.59%	4 Missing ⚠️
...ckages/datadog-instrumentations/src/mocha/utils.js	95.00%	3 Missing ⚠️
.../ci-visibility/exporters/ci-visibility-exporter.js	90.47%	2 Missing ⚠️
...ackages/datadog-instrumentations/src/mocha/main.js	93.33%	1 Missing ⚠️
... and 1 more

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #9193      +/-   ##
==========================================
- Coverage   93.55%   93.48%   -0.08%     
==========================================
  Files         900      901       +1     
  Lines       52941    53307     +366     
  Branches    12491    12581      +90     
==========================================
+ Hits        49531    49834     +303     
- Misses       3410     3473      +63

Flag	Coverage Δ
aiguard	`34.88% <14.70%> (-0.17%)`	⬇️
aiguard-integration	`41.86% <14.70%> (-0.10%)`	⬇️
apm-bucket-0	`36.23% <14.70%> (+1.26%)`	⬆️
apm-bucket-1	`41.08% <14.70%> (+0.51%)`	⬆️
apm-bucket-2	`37.77% <14.70%> (+0.21%)`	⬆️
apm-capabilities-tracing	`49.01% <47.71%> (+0.07%)`	⬆️
apm-integrations-aerospike	`33.03% <14.70%> (-0.16%)`	⬇️
apm-integrations-confluentinc-kafka-javascript	`39.90% <14.70%> (-0.19%)`	⬇️
apm-integrations-couchbase	`33.15% <14.70%> (-0.32%)`	⬇️
apm-integrations-http	`41.93% <14.70%> (-0.18%)`	⬇️
apm-integrations-kafkajs	`40.13% <14.70%> (-0.20%)`	⬇️
apm-integrations-next	`29.55% <14.70%> (-0.14%)`	⬇️
apm-integrations-prisma	`35.08% <14.70%> (-0.16%)`	⬇️
apm-integrations-tedious	`33.95% <14.70%> (-0.16%)`	⬇️
appsec	`57.75% <14.70%> (-0.12%)`	⬇️
appsec-express_fastify_graphql	`53.74% <14.70%> (-0.21%)`	⬇️
appsec-integration	`35.20% <6.76%> (-0.15%)`	⬇️
appsec-kafka_ldapjs_lodash	`43.63% <14.70%> (-0.18%)`	⬇️
appsec-mongodb-core_mongoose_mysql	`48.77% <14.70%> (-0.17%)`	⬇️
appsec-next	`27.61% <13.88%> (-0.12%)`	⬇️
appsec-node-serialize_passport_postgres	`47.96% <14.70%> (-0.17%)`	⬇️
appsec-sourcing_stripe_template	`45.52% <14.70%> (-0.17%)`	⬇️
debugger	`44.54% <14.70%> (-0.07%)`	⬇️
instrumentations-bucket-0	`28.09% <14.70%> (-0.14%)`	⬇️
instrumentations-bucket-1	`37.49% <14.70%> (-0.17%)`	⬇️
instrumentations-bucket-10	`40.40% <14.70%> (-0.17%)`	⬇️
instrumentations-bucket-11	`27.89% <14.70%> (-0.14%)`	⬇️
instrumentations-bucket-12	`28.72% <14.70%> (-0.14%)`	⬇️
instrumentations-bucket-13	`27.72% <14.70%> (-0.14%)`	⬇️
instrumentations-bucket-2	`30.28% <14.70%> (-0.14%)`	⬇️
instrumentations-bucket-3	`35.96% <14.70%> (-0.16%)`	⬇️
instrumentations-bucket-4	`28.52% <14.70%> (-0.14%)`	⬇️
instrumentations-bucket-5	`36.19% <8.39%> (-0.28%)`	⬇️
instrumentations-bucket-6	`38.31% <14.70%> (-0.18%)`	⬇️
instrumentations-bucket-7	`36.05% <14.70%> (-0.16%)`	⬇️
instrumentations-bucket-8	`37.00% <14.70%> (-0.16%)`	⬇️
instrumentations-bucket-9	`39.50% <14.70%> (-0.17%)`	⬇️
instrumentations-instrumentation-couchbase	`46.48% <ø> (ø)`
instrumentations-integration-esbuild	`24.19% <6.76%> (-0.10%)`	⬇️
llmobs-ai_anthropic_bedrock	`39.58% <14.70%> (-0.16%)`	⬇️
llmobs-google-genai_langchain_vertex-ai	`36.89% <14.70%> (-0.13%)`	⬇️
llmobs-openai	`39.63% <14.70%> (-0.17%)`	⬇️
llmobs-sdk	`43.44% <14.70%> (-0.21%)`	⬇️
master-coverage	`93.48% <88.97%> (?)`
openfeature	`37.83% <14.70%> (-0.10%)`	⬇️
openfeature-unit	`50.47% <ø> (ø)`
platform-core_esbuild_instrumentations-misc	`22.63% <6.76%> (-0.10%)`	⬇️
platform-integration	`47.43% <14.70%> (-0.13%)`	⬇️
platform-shimmer_unit-guardrails_webpack	`18.18% <2.87%> (-0.08%)`	⬇️
plugins-bucket-0	`36.42% <14.70%> (-0.15%)`	⬇️
plugins-bucket-1	`39.66% <14.70%> (-0.11%)`	⬇️
plugins-bucket-11	`38.51% <14.70%> (-0.18%)`	⬇️
plugins-bucket-17	`39.10% <14.70%> (-0.18%)`	⬇️
plugins-bucket-18	`42.05% <14.70%> (-0.18%)`	⬇️
plugins-bucket-19	`39.85% <14.70%> (-0.18%)`	⬇️
plugins-bucket-20	`43.30% <14.70%> (-0.19%)`	⬇️
plugins-bucket-4	`37.78% <14.70%> (-0.17%)`	⬇️
plugins-bullmq_cassandra_cookie	`39.81% <14.70%> (-0.18%)`	⬇️
plugins-cookie-parser_crypto_dd-trace-api	`33.13% <14.70%> (-0.16%)`	⬇️
plugins-fetch_fs_generic-pool	`36.10% <14.70%> (-0.17%)`	⬇️
plugins-google-cloud-pubsub_grpc_handlebars	`42.95% <14.70%> (-0.19%)`	⬇️
plugins-hapi_hono_ioredis	`37.83% <14.70%> (-0.17%)`	⬇️
plugins-jest_knex_langgraph	`32.52% <14.70%> (-0.15%)`	⬇️
plugins-ldapjs_light-my-request_limitd-client	`27.85% <14.70%> (-0.08%)`	⬇️
plugins-lodash_mariadb_memcached	`35.07% <14.70%> (-0.17%)`	⬇️
plugins-mongodb_mongodb-core_mongoose	`36.36% <14.70%> (-0.17%)`	⬇️
plugins-multer_mysql_mysql2	`35.05% <14.70%> (-0.17%)`	⬇️
plugins-nats_node-serialize_opensearch	`37.20% <14.70%> (-0.17%)`	⬇️
plugins-passport-http_pino_postgres	`35.42% <14.70%> (-0.17%)`	⬇️
plugins-process_pug_redis	`34.16% <14.70%> (-0.17%)`	⬇️
plugins-undici_url_valkey	`35.93% <14.70%> (-0.15%)`	⬇️
plugins-vm_winston_ws	`37.60% <14.70%> (-0.17%)`	⬇️
profiling	`43.70% <14.70%> (-0.18%)`	⬇️
serverless-aws-sdk-aws-sdk	`33.13% <8.39%> (-0.25%)`	⬇️
serverless-aws-sdk-bedrockruntime	`32.11% <14.70%> (-0.14%)`	⬇️
serverless-aws-sdk-client	`37.16% <ø> (ø)`
serverless-aws-sdk-dynamodb	`34.10% <14.70%> (-0.15%)`	⬇️
serverless-aws-sdk-eventbridge	`27.20% <8.39%> (-0.22%)`	⬇️
serverless-aws-sdk-kinesis	`37.44% <14.70%> (-0.16%)`	⬇️
serverless-aws-sdk-lambda	`34.55% <14.70%> (?)`
serverless-aws-sdk-s3	`32.55% <14.70%> (-0.14%)`	⬇️
serverless-aws-sdk-serverless-peer-service	`39.61% <14.70%> (-0.10%)`	⬇️
serverless-aws-sdk-sns	`38.30% <14.70%> (-0.16%)`	⬇️
serverless-aws-sdk-sqs	`38.04% <14.70%> (-0.16%)`	⬇️
serverless-aws-sdk-stepfunctions	`33.13% <14.70%> (-0.14%)`	⬇️
serverless-aws-sdk-util	`47.95% <ø> (ø)`
serverless-azure-functions-servicebus_lambda	`?`
serverless-bucket-0	`39.52% <14.70%> (+0.88%)`	⬆️
serverless-lambda	`34.22% <14.70%> (?)`
test-optimization-cucumber	`52.54% <80.86%> (+0.25%)`	⬆️
test-optimization-cypress	`49.86% <14.60%> (-0.11%)`	⬇️
test-optimization-jest	`56.10% <91.37%> (+0.24%)`	⬆️
test-optimization-mocha	`53.78% <87.75%> (+0.38%)`	⬆️
test-optimization-playwright-playwright-atr	`43.28% <14.60%> (-0.29%)`	⬇️
test-optimization-playwright-playwright-efd	`43.56% <14.60%> (-0.06%)`	⬇️
test-optimization-playwright-playwright-final-status	`43.68% <14.60%> (-0.06%)`	⬇️
test-optimization-playwright-playwright-impacted-tests	`43.11% <14.60%> (-0.14%)`	⬇️
test-optimization-playwright-playwright-reporting	`43.22% <14.60%> (-0.06%)`	⬇️
test-optimization-playwright-playwright-test-management	`44.66% <14.60%> (-0.28%)`	⬇️
test-optimization-playwright-playwright-test-span	`44.46% <14.60%> (-0.07%)`	⬇️
test-optimization-selenium	`45.07% <23.95%> (-0.21%)`	⬇️
test-optimization-testopt	`46.35% <18.91%> (-0.24%)`	⬇️
test-optimization-vitest	`52.77% <73.88%> (+0.21%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

juan-fernandez · 2026-07-02T15:43:44Z

@codex review

juan-fernandez · 2026-07-02T15:43:49Z

/datadog autotest review

datadog-datadog-prod-us1-2

More details

All 16 adversarial scenarios across the new DI-wait promise lifecycle (prepareDiBreakpointHitWait/cancelDiBreakpointHitWait/waitForDiBreakpointHits) and the worker drain mechanism passed. The mocha hookUp shimmer's missing return is intentional — Mocha's retry progression is callback-driven, not return-value-driven, so deferring the completion callback via .then() correctly pauses the test lifecycle until the probe is set. The worker drain ordering guarantee (snapshot in try, drain response in finally, FIFO MessageChannel) ensures all snapshots are attached before spans are closed.

Was this helpful? React 👍 or 👎

_{📊 Validated against 16 scenarios · Open Bits AI session}

_{🤖 Datadog Autotest · Commit 60949d1 · What is Autotest? · Any feedback? Reach out in #autotest}

juan-fernandez · 2026-07-02T21:38:59Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 60949d19ad

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

juan-fernandez · 2026-07-02T21:50:07Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3f24425add

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

juan-fernandez · 2026-07-03T06:08:11Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ff294e5915

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

juan-fernandez · 2026-07-03T06:22:31Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fef5e93cae

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

juan-fernandez · 2026-07-03T08:28:32Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fb5eadf1db

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

juan-fernandez · 2026-07-03T08:42:32Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4193f27355

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

juan-fernandez · 2026-07-03T09:07:11Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b08c469095

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

juan-fernandez · 2026-07-06T14:04:10Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b4b0922819

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

juan-fernandez · 2026-07-06T14:26:34Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f1e9c05c03

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f1e9c05c03

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

juan-fernandez · 2026-07-06T15:54:26Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2bb152f434

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

juan-fernandez · 2026-07-06T19:49:46Z

@codex review

chatgpt-codex-connector · 2026-07-06T20:07:12Z

Codex Review: Didn't find any major issues. What shall we delve into next?

Reviewed commit: c9fb5c982f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

juan-fernandez · 2026-07-06T21:35:55Z

@codex review

chatgpt-codex-connector · 2026-07-06T21:42:44Z

Codex Review: Didn't find any major issues. Another round soon, please!

Reviewed commit: 63c2b0c2c5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

datadog-datadog-prod-us1-2

More details

All 13 scenarios executed against real code paths — drain mechanism, withTimeout timeout/cancel/resolve ordering, prepareDiBreakpointHitWait multi-call state cleanup, removeEmptyCollectionProperties depth, and the full waitForPreparedDiBreakpointHit/cancel/timeout cycle. No behavioral regression found. The replacement of fixed setTimeout grace periods with DI-aware async waits is mechanically sound: the snapshot message always arrives on the channel before the drain acknowledgment (same breakpointHitChannel, ordered delivery), so finish() is never called before the snapshot is attached to the active test span.

Was this helpful? React 👍 or 👎

_{📊 Validated against 13 scenarios · Open Bits AI session}

_{🤖 Datadog Autotest · Commit 63c2b0c · What is Autotest? · Any feedback? Reach out in #autotest}

github-actions Bot added semver-patch test-optimization labels Jul 2, 2026

datadog-datadog-prod-us1-2 Bot reviewed Jul 2, 2026

View reviewed changes

chatgpt-codex-connector Bot reviewed Jul 2, 2026

View reviewed changes

Comment thread packages/datadog-instrumentations/src/cucumber.js Outdated

juan-fernandez force-pushed the juan-fernandez/ftr-di-replay-waits branch from 60949d1 to 3f24425 Compare July 2, 2026 21:49

chatgpt-codex-connector Bot reviewed Jul 2, 2026

View reviewed changes

Comment thread packages/datadog-instrumentations/src/mocha/main.js Outdated

juan-fernandez force-pushed the juan-fernandez/ftr-di-replay-waits branch from 3f24425 to ff294e5 Compare July 3, 2026 06:07

chatgpt-codex-connector Bot reviewed Jul 3, 2026

View reviewed changes

Comment thread packages/datadog-instrumentations/src/mocha/utils.js Outdated

juan-fernandez force-pushed the juan-fernandez/ftr-di-replay-waits branch from ff294e5 to fef5e93 Compare July 3, 2026 06:21

chatgpt-codex-connector Bot reviewed Jul 3, 2026

View reviewed changes

Comment thread packages/datadog-instrumentations/src/cucumber.js Outdated

juan-fernandez force-pushed the juan-fernandez/ftr-di-replay-waits branch from fef5e93 to fb5eadf Compare July 3, 2026 08:12

chatgpt-codex-connector Bot reviewed Jul 3, 2026

View reviewed changes

Comment thread packages/datadog-plugin-cucumber/src/index.js Outdated

juan-fernandez force-pushed the juan-fernandez/ftr-di-replay-waits branch from fb5eadf to 4193f27 Compare July 3, 2026 08:41

chatgpt-codex-connector Bot reviewed Jul 3, 2026

View reviewed changes

Comment thread packages/dd-trace/src/plugins/ci_plugin.js Outdated

juan-fernandez force-pushed the juan-fernandez/ftr-di-replay-waits branch from 4193f27 to b08c469 Compare July 3, 2026 09:06

chatgpt-codex-connector Bot reviewed Jul 3, 2026

View reviewed changes

Comment thread packages/datadog-plugin-cucumber/src/index.js Outdated

juan-fernandez force-pushed the juan-fernandez/ftr-di-replay-waits branch from b08c469 to 58d705b Compare July 3, 2026 09:20

chatgpt-codex-connector Bot reviewed Jul 6, 2026

View reviewed changes

Comment thread packages/datadog-plugin-cucumber/src/index.js

Comment thread packages/dd-trace/src/ci-visibility/dynamic-instrumentation/worker/index.js

Comment thread packages/dd-trace/src/ci-visibility/dynamic-instrumentation/worker/index.js Outdated

fix(test-optimization): defer DI worker drain acknowledgements

f1e9c05

chatgpt-codex-connector Bot reviewed Jul 6, 2026

View reviewed changes

Comment thread packages/datadog-plugin-cucumber/src/index.js

Comment thread packages/dd-trace/src/plugins/ci_plugin.js

test(test-optimization): move cucumber FTR wait coverage to integration

314431a

chatgpt-codex-connector Bot reviewed Jul 6, 2026

View reviewed changes

Comment thread packages/dd-trace/src/ci-visibility/dynamic-instrumentation/worker/index.js

Comment thread packages/datadog-plugin-jest/src/index.js

Comment thread packages/datadog-plugin-vitest/src/index.js

Comment thread packages/datadog-plugin-cucumber/src/index.js

juan-fernandez added 2 commits July 6, 2026 17:34

fix(test-optimization): preserve FTR process tags

1ca442b

fix(test-optimization): rearm FTR waits across retries

2bb152f

chatgpt-codex-connector Bot reviewed Jul 6, 2026

View reviewed changes

Comment thread packages/datadog-instrumentations/src/mocha/worker.js

Comment thread packages/datadog-instrumentations/src/cucumber.js

docs(test-optimization): note cucumber FTR retry boundary

c9fb5c9

fix(test-optimization): propagate vitest worker metadata

63c2b0c

juan-fernandez marked this pull request as ready for review July 7, 2026 06:21

datadog-datadog-prod-us1-2 Bot reviewed Jul 7, 2026

View reviewed changes

Merge origin/master into juan-fernandez/ftr-di-replay-waits

ef99102

anmarchenko approved these changes Jul 7, 2026

View reviewed changes

juan-fernandez merged commit 2fec129 into master Jul 7, 2026
1171 of 1179 checks passed

juan-fernandez deleted the juan-fernandez/ftr-di-replay-waits branch July 7, 2026 10:06

dd-octo-sts Bot pushed a commit that referenced this pull request Jul 7, 2026

fix(test-optimization): stabilize failed test replay waits (#9193)

158f361

dd-octo-sts Bot mentioned this pull request Jul 7, 2026

v6.1.0 proposal #9220

Merged

dd-octo-sts Bot pushed a commit that referenced this pull request Jul 7, 2026

fix(test-optimization): stabilize failed test replay waits (#9193)

5546912

dd-octo-sts Bot mentioned this pull request Jul 7, 2026

v5.112.0 proposal #9221

Merged

juan-fernandez added a commit that referenced this pull request Jul 8, 2026

fix(test-optimization): stabilize failed test replay waits (#9193)

4fd1a66

juan-fernandez added a commit that referenced this pull request Jul 8, 2026

fix(test-optimization): stabilize failed test replay waits (#9193)

0dd62f3

juan-fernandez mentioned this pull request Jul 8, 2026

fix(vitest): stop propagating metadata to workers #9261

Merged

Uh oh!

Conversation

juan-fernandez commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Motivation

Uh oh!

dd-octo-sts Bot commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overall package size

Uh oh!

datadog-datadog-prod-us1-2 Bot commented Jul 2, 2026 • edited by datadog-official Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter Bot commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Explanation

More details about the CI and significant changes

Unstable benchmarks

scenario:appsec-appsec-enabled-24

scenario:appsec-appsec-enabled-26

scenario:appsec-appsec-enabled-with-attacks-26

scenario:appsec-control-20

scenario:appsec-control-24

scenario:appsec-control-26

scenario:debugger-line-probe-with-snapshot-default-24

scenario:debugger-line-probe-with-snapshot-default-26

scenario:debugger-line-probe-with-snapshot-minimal-24

scenario:debugger-line-probe-with-snapshot-minimal-26

scenario:debugger-line-probe-without-snapshot-24

scenario:debugger-line-probe-without-snapshot-26

scenario:dogstatsd-with-tags-20

scenario:log-with-debug-20

scenario:plugin-graphql-long-with-depth-off-20

scenario:plugin-graphql-long-with-depth-on-max-20

scenario:test-optimization-large-suite-20

Uh oh!

codecov Bot commented Jul 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

juan-fernandez commented Jul 2, 2026

Uh oh!

juan-fernandez commented Jul 2, 2026

Uh oh!

datadog-datadog-prod-us1-2 Bot left a comment

Choose a reason for hiding this comment

Uh oh!

juan-fernandez commented Jul 2, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

juan-fernandez commented Jul 2, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

juan-fernandez commented Jul 3, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

juan-fernandez commented Jul 3, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

juan-fernandez commented Jul 3, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

juan-fernandez commented Jul 2, 2026 •

edited

Loading

dd-octo-sts Bot commented Jul 2, 2026 •

edited

Loading

datadog-datadog-prod-us1-2 Bot commented Jul 2, 2026 •

edited by datadog-official Bot

Loading

pr-commenter Bot commented Jul 2, 2026 •

edited

Loading

codecov Bot commented Jul 2, 2026 •

edited

Loading