Skip to content

fix(vitest): report typecheck tests#9176

Merged
juan-fernandez merged 5 commits into
masterfrom
codex/fix-vitest-typecheck-reporting
Jul 2, 2026
Merged

fix(vitest): report typecheck tests#9176
juan-fernandez merged 5 commits into
masterfrom
codex/fix-vitest-typecheck-reporting

Conversation

@juan-fernandez

@juan-fernandez juan-fernandez commented Jul 1, 2026

Copy link
Copy Markdown
Collaborator

What does this PR do?

Adds Vitest typecheck reporting for Test Optimization by wrapping Typechecker.prepareResults in the main Vitest instrumentation and emitting suite/test events for .test-d.ts tasks.

The reporting path supports Vitest 3.2.x and 4.x typecheck flows, preserves nested test names, reports typecheck failures with TypeCheckError metadata, and applies Known Tests, Impacted Tests, and Test Management metadata to typecheck tasks.

Motivation

Vitest typecheck-only runs do not execute through the normal @vitest/runner worker path. They can report session/module events while missing per-suite and per-test events, which makes typecheck commands look unsupported in Test Optimization.

@dd-octo-sts

dd-octo-sts Bot commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Overall package size

Self size: 6.45 MB
Deduped: 7.52 MB
No deduping: 7.52 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | import-in-the-middle | 3.2.0 | 104.26 kB | 843.44 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | dc-polyfill | 0.1.11 | 25.74 kB | 25.74 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

@juan-fernandez juan-fernandez changed the title [codex] fix Vitest typecheck reporting fix(vitest): report typecheck tests Jul 1, 2026
@datadog-datadog-prod-us1-2

datadog-datadog-prod-us1-2 Bot commented Jul 1, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 87.05%
Overall Coverage: 88.05% (-0.01%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 9e5c56e | Docs | Datadog PR Page | Give us feedback!

@codecov

codecov Bot commented Jul 1, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.84173% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.77%. Comparing base (e035b64) to head (9e5c56e).
⚠️ Report is 5 commits behind head on master.

Files with missing lines Patch % Lines
...ckages/datadog-instrumentations/src/vitest-main.js 97.84% 3 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #9176      +/-   ##
==========================================
+ Coverage   93.74%   93.77%   +0.03%     
==========================================
  Files         895      895              
  Lines       51399    51538     +139     
  Branches    11980    12020      +40     
==========================================
+ Hits        48182    48329     +147     
+ Misses       3217     3209       -8     
Flag Coverage Δ
aiguard 34.87% <ø> (-0.08%) ⬇️
aiguard-integration 41.79% <ø> (ø)
apm-bucket-0 34.84% <ø> (-0.08%) ⬇️
apm-bucket-1 40.33% <ø> (-0.07%) ⬇️
apm-bucket-2 37.34% <ø> (-0.07%) ⬇️
apm-capabilities-tracing 48.69% <ø> (ø)
apm-integrations-aerospike 33.12% <ø> (-0.08%) ⬇️
apm-integrations-confluentinc-kafka-javascript 40.02% <ø> (-0.08%) ⬇️
apm-integrations-couchbase 33.24% <ø> (-0.24%) ⬇️
apm-integrations-http 41.95% <ø> (-0.08%) ⬇️
apm-integrations-kafkajs 40.26% <ø> (-0.02%) ⬇️
apm-integrations-next 29.50% <ø> (-0.07%) ⬇️
apm-integrations-prisma 35.03% <ø> (-0.07%) ⬇️
apm-integrations-tedious 33.89% <ø> (-0.08%) ⬇️
appsec 57.69% <ø> (-0.07%) ⬇️
appsec-express_fastify_graphql 53.68% <ø> (-0.02%) ⬇️
appsec-integration 36.29% <1.43%> (-0.16%) ⬇️
appsec-kafka_ldapjs_lodash 43.58% <ø> (-0.07%) ⬇️
appsec-mongodb-core_mongoose_mysql 48.74% <ø> (-0.06%) ⬇️
appsec-next 28.00% <ø> (-0.07%) ⬇️
appsec-node-serialize_passport_postgres 47.91% <ø> (-0.06%) ⬇️
appsec-sourcing_stripe_template 45.47% <ø> (-0.06%) ⬇️
debugger 44.49% <ø> (+0.12%) ⬆️
instrumentations-bucket-0 28.15% <ø> (-0.08%) ⬇️
instrumentations-bucket-1 37.40% <ø> (-0.07%) ⬇️
instrumentations-bucket-10 40.35% <ø> (-0.07%) ⬇️
instrumentations-bucket-11 27.95% <ø> (-0.08%) ⬇️
instrumentations-bucket-12 28.67% <ø> (-0.08%) ⬇️
instrumentations-bucket-13 27.78% <ø> (-0.08%) ⬇️
instrumentations-bucket-2 30.23% <ø> (-0.07%) ⬇️
instrumentations-bucket-3 35.90% <ø> (-0.07%) ⬇️
instrumentations-bucket-4 28.56% <ø> (-0.08%) ⬇️
instrumentations-bucket-5 36.28% <ø> (-0.07%) ⬇️
instrumentations-bucket-6 38.25% <ø> (-0.08%) ⬇️
instrumentations-bucket-7 35.92% <ø> (-0.07%) ⬇️
instrumentations-bucket-8 36.95% <ø> (-0.07%) ⬇️
instrumentations-bucket-9 39.45% <ø> (-0.07%) ⬇️
instrumentations-instrumentation-couchbase 46.48% <ø> (ø)
instrumentations-integration-esbuild 24.94% <1.43%> (-0.12%) ⬇️
llmobs-ai_anthropic_bedrock 39.53% <ø> (-0.07%) ⬇️
llmobs-google-genai_langchain_vertex-ai 36.97% <ø> (-0.06%) ⬇️
llmobs-openai 39.58% <ø> (-0.07%) ⬇️
llmobs-sdk 43.59% <ø> (-0.08%) ⬇️
master-coverage 93.77% <97.84%> (?)
openfeature 37.76% <ø> (ø)
openfeature-unit 50.39% <ø> (ø)
platform-core_esbuild_instrumentations-misc 23.38% <1.43%> (-0.12%) ⬇️
platform-integration 47.44% <ø> (+0.01%) ⬆️
platform-shimmer_unit-guardrails_webpack 18.91% <1.43%> (-0.12%) ⬇️
plugins-bucket-0 36.38% <ø> (-0.07%) ⬇️
plugins-bucket-1 39.60% <ø> (ø)
plugins-bucket-11 38.46% <ø> (-0.08%) ⬇️
plugins-bucket-17 39.15% <ø> (-0.07%) ⬇️
plugins-bucket-18 42.01% <ø> (-0.07%) ⬇️
plugins-bucket-19 39.56% <ø> (-0.07%) ⬇️
plugins-bucket-20 43.25% <ø> (-0.02%) ⬇️
plugins-bucket-4 37.71% <ø> (-0.07%) ⬇️
plugins-bullmq_cassandra_cookie 39.76% <ø> (-0.07%) ⬇️
plugins-cookie-parser_crypto_dd-trace-api 33.22% <ø> (-0.08%) ⬇️
plugins-fetch_fs_generic-pool 36.05% <ø> (-0.07%) ⬇️
plugins-google-cloud-pubsub_grpc_handlebars 43.10% <ø> (-0.07%) ⬇️
plugins-hapi_hono_ioredis 37.78% <ø> (-0.07%) ⬇️
plugins-jest_knex_langgraph 32.60% <ø> (-0.08%) ⬇️
plugins-ldapjs_light-my-request_limitd-client 27.91% <ø> (-0.08%) ⬇️
plugins-lodash_mariadb_memcached 35.17% <ø> (-0.08%) ⬇️
plugins-mongodb_mongodb-core_mongoose 36.31% <ø> (-0.07%) ⬇️
plugins-multer_mysql_mysql2 35.14% <ø> (-0.08%) ⬇️
plugins-nats_node-serialize_opensearch 37.14% <ø> (-0.08%) ⬇️
plugins-passport-http_pino_postgres 35.62% <ø> (+0.01%) ⬆️
plugins-process_pug_redis 34.26% <ø> (-0.08%) ⬇️
plugins-undici_url_valkey 35.88% <ø> (-0.07%) ⬇️
plugins-vm_winston_ws 37.55% <ø> (-0.08%) ⬇️
profiling 43.62% <ø> (-0.07%) ⬇️
serverless-aws-sdk-aws-sdk 33.29% <ø> (-0.07%) ⬇️
serverless-aws-sdk-bedrockruntime 32.16% <ø> (-0.07%) ⬇️
serverless-aws-sdk-client 37.16% <ø> (ø)
serverless-aws-sdk-dynamodb 34.14% <ø> (-0.07%) ⬇️
serverless-aws-sdk-eventbridge 27.25% <ø> (-0.07%) ⬇️
serverless-aws-sdk-kinesis 37.40% <ø> (-0.07%) ⬇️
serverless-aws-sdk-lambda 34.59% <ø> (-0.07%) ⬇️
serverless-aws-sdk-s3 32.59% <ø> (-0.07%) ⬇️
serverless-aws-sdk-serverless-peer-service 39.47% <ø> (-0.08%) ⬇️
serverless-aws-sdk-sns 38.25% <ø> (-0.07%) ⬇️
serverless-aws-sdk-sqs 38.00% <ø> (-0.07%) ⬇️
serverless-aws-sdk-stepfunctions 33.18% <ø> (-0.07%) ⬇️
serverless-aws-sdk-util 47.95% <ø> (ø)
serverless-bucket-0 39.46% <ø> (ø)
serverless-lambda 34.33% <ø> (-0.09%) ⬇️
test-optimization-cucumber 52.30% <ø> (+0.11%) ⬆️
test-optimization-cypress 49.41% <ø> (+0.06%) ⬆️
test-optimization-jest 55.34% <ø> (+0.04%) ⬆️
test-optimization-mocha 53.38% <ø> (+0.08%) ⬆️
test-optimization-playwright-playwright-atr 43.33% <ø> (+0.08%) ⬆️
test-optimization-playwright-playwright-efd 43.61% <ø> (+0.08%) ⬆️
test-optimization-playwright-playwright-final-status 43.73% <ø> (+0.08%) ⬆️
test-optimization-playwright-playwright-impacted-tests 43.16% <ø> (ø)
test-optimization-playwright-playwright-reporting 43.26% <ø> (+0.08%) ⬆️
test-optimization-playwright-playwright-test-management 44.72% <ø> (+0.08%) ⬆️
test-optimization-playwright-playwright-test-span 44.52% <ø> (+0.08%) ⬆️
test-optimization-selenium 45.26% <ø> (+0.06%) ⬆️
test-optimization-testopt 47.78% <8.63%> (-0.14%) ⬇️
test-optimization-vitest 51.37% <97.84%> (+0.46%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@pr-commenter

pr-commenter Bot commented Jul 1, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-07-02 10:17:30

Comparing candidate commit 9e5c56e in PR branch codex/fix-vitest-typecheck-reporting with baseline commit e035b64 in branch master.

📊 Benchmarking dashboard

Found 0 performance improvements and 0 performance regressions! Performance is the same for 2261 metrics, 25 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

Unstable benchmarks

These benchmarks have a confidence interval too wide to call a change; treat them as noise rather than signal.

scenario:appsec-appsec-enabled-24

  • unstable execution_time [-180112.819µs; +179622.919µs] or [-6.828%; +6.809%]

scenario:appsec-appsec-enabled-26

  • unstable execution_time [-204.853ms; +206.897ms] or [-8.113%; +8.194%]

scenario:appsec-appsec-enabled-with-attacks-26

  • unstable execution_time [-153.179ms; +159.857ms] or [-5.326%; +5.558%]

scenario:appsec-control-20

  • unstable execution_time [-168.945ms; +161.733ms] or [-9.625%; +9.214%]

scenario:appsec-control-24

  • unstable execution_time [-92.131ms; +108.505ms] or [-7.634%; +8.991%]

scenario:appsec-control-26

  • unstable execution_time [-111.468ms; +113.862ms] or [-9.243%; +9.442%]

scenario:debugger-line-probe-with-snapshot-minimal-24

  • unstable cpu_user_time [-2.569s; +0.342s] or [-30.356%; +4.044%]
  • unstable execution_time [-2.587s; +0.339s] or [-28.278%; +3.704%]
  • unstable instructions [-21.8G instructions; +2.9G instructions] or [-31.578%; +4.164%]
  • unstable throughput [-99.614op/s; +766.009op/s] or [-2.710%; +20.840%]

scenario:debugger-line-probe-without-snapshot-24

  • unstable cpu_user_time [-1.956s; +4.210s] or [-23.009%; +49.532%]
  • unstable execution_time [-1.952s; +4.228s] or [-21.222%; +45.968%]
  • unstable instructions [-16.7G instructions; +36.0G instructions] or [-24.094%; +51.989%]
  • unstable max_rss_usage [-7.449MB; +15.829MB] or [-4.645%; +9.870%]
  • unstable throughput [-1233.842op/s; +567.806op/s] or [-33.723%; +15.519%]

scenario:debugger-line-probe-without-snapshot-26

  • unstable cpu_user_time [-2.430s; +5.200s] or [-23.986%; +51.331%]
  • unstable execution_time [-2.493s; +5.267s] or [-23.019%; +48.623%]
  • unstable instructions [-21.3G instructions; +46.3G instructions] or [-25.125%; +54.653%]
  • unstable max_rss_usage [-8.431MB; +16.533MB] or [-5.225%; +10.246%]
  • unstable throughput [-1084.653op/s; +515.497op/s] or [-34.828%; +16.553%]

scenario:dogstatsd-with-tags-20

  • unstable cpu_user_time [-123.845ms; +543.184ms] or [-2.554%; +11.200%]
  • unstable execution_time [-130.285ms; +539.469ms] or [-2.645%; +10.951%]
  • unstable throughput [-184817.662op/s; +42804.106op/s] or [-10.840%; +2.511%]

scenario:plugin-graphql-long-with-depth-and-collapse-off-20

  • unstable max_rss_usage [-28.754MB; +15.432MB] or [-7.602%; +4.080%]

scenario:test-optimization-large-suite-20

  • unstable max_rss_usage [-4027.025KB; +3985.025KB] or [-5.075%; +5.022%]

@juan-fernandez

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 083be8b61b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/datadog-instrumentations/src/vitest-main.js Outdated
@juan-fernandez

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 017227b4a9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/datadog-instrumentations/src/vitest-main.js Outdated
@juan-fernandez

Copy link
Copy Markdown
Collaborator Author

@codex review

@juan-fernandez
juan-fernandez force-pushed the codex/fix-vitest-typecheck-reporting branch from 97dc11d to c7e9531 Compare July 1, 2026 21:45

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 97dc11df84

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/datadog-instrumentations/src/vitest-main.js Outdated
@juan-fernandez
juan-fernandez force-pushed the codex/fix-vitest-typecheck-reporting branch from c7e9531 to 9dd7d68 Compare July 1, 2026 21:57
@juan-fernandez

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9dd7d68b86

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/datadog-instrumentations/src/vitest-main.js
@juan-fernandez
juan-fernandez force-pushed the codex/fix-vitest-typecheck-reporting branch from 9dd7d68 to f50197c Compare July 2, 2026 08:03
@juan-fernandez

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f50197ce26

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/datadog-instrumentations/src/vitest-main.js Outdated
@juan-fernandez
juan-fernandez force-pushed the codex/fix-vitest-typecheck-reporting branch from f50197c to 8b2481a Compare July 2, 2026 08:31
@juan-fernandez

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8b2481a42c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/datadog-instrumentations/src/vitest-main.js
@juan-fernandez
juan-fernandez force-pushed the codex/fix-vitest-typecheck-reporting branch from 8b2481a to 0d6b5cb Compare July 2, 2026 08:58
@juan-fernandez

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0d6b5cb801

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/datadog-instrumentations/src/vitest-main.js
@juan-fernandez
juan-fernandez force-pushed the codex/fix-vitest-typecheck-reporting branch from 0d6b5cb to a5116ab Compare July 2, 2026 09:15
@juan-fernandez

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Another round soon, please!

Reviewed commit: a5116abf10

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@juan-fernandez
juan-fernandez marked this pull request as ready for review July 2, 2026 09:34
@juan-fernandez
juan-fernandez requested review from a team as code owners July 2, 2026 09:34
@juan-fernandez
juan-fernandez requested review from tlhunter and removed request for a team July 2, 2026 09:34
anmarchenko
anmarchenko previously approved these changes Jul 2, 2026

@datadog-datadog-prod-us1-2 datadog-datadog-prod-us1-2 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Datadog Autotest: FAIL

The typecheck reporting feature emits suite and test events but omits session and module events required by downstream Test Optimization services. This causes typecheck test results to drop in processing pipelines that expect complete event sequences. The PR passes local typecheck runs but fails CI integration tests that verify event completeness.

View proposed fix
📊 Validated against 1 scenarios · Open Bits AI session

🤖 Datadog Autotest · Commit a5116ab · What is Autotest? · Any feedback? Reach out in #autotest

Comment thread packages/datadog-instrumentations/src/vitest-main.js
@juan-fernandez

Copy link
Copy Markdown
Collaborator Author

/datadog autotest review

@datadog-datadog-prod-us1-2 datadog-datadog-prod-us1-2 Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Datadog Autotest: PASS

More details

Typecheck reporting implementation is sound. The code correctly wraps Vitest's Typechecker.prepareResults to emit CI Visibility events for typecheck-only tests, properly handles nested test names, and applies all Test Optimization metadata (Known Tests, Test Management, Impacted Tests) consistently with the existing pattern used for regular tests. No behavioral regressions detected across test state transitions, nested suite handling, or Test Management semantics.

Was this helpful? React 👍 or 👎

📊 Validated against 15 scenarios · Open Bits AI session

🤖 Datadog Autotest · Commit 9e5c56e · What is Autotest? · Any feedback? Reach out in #autotest

@juan-fernandez
juan-fernandez merged commit 1bdd6fc into master Jul 2, 2026
650 checks passed
@juan-fernandez
juan-fernandez deleted the codex/fix-vitest-typecheck-reporting branch July 2, 2026 10:37
@dd-octo-sts dd-octo-sts Bot mentioned this pull request Jul 2, 2026
dd-octo-sts Bot pushed a commit that referenced this pull request Jul 2, 2026
@dd-octo-sts dd-octo-sts Bot mentioned this pull request Jul 2, 2026
rochdev pushed a commit that referenced this pull request Jul 2, 2026
rochdev pushed a commit that referenced this pull request Jul 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants