Skip to content

coverage-v8: a file both inlined by vite and loaded natively (require / createRequire) gets wrong, run-to-run unstable coverage #11541

Open

Description

@islanddave

Describe the bug

When the same source file reaches V8 in two forms during one run, the v8 provider reports wrong coverage for it, and
the wrong answer can change between identical runs.

The two forms:

  1. a test imports the file, so vitest inlines it and runs vite's transformed text behind the module wrapper
    ('use strict';async (...)=>{{ + code). The worker records startOffset = wrapper length for that path;
  2. something loads the same file natively: an inlined CommonJS module calling require('./that-file'), or a test
    calling createRequire(import.meta.url)('./that-file'). Node compiles the file's own text. Its ranges are in the
    file's own coordinates, at offset 0.

Both scripts have the same URL. What then happens (read in 4.1.11):

  • @vitest/coverage-v8/dist/index.js takeCoverage attaches startOffset by a lookup on the file PATH in
    moduleExecutionInfo, falling back to 0. So a native script gets 0, unless the same worker also inlined that
    file, in which case the native script gets the wrapper length too.
  • dist/provider.js generateCoverage folds every per-test-file result into one list with mergeProcessCovs, which
    merges scripts by URL and drops startOffset on any URL with two or more scripts. The provider refills it from the
    result it is reading at that moment (the "mergeProcessCovs sometimes loses startOffset" block). The surviving
    offset for a URL is therefore that of the LAST per-test-file result read that contains the URL.
  • readCoverageFiles (BaseCoverageProvider) reads those results processingConcurrency at a time with
    Promise.all, so "last" is decided by the order tests finished and the order disk reads complete.
  • convertCoverage then converts the single merged script with that one offset, against vite's transformed code
    and source map (getSources -> transformFile), including the ranges that were counted in the file's own text.

Result: ranges from one form are read in the other form's coordinates. Functions that were called read as never
called, hit counts go negative, and which form "wins" depends on scheduling.

Reproduction (outline; 5 small files)

package.json next to the module (or use .cjs):

{ "type": "commonjs" }

forms.js:

'use strict'
// (a few comment lines here so the functions sit a couple of hundred characters into the file)

function throughVite(n) {
  if (n > 0) return 'positive'
  return 'not positive'
}

function byNode(n) {
  if (n > 0) return 'positive'
  return 'not positive'
}

function bothWays(n) {
  return n % 2 === 0 ? 'even' : 'odd'
}

function never() {
  return 'never called'
}

module.exports = { throughVite, byNode, bothWays, never }

imported.test.mjs (vite form only):

import { test, expect } from 'vitest'
import forms from './forms.js'
test('imported', () => {
  expect([1, 2, 3].map((n) => forms.throughVite(n))).toEqual(['positive', 'positive', 'positive'])
  expect(forms.bothWays(2)).toBe('even')
})

required.test.mjs (native form only):

import { test, expect } from 'vitest'
import { createRequire } from 'node:module'
const forms = createRequire(import.meta.url)('./forms.js')
test('required', () => {
  expect([forms.byNode(-1), forms.byNode(-2)]).toEqual(['not positive', 'not positive'])
  expect(forms.bothWays(3)).toBe('odd')
})

both-ways.test.mjs (both forms in one worker):

import { test, expect } from 'vitest'
import { createRequire } from 'node:module'
import imported from './forms.js'
const required = createRequire(import.meta.url)('./forms.js')
test('both', () => {
  expect(imported.throughVite(-1)).toBe('not positive')
  expect(required.byNode(1)).toBe('positive')
})

Config: no plugins needed.

export default {
  test: {
    environment: 'node',
    coverage: { provider: 'v8', include: ['forms.js'], reporter: ['json'] },
  },
}

Run vitest run --coverage a few times and read coverage/coverage-final.json.

Expected

All three tests pass, and the calls they make are: throughVite 4, byNode 3, bothWays 2, never 0. Branch arms:
throughVite's if [3, 1], byNode's if [1, 2], the ternary [1, 1].

Actual

Measured with vitest 4.1.11, @vitest/coverage-v8 4.1.11, vite 8.2.2, Node 26.4.0, macOS arm64, no plugins, three
consecutive runs of the same tree. The tests pass every time. Function hits and branch arms:

Run throughVite byNode bothWays never branch arms
1 0 0 1 0 [3, -3], [0, 0], [1, 0]
2 0 3 1 0 [0, 0], [1, 2], [0, 1]
3 0 3 1 0 [0, 0], [1, 2], [0, 1]

Two different readings from one tree, neither right, one with a negative hit count. With @vitejs/plugin-react in the
config (which reprints .js files, so the two texts differ as well as the offsets) the reading was the run-1 row,
twice.

In the project this was found in, on a 59-test-file subset: a 130 KB CommonJS module that 36 of the files import
and 29 reach through a handler's require (6 do both) read 1,214 or 1,257 covered items (of 1,374) by draw, four
runs each in eight, where converting each form on its own gives 1,325 every time; the handler's main function read
as never entered where the per-form reading counts 363 entries; and every stock reading carried 62 or 63 negative
hit counts.

A related case of the same one-offset-per-URL mechanism

A module that is still being evaluated when a worker's coverage is taken has NO recorded startOffset, because
moduleExecutionInfo.set(...) runs in a .finally after evaluation. Example: a component does a lazy
import('./data-module.js') that a test does not wait for; the module is compiled, entered, and suspended at its
first await __vite_ssr_import__(...). Its script is vite's text behind the wrapper, but it arrives with offset 0.
Merged with the file's other scripts it is harmless while a later-read result restores the real offset, and it puts
the whole file at offset 0 if its result is read last. Seen in a 1,505-test-file suite: 2 such scripts among 63,372,
for one file, in the same two test files every run. Reproduced in the fixture by adding:

// gate.mjs
export const gate = globalThis.__HOLD__ ? await new Promise(() => {}) : 'open'
// late.mjs
import { gate } from './gate.mjs'
export function early() { return 'early' }
export function late() { return 'late' }
export const loaded = [early(), gate]
// pending.test.mjs: sets globalThis.__HOLD__ = true, starts import('./late.mjs') and does not wait for it

with one other test that imports late.mjs normally and calls late() twice. Truth: early 1, late 2. Stock
(with @vitejs/plugin-react in the config), four consecutive runs of one tree: early 2, late 2 in three runs and
early 1, late 2 in one. Recording the offset before the module is evaluated, rather than after, would remove
this case.

Where the fix probably belongs (suggestions, not a patch)

  • Do not merge scripts of different source forms under one URL. Key the merge on (URL, form), or keep native scripts
    out of the merged list and convert them separately.
  • Convert a natively loaded script against the file's own text with wrapper length 0 and no source map, never against
    transformFile's output.
  • Decide the form in the worker, per script, instead of looking startOffset up by path: the lookup gives a native
    script the wrapper length whenever the same worker also inlined that file. (A main-side test that works today: a
    script is native when its outermost range ends at the file's own length. "No recorded offset" alone does not say
    so: see the related case above.)
  • When both conversions exist for a file, their istanbul maps do not match position for position (source-mapped
    columns differ slightly from parsed ones), so a plain coverageMap.merge doubles the item count. The maps need
    aligning by item order, or one conversion's maps need to be reused for the other.
  • Cheap tripwire worth having in any case: a negative hit count in the final map means ranges were read in the wrong
    coordinates.

Workaround in use

A custom coverage provider that subclasses V8CoverageProvider and does the above on the main side
(coverage.provider: 'custom', customProviderModule). It overrides generateCoverage, generateReports,
readCoverageFiles, convertCoverage and getSources, so it depends on internals; a public hook for "convert this
script against this text" would remove that dependency.

System info

  • vitest 4.1.11, @vitest/coverage-v8 4.1.11, vite 8.2.2, @bcoe/v8-coverage 1.0.2, ast-v8-to-istanbul 1.0.6
  • Node 26.4.0, macOS (Darwin 25.5.0, arm64)
  • The fixture was run only there. The project-suite symptom (totals changing between identical runs) was also seen in CI on Node 20.19.0, ubuntu-24.04.

Not established

  • Whether pool, isolate or vmThreads change anything (default pool only).
  • Whether the same happens for a natively import()ed ESM file that is also inlined; only CommonJS was tried.
  • The fixture on Node 20 or 22 (only the project suite was run on Node 20).

Activity

  1. AriPerkkio commented on Oct 9, 2026

    @AriPerkkio
    Member

    A module that is still being evaluated when a worker's coverage is taken has NO recorded startOffset, because moduleExecutionInfo.set(...) runs in a .finally after evaluation.

    This was fixed by #10643.

    a file both inlined by vite and loaded natively (require / createRequire)

    This is known issue, seen first time in #6297 and #7417. Back then it was accepted as limitation due to Vite not patching require.

  2. islanddave commented on Oct 10, 2026

    @islanddave
    Author

    Thanks for the quick and clear answer. That matches what we see: the require/createRequire case is the one we hit, so we'll keep our workaround on our side. Appreciate the pointers to #6297 and #7417.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions