Skip to content

CI: run webview tests with four workers #2288

Closed

Description

@edelauna

CI: run webview tests with four workers

Problem and evidence

Webview CI currently caps Vitest at one worker. Controlled local profiling and a GitHub-hosted benchmark both found four workers substantially faster without reducing the test inventory.

Hosted benchmark run, commit 86197abeb4527c810ed2e523efce8d95bf3056fb, Node 22.23.4 / Vitest 4.1.12, two repetitions per setting:

Test configuration 1 worker median 2 workers median 4 workers median
Ubuntu with coverage 283s 114s 95s
Windows without coverage 246s 238s 105s

All 12 repetitions ran the same 174 files / 2,269 tests, with all assertions passing and no retries. All six Linux covered-line sets matched (6,579 covered lines). Ubuntu baseline hardware differed (EPYC 7763 versus 9V45); Windows one/four used the same CPU model, but separate VMs. These are observed test-step improvements, not a demonstrated whole-pipeline 50% reduction. Earlier same-machine local profiling independently favored four workers.

One Ubuntu four-worker repetition exited 1 with Closing rpc while "onUserConsoleLog" was pending; its second repetition passed. Matching Vitest issue #11153 was addressed by PR #11619, listed in 5.1.0-beta.1. Our version lacks that fix. This sample does not establish that increasing workers introduced or increased the flake.

Implementation

  • Set the webview CI worker budget to four on the measured Ubuntu/Windows hosted runner classes.
  • Preserve forks, isolation, assertions, coverage and unhandled-error handling.
  • Keep the change independently reversible; the temporary benchmark harness is evidence, not required production infrastructure.
  • Track the upstream teardown fix separately; do not expand this change into an unvalidated major-version upgrade or suppress errors.

Acceptance criteria

  • Full webview inventory and coverage contract remain intact on both platforms.
  • Relevant package checks and full CI pass; retain and investigate any failures.
  • Record before/after test duration, job duration, runner minutes and time until required checks turn green.
  • Monitor comparable PRs for failures and duration regressions, recording sample sizes; two repetitions do not establish reliability or p95.
  • Document a simple rollback to the previous worker budget.

This increases parallelism within existing runner jobs rather than adding runner shards.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions