Migrate browser tests from Karma to Web Test Runner by matux · Pull Request #1300 · rollbar/rollbar.js

matux · 2025-09-08T19:05:32Z

Note

This PR is being merged into a feature branch: matux/wtr.
Multiple comments in the diff with details/reasoning about specific changes.
The PR description was mostly generated by Claude.

Description of the change

This PR completes the migration of browser tests from the deprecated Karma test runner to the modern Web Test Runner (WTR), continuing the modernization effort started in #1289. All browser tests have been converted from CommonJS to ES modules and updated to use modern async/await patterns.

Changes

Test Infrastructure

✅ Migrated all remaining browser tests from Karma to Web Test Runner
✅ Converted all test files from CommonJS (require/module.exports) to ES modules (import/export)
✅ Replaced done-callback async patterns with modern async/await
✅ Added web-test-runner.config.mjs configuration
✅ Created test utilities:
- test/util/timers.js - Promise-based setTimeout helper
- test/util/fixtures.js - HTML fixture loading utility

Test Organization

📁 Extracted large autoInstrument test section from browser.rollbar.test.js into focused test files:
- browser.rollbar.autoInstrument.test.js - Core autoInstrument tests
- browser.rollbar.autoInstrument.xhr.test.js - XHR telemetry tests
- browser.rollbar.autoInstrument.fetch.test.js - Fetch telemetry tests
- browser.rollbar.autoInstrument.csp.test.js - Content Security Policy tests
📁 Created browser.rollbar.test-utils.js for shared test utilities and stubs

Key Improvements

🚀 Faster test execution - WTR runs tests significantly faster than Karma
🔧 Better debugging - Native browser DevTools support with --manual flag
📦 Simplified dependencies - Removed Karma and its associated plugins
🎯 Modern patterns - Async/await makes tests more readable and maintainable
🧩 Better test organization - Separated concerns for easier maintenance

Technical Details

CSP Test Fix

Fixed Content Security Policy violation test that was failing under WTR by simulating SecurityPolicyViolationEvent directly. Details: https://github.com/rollbar/rollbar.js/pull/1300/files#r2331149823

ES Module Migration Pattern

// Before (CommonJS)
var expect = require('chai').expect;
module.exports = { TestClientGen };

// After (ES Modules)
import { expect } from 'chai';
export { TestClientGen };

Async Pattern Migration

// Before (done callback)
it('should test async', function(done) {
  setTimeout(function() {
    expect(true).to.be.true;
    done();
  }, 100);
});

// After (async/await)
it('should test async', async function() {
  await setTimeout(100);
  expect(true).to.be.true;
});

Testing

✅ All browser tests passing with Web Test Runner
✅ CI workflow updated and passing
✅ Backward compatibility maintained - no changes to SDK functionality

Next Steps

Future PRs will continue the modernization effort:

Further break down large test files for better maintainability
Complete migration of any remaining Karma configuration
Add TypeScript support to the build pipeline

matux · 2025-09-08T19:16:56Z

I wasn't able to make this work by loading the HTML, for some reason WTR+Mocha would capture the CSP event itself and bail out.

This is was happening in our new custom loadHtml() function here:

await new Promise((resolve, reject) => { newScript.onload = resolve; newScript.onerror = reject; // <-- This rejects the promise on script error document.body.appendChild(newScript); });

So, the rejected promise cause the test to fail with:

❌ options.autoInstrument > contentSecurityPolicy > should report content security policy errors Error: the event {"isTrusted":true} was thrown, throw an Error :) at Ws (node_modules/@web/test-runner-mocha/dist/autorun.js:1:204946) at Ys.fail (node_modules/@web/test-runner-mocha/dist/autorun.js:1:207579) at node_modules/@web/test-runner-mocha/dist/autorun.js:1:210970 at s (node_modules/@web/test-runner-mocha/dist/autorun.js:1:194229) at node_modules/@web/test-runner-mocha/dist/autorun.js:1:195193

I couldn't find a way to dissuade the test runner from meddling. After some research, I found out you can trigger a CSP violation event programmatically, and opted to go that route.

matux · 2025-09-08T19:31:16Z

These tests were extracted from browser.rollbar.test.js since they'd leave a dirty state that would make subsequent tests fail. I suspect this is due to concurrency. Separating these tests into their own file fixed the issues.

matux · 2025-09-08T19:32:11Z

Similar to fetch, these tests were extracted from browser.rollbar.test.js since they'd leave a dirty state that would make subsequent tests fail. I suspect this is due to concurrency. Separating these tests into their own file fixed the issues.

matux · 2025-09-08T19:33:12Z

These were extracted from browser.rollbar.test.js just for consistency (so all autoInstrument tests live separately from the main browser.rollbar.test.js.)

matux · 2025-09-08T19:33:44Z

+      // Manually trigger a CSP violation event,
+      // since WTR+Mocha will capture the rejection from loadHtml.
+      const cspEvent = new SecurityPolicyViolationEvent(
+        'securitypolicyviolation',
+        {
+          blockedURI: 'https://example.com/v3/',
+          violatedDirective: 'script-src',
+          effectiveDirective: 'script-src',
+          originalPolicy: "default-src 'self' 'unsafe-inline' 'unsafe-eval';",
+          sourceFile: window.location.href,
+          lineNumber: 1,
+          columnNumber: 1,
+        },
+      );


Refer to this comment for more details: https://github.com/rollbar/rollbar.js/pull/1300/files#r2331149823

Copilot

Pull Request Overview

This PR completes the migration of browser tests from the deprecated Karma test runner to the modern Web Test Runner (WTR), modernizing test infrastructure and patterns. The migration includes converting all test files from CommonJS to ES modules and replacing done-callback async patterns with modern async/await.

Migration from Karma to Web Test Runner for all browser tests
Conversion from CommonJS to ES modules with modern async/await patterns
Extraction of large test files into focused, maintainable modules

Reviewed Changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
test/fixtures/html/csp-errors.html	Removed HTML fixture file (no longer needed for WTR approach)
test/browser.rollbar.test.js	Migrated main test file to ES modules and async/await patterns
test/browser.rollbar.test-utils.js	Created shared utilities module for test stubs and helpers
test/browser.rollbar.autoInstrument.xhr.test.js	Extracted XHR telemetry tests into focused module
test/browser.rollbar.autoInstrument.test.js	Extracted core autoInstrument tests into focused module
test/browser.rollbar.autoInstrument.fetch.test.js	Extracted fetch telemetry tests into focused module
test/browser.rollbar.autoInstrument.csp.test.js	Extracted CSP tests into focused module with event simulation

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Co-authored-by: Copilot <[email protected]>

* Initial Karma to Web Test Runner migration (#1289) * Fix hanging tests and expand WTR migration (#1295) * Migrate example tests for Web Test Runner (#1298) * Migrate browser transform and core tests for Web Test Runner (#1299) * Migrate browser tests from Karma to Web Test Runner (#1300) * Initial migration of react native tests (#1301) * Remove Grunt and Karma (#1302)

matux added 5 commits September 8, 2025 13:08

fixes to rollbar-test

c345507

rollbar test utils

27de922

closer

b51483b

browser tests

e1ba951

fixed all remaining browser tests

9daf2b9

matux self-assigned this Sep 8, 2025

matux commented Sep 8, 2025

View reviewed changes

improve comment

f8527e6

matux commented Sep 8, 2025

View reviewed changes

matux requested a review from Copilot September 8, 2025 19:33

Copilot AI reviewed Sep 8, 2025

View reviewed changes

Comment thread test/browser.rollbar.test.js Outdated

Comment thread test/browser.rollbar.test.js

Comment thread test/browser.rollbar.test.js

Comment thread test/browser.rollbar.test.js

Comment thread test/browser.rollbar.test.js

Comment thread test/browser.rollbar.test.js

Update test/browser.rollbar.test.js

e2291c6

Co-authored-by: Copilot <[email protected]>

waltjones approved these changes Sep 8, 2025

View reviewed changes

remove exclude

ae0e589

matux merged commit 21de105 into matux/wtr Sep 8, 2025
4 checks passed

matux deleted the matux/wtr-browser branch September 8, 2025 20:42

matux mentioned this pull request Sep 8, 2025

[Meta-branch] Karma -> WTR migration and Grunt removal #1303

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Migrate browser tests from Karma to Web Test Runner#1300

Migrate browser tests from Karma to Web Test Runner#1300
matux merged 8 commits into
matux/wtrfrom
matux/wtr-browser

matux commented Sep 8, 2025 •

edited

Loading

Uh oh!

matux Sep 8, 2025

Uh oh!

matux Sep 8, 2025

Uh oh!

matux Sep 8, 2025

Uh oh!

matux Sep 8, 2025

Uh oh!

matux Sep 8, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

matux commented Sep 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of the change

Changes

Test Infrastructure

Test Organization

Key Improvements

Technical Details

CSP Test Fix

ES Module Migration Pattern

Async Pattern Migration

Testing

Next Steps

Uh oh!

matux Sep 8, 2025

Choose a reason for hiding this comment

Uh oh!

matux Sep 8, 2025

Choose a reason for hiding this comment

Uh oh!

matux Sep 8, 2025

Choose a reason for hiding this comment

Uh oh!

matux Sep 8, 2025

Choose a reason for hiding this comment

Uh oh!

matux Sep 8, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

matux commented Sep 8, 2025 •

edited

Loading