Skip to content

feat(serve): Add server-side handlers for Test262 tests [pt 2/5]#56841

Merged
jcscottiii merged 8 commits intomasterfrom
feat/test262-serve-handler
Mar 17, 2026
Merged

feat(serve): Add server-side handlers for Test262 tests [pt 2/5]#56841
jcscottiii merged 8 commits intomasterfrom
feat/test262-serve-handler

Conversation

@jcscottiii
Copy link
Copy Markdown
Contributor

This commit introduces the necessary server-side handlers within wptserve to dynamically generate HTML wrappers for Test262 JavaScript tests. This is needed to enable Test262 execution within WPT.

Key changes and their purpose:

  • Introduction of several new HtmlWrapperHandler and WrapperHandler subclasses (e.g., Test262WindowHandler, Test262WindowTestHandler, Test262StrictHandler). These handlers are responsible for:
    • Identifying Test262 test requests based on URL patterns.
    • Dynamically constructing an HTML page that loads the Test262 .js test file within an isolated iframe.
    • Injecting the required Test262 harness files (assert.js, sta.js) and the WPT-specific testharness-client.js and harness-adapter.js into the generated HTML.
    • Processing Test262-specific metadata (like includes and negative expectations) extracted by the manifest tooling from PR 1 (feat(manifest): Add initial tooling for Test262 test types [pt 1/5] #56840 .
  • Updates to RoutesBuilder in serve.py to map incoming requests for Test262 test URLs to the appropriate new handler.
  • Unit tests in test_serve.py to validate the correct behavior of these new handlers, including URL rewriting, metadata processing, and the structure of the generated HTML wrappers.

This work directly supports the integration of Test262 into WPT as detailed in the RFC: web-platform-tests/rfcs#229

This commit is the second in a series of smaller PRs split from the larger, original implementation in #55997.

jcscottiii pushed a commit that referenced this pull request Dec 18, 2025
…e tests

This commit completes the core integration of Test262 into the WPT
runner (`wptrunner`), enabling the execution of Test262 JavaScript
conformance tests within the browser environment. This work builds upon
the manifest tooling (PR 1 #56840) and server-side handling (PR 2 #56841) to provide
a full end-to-end testing solution for Test262.

Key components introduced and integrated:
- **Client-Side Harness:** New JavaScript files (`harness-adapter.js`,
  `testharness-client.js`, `testharness.js`) under `resources/test262/`
  provide the necessary environment for Test262 tests to execute in an
  `iframe` and report results back to the main WPT testharness.
- **Test262 Assertion Libraries (Vendored):** For the purpose of enabling these
  initial smoke tests, the standard Test262 assertion libraries (`assert.js`
  and `sta.js`) are directly placed into `third_party/test262/harness/`.
  This provides the immediate dependencies required by the smoke tests.
  An autoamted vendoring solution, including version tracking via
  `vendored.toml`, will be implemented in a subsequent PR focused on
  automated updates, as described in the RFC.
- **Wptrunner Integration:** Modifications to `tools/wptrunner/wpttest.py`
  and `tools/wptrunner/browsers/*.py` enable `wptrunner` to recognize the
  `test262` test type and execute it using existing testharness executors.
- **Smoke Tests and Metadata:** A suite of basic Test262 smoke tests
  (`infrastructure/test262/`) and their corresponding expected results
  (`infrastructure/metadata/infrastructure/test262/`) are added to verify
  the correct functioning of the entire integration.
- **Linting Exemption:** `lint.ignore` is updated to exclude the vendored
  Test262 harness files from linting, respecting their upstream style.

This integration allows browser vendors to run Test262 directly with `wptrunner`,
streamlining conformance testing efforts as specified in the RFC:
web-platform-tests/rfcs#229

This commit is the third in a series of smaller PRs split from the larger,
original implementation in #55997.
@jcscottiii jcscottiii changed the title feat(serve): Add server-side handlers for Test262 tests feat(serve): Add server-side handlers for Test262 tests [pt 2/4] Dec 18, 2025
@jcscottiii jcscottiii changed the title feat(serve): Add server-side handlers for Test262 tests [pt 2/4] feat(serve): Add server-side handlers for Test262 tests [pt 2/5] Dec 18, 2025
jcscottiii pushed a commit that referenced this pull request Dec 18, 2025
…e tests

This commit completes the core integration of Test262 into the WPT
runner (`wptrunner`), enabling the execution of Test262 JavaScript
conformance tests within the browser environment. This work builds upon
the manifest tooling (PR 1 #56840) and server-side handling (PR 2 #56841) to provide
a full end-to-end testing solution for Test262.

Key components introduced and integrated:
- **Client-Side Harness:** New JavaScript files (`harness-adapter.js`,
  `testharness-client.js`, `testharness.js`) under `resources/test262/`
  provide the necessary environment for Test262 tests to execute in an
  `iframe` and report results back to the main WPT testharness.
- **Test262 Assertion Libraries (Vendored):** For the purpose of enabling these
  initial smoke tests, the standard Test262 assertion libraries (`assert.js`
  and `sta.js`) are directly placed into `third_party/test262/harness/`.
  This provides the immediate dependencies required by the smoke tests.
  An autoamted vendoring solution, including version tracking via
  `vendored.toml`, will be implemented in a subsequent PR focused on
  automated updates, as described in the RFC.
- **Wptrunner Integration:** Modifications to `tools/wptrunner/wpttest.py`
  and `tools/wptrunner/browsers/*.py` enable `wptrunner` to recognize the
  `test262` test type and execute it using existing testharness executors.
- **Smoke Tests and Metadata:** A suite of basic Test262 smoke tests
  (`infrastructure/test262/`) and their corresponding expected results
  (`infrastructure/metadata/infrastructure/test262/`) are added to verify
  the correct functioning of the entire integration.
- **Linting Exemption:** `lint.ignore` is updated to exclude the vendored
  Test262 harness files from linting, respecting their upstream style.

This integration allows browser vendors to run Test262 directly with `wptrunner`,
streamlining conformance testing efforts as specified in the RFC:
web-platform-tests/rfcs#229

This commit is the third in a series of smaller PRs split from the larger,
original implementation in #55997.
jcscottiii pushed a commit that referenced this pull request Dec 18, 2025
…e tests

This commit completes the core integration of Test262 into the WPT
runner (`wptrunner`), enabling the execution of Test262 JavaScript
conformance tests within the browser environment. This work builds upon
the manifest tooling (PR 1 #56840) and server-side handling (PR 2 #56841) to provide
a full end-to-end testing solution for Test262.

Key components introduced and integrated:
- **Client-Side Harness:** New JavaScript files (`harness-adapter.js`,
  `testharness-client.js`, `testharness.js`) under `resources/test262/`
  provide the necessary environment for Test262 tests to execute in an
  `iframe` and report results back to the main WPT testharness.
- **Test262 Assertion Libraries (Vendored):** For the purpose of enabling these
  initial smoke tests, the standard Test262 assertion libraries (`assert.js`
  and `sta.js`) are directly placed into `third_party/test262/harness/`.
  This provides the immediate dependencies required by the smoke tests.
  An autoamted vendoring solution, including version tracking via
  `vendored.toml`, will be implemented in a subsequent PR focused on
  automated updates, as described in the RFC.
- **Wptrunner Integration:** Modifications to `tools/wptrunner/wpttest.py`
  and `tools/wptrunner/browsers/*.py` enable `wptrunner` to recognize the
  `test262` test type and execute it using existing testharness executors.
- **Smoke Tests and Metadata:** A suite of basic Test262 smoke tests
  (`infrastructure/test262/`) and their corresponding expected results
  (`infrastructure/metadata/infrastructure/test262/`) are added to verify
  the correct functioning of the entire integration.
- **Linting Exemption:** `lint.ignore` is updated to exclude the vendored
  Test262 harness files from linting, respecting their upstream style.

This integration allows browser vendors to run Test262 directly with `wptrunner`,
streamlining conformance testing efforts as specified in the RFC:
web-platform-tests/rfcs#229

This commit is the third in a series of smaller PRs split from the larger,
original implementation in #55997.
@jcscottiii jcscottiii force-pushed the feat/test262-manifest-tooling branch from ad5fa6b to 72ff9c7 Compare December 18, 2025 14:01
@jcscottiii jcscottiii force-pushed the feat/test262-serve-handler branch from 16aa645 to d5eb82e Compare December 18, 2025 14:05
jcscottiii added a commit that referenced this pull request Dec 18, 2025
…e tests

This commit completes the core integration of Test262 into the WPT
runner (`wptrunner`), enabling the execution of Test262 JavaScript
conformance tests within the browser environment. This work builds upon
the manifest tooling (PR 1 #56840) and server-side handling (PR 2 #56841) to provide
a full end-to-end testing solution for Test262.

Key components introduced and integrated:
- **Client-Side Harness:** New JavaScript files (`harness-adapter.js`,
  `testharness-client.js`, `testharness.js`) under `resources/test262/`
  provide the necessary environment for Test262 tests to execute in an
  `iframe` and report results back to the main WPT testharness.
- **Test262 Assertion Libraries (Vendored):** For the purpose of enabling these
  initial smoke tests, the standard Test262 assertion libraries (`assert.js`
  and `sta.js`) are directly placed into `third_party/test262/harness/`.
  This provides the immediate dependencies required by the smoke tests.
  An autoamted vendoring solution, including version tracking via
  `vendored.toml`, will be implemented in a subsequent PR focused on
  automated updates, as described in the RFC.
- **Wptrunner Integration:** Modifications to `tools/wptrunner/wpttest.py`
  and `tools/wptrunner/browsers/*.py` enable `wptrunner` to recognize the
  `test262` test type and execute it using existing testharness executors.
- **Smoke Tests and Metadata:** A suite of basic Test262 smoke tests
  (`infrastructure/test262/`) and their corresponding expected results
  (`infrastructure/metadata/infrastructure/test262/`) are added to verify
  the correct functioning of the entire integration.
- **Linting Exemption:** `lint.ignore` is updated to exclude the vendored
  Test262 harness files from linting, respecting their upstream style.

This integration allows browser vendors to run Test262 directly with `wptrunner`,
streamlining conformance testing efforts as specified in the RFC:
web-platform-tests/rfcs#229

This commit is the third in a series of smaller PRs split from the larger,
original implementation in #55997.
Copy link
Copy Markdown
Contributor

@DanielRyanSmith DanielRyanSmith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This suggestion came from a Gemini code review:

File: tools/serve/serve.py
Code: with open(path, encoding='ISO-8859-1') as f:
Criticism: Using ISO-8859-1 to read Test262 source files is risky and possibly incorrect.

The Problem: Test262 contains tests with Unicode identifiers and string literals (stored as UTF-8). Reading them as ISO-8859-1 treats the bytes as Latin-1 characters. If the server framework later encodes the response as UTF-8 (which is standard), multi-byte characters will be "double-encoded" (mangled). For example, a UTF-8 byte sequence for a specific emoji will be read as separate Latin-1 characters and then encoded again.

Fix: Use encoding='utf-8'. If you are worried about mixed encodings in legacy tests, use errors='replace', but Test262 should be UTF-8 compliant.

@jcscottiii jcscottiii force-pushed the feat/test262-serve-handler branch from d5eb82e to dcdca82 Compare December 19, 2025 17:56
@jcscottiii
Copy link
Copy Markdown
Contributor Author

This suggestion came from a Gemini code review:

File: tools/serve/serve.py Code: with open(path, encoding='ISO-8859-1') as f: Criticism: Using ISO-8859-1 to read Test262 source files is risky and possibly incorrect.

The Problem: Test262 contains tests with Unicode identifiers and string literals (stored as UTF-8). Reading them as ISO-8859-1 treats the bytes as Latin-1 characters. If the server framework later encodes the response as UTF-8 (which is standard), multi-byte characters will be "double-encoded" (mangled). For example, a UTF-8 byte sequence for a specific emoji will be read as separate Latin-1 characters and then encoded again.

Fix: Use encoding='utf-8'. If you are worried about mixed encodings in legacy tests, use errors='replace', but Test262 should be UTF-8 compliant.

Handled

@jcscottiii jcscottiii force-pushed the feat/test262-serve-handler branch from dcdca82 to df146d5 Compare December 19, 2025 18:44
jcscottiii added a commit that referenced this pull request Dec 19, 2025
…e tests

This commit completes the core integration of Test262 into the WPT
runner (`wptrunner`), enabling the execution of Test262 JavaScript
conformance tests within the browser environment. This work builds upon
the manifest tooling (PR 1 #56840) and server-side handling (PR 2 #56841) to provide
a full end-to-end testing solution for Test262.

Key components introduced and integrated:
- **Client-Side Harness:** New JavaScript files (`harness-adapter.js`,
  `testharness-client.js`, `testharness.js`) under `resources/test262/`
  provide the necessary environment for Test262 tests to execute in an
  `iframe` and report results back to the main WPT testharness.
- **Test262 Assertion Libraries (Vendored):** For the purpose of enabling these
  initial smoke tests, the standard Test262 assertion libraries (`assert.js`
  and `sta.js`) are directly placed into `third_party/test262/harness/`.
  This provides the immediate dependencies required by the smoke tests.
  An autoamted vendoring solution, including version tracking via
  `vendored.toml`, will be implemented in a subsequent PR focused on
  automated updates, as described in the RFC.
- **Wptrunner Integration:** Modifications to `tools/wptrunner/wpttest.py`
  and `tools/wptrunner/browsers/*.py` enable `wptrunner` to recognize the
  `test262` test type and execute it using existing testharness executors.
- **Smoke Tests and Metadata:** A suite of basic Test262 smoke tests
  (`infrastructure/test262/`) and their corresponding expected results
  (`infrastructure/metadata/infrastructure/test262/`) are added to verify
  the correct functioning of the entire integration.
- **Linting Exemption:** `lint.ignore` is updated to exclude the vendored
  Test262 harness files from linting, respecting their upstream style.

This integration allows browser vendors to run Test262 directly with `wptrunner`,
streamlining conformance testing efforts as specified in the RFC:
web-platform-tests/rfcs#229

This commit is the third in a series of smaller PRs split from the larger,
original implementation in #55997.
@jcscottiii jcscottiii requested a review from jgraham January 27, 2026 16:00
@jcscottiii jcscottiii dismissed jgraham’s stale review January 27, 2026 16:00

Addressed feedback

@jcscottiii
Copy link
Copy Markdown
Contributor Author

@jgraham This should be good to go now

jcscottiii added a commit that referenced this pull request Mar 12, 2026
…e tests

This commit completes the core integration of Test262 into the WPT
runner (`wptrunner`), enabling the execution of Test262 JavaScript
conformance tests within the browser environment. This work builds upon
the manifest tooling (PR 1 #56840) and server-side handling (PR 2 #56841) to provide
a full end-to-end testing solution for Test262.

Key components introduced and integrated:
- **Client-Side Harness:** New JavaScript files (`harness-adapter.js`,
  `testharness-client.js`, `testharness.js`) under `resources/test262/`
  provide the necessary environment for Test262 tests to execute in an
  `iframe` and report results back to the main WPT testharness.
- **Test262 Assertion Libraries (Vendored):** For the purpose of enabling these
  initial smoke tests, the standard Test262 assertion libraries (`assert.js`
  and `sta.js`) are directly placed into `third_party/test262/harness/`.
  This provides the immediate dependencies required by the smoke tests.
  An autoamted vendoring solution, including version tracking via
  `vendored.toml`, will be implemented in a subsequent PR focused on
  automated updates, as described in the RFC.
- **Wptrunner Integration:** Modifications to `tools/wptrunner/wpttest.py`
  and `tools/wptrunner/browsers/*.py` enable `wptrunner` to recognize the
  `test262` test type and execute it using existing testharness executors.
- **Smoke Tests and Metadata:** A suite of basic Test262 smoke tests
  (`infrastructure/test262/`) and their corresponding expected results
  (`infrastructure/metadata/infrastructure/test262/`) are added to verify
  the correct functioning of the entire integration.
- **Linting Exemption:** `lint.ignore` is updated to exclude the vendored
  Test262 harness files from linting, respecting their upstream style.

This integration allows browser vendors to run Test262 directly with `wptrunner`,
streamlining conformance testing efforts as specified in the RFC:
web-platform-tests/rfcs#229

This commit is the third in a series of smaller PRs split from the larger,
original implementation in #55997.
jcscottiii added a commit that referenced this pull request Mar 12, 2026
…e tests

This commit completes the core integration of Test262 into the WPT
runner (`wptrunner`), enabling the execution of Test262 JavaScript
conformance tests within the browser environment. This work builds upon
the manifest tooling (PR 1 #56840) and server-side handling (PR 2 #56841) to provide
a full end-to-end testing solution for Test262.

Key components introduced and integrated:
- **Client-Side Harness:** New JavaScript files (`harness-adapter.js`,
  `testharness-client.js`, `testharness.js`) under `resources/test262/`
  provide the necessary environment for Test262 tests to execute in an
  `iframe` and report results back to the main WPT testharness.
- **Test262 Assertion Libraries (Vendored):** For the purpose of enabling these
  initial smoke tests, the standard Test262 assertion libraries (`assert.js`
  and `sta.js`) are directly placed into `third_party/test262/harness/`.
  This provides the immediate dependencies required by the smoke tests.
  An autoamted vendoring solution, including version tracking via
  `vendored.toml`, will be implemented in a subsequent PR focused on
  automated updates, as described in the RFC.
- **Wptrunner Integration:** Modifications to `tools/wptrunner/wpttest.py`
  and `tools/wptrunner/browsers/*.py` enable `wptrunner` to recognize the
  `test262` test type and execute it using existing testharness executors.
- **Smoke Tests and Metadata:** A suite of basic Test262 smoke tests
  (`infrastructure/test262/`) and their corresponding expected results
  (`infrastructure/metadata/infrastructure/test262/`) are added to verify
  the correct functioning of the entire integration.
- **Linting Exemption:** `lint.ignore` is updated to exclude the vendored
  Test262 harness files from linting, respecting their upstream style.

This integration allows browser vendors to run Test262 directly with `wptrunner`,
streamlining conformance testing efforts as specified in the RFC:
web-platform-tests/rfcs#229

This commit is the third in a series of smaller PRs split from the larger,
original implementation in #55997.
@jcscottiii
Copy link
Copy Markdown
Contributor Author

As stated in the other PR, I adjusted the names of the javascript files. If all is well, I can adjust the PR description with the updated names.

Copy link
Copy Markdown
Contributor

@DanielRyanSmith DanielRyanSmith left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some more optional suggestions:

1. Module Execution Order in Test262WindowModuleTestHandler

The wrapper for Test262WindowModuleTestHandler defines the following script:

<script type="module">
  test262Setup();
  import {} from "%(path)s";
  test262Done();
</script>

This will not execute in the intended order. In JavaScript ES modules, static import declarations are hoisted. This means that the imported test module (%(path)s) will be fetched and evaluated before test262Setup() is executed. Since test262Setup() initializes the testing harness context (like assert or reporting mechanisms), the test will run without the required setup.

Furthermore, if the imported Test262 module utilizes top-level await, test262Done() will not wait for the module's execution to complete because import inside the same script tag doesn't block the execution of the surrounding synchronous code in the way one might expect when mixing static imports and inline code.

Recommendation:
Separate the setup and teardown into their own script blocks, or use dynamic import() to guarantee execution order.

<script type="module">
  test262Setup();
</script>
<script type="module">
  import "%(path)s";
</script>
<script type="module">
  test262Done();
</script>

Note: Using dynamic import("%(path)s").then(test262Done) might be more robust if top-level await is involved.

Optimizations

1. File Read Efficiency in Test262StrictHandler._get_script

In Test262StrictHandler._get_script, the file is loaded into memory entirely:

    def _get_script(self, request):
        path = self._get_filesystem_path(request)
        try:
            with open(path, encoding='utf-8') as f:
                yield f.read()
        except OSError:
            raise HTTPException(404)

Test262 files can occasionally be large. Reading the entire file into memory at once with f.read() before yielding might cause unnecessary memory overhead, especially if the server processes many concurrent requests.

Recommendation:
Yield the file in chunks instead of a single string.

            with open(path, encoding='utf-8') as f:
                while True:
                    chunk = f.read(8192)
                    if not chunk:
                        break
                    yield chunk

@jcscottiii
Copy link
Copy Markdown
Contributor Author

Some more optional suggestions:

1. Module Execution Order in Test262WindowModuleTestHandler

The wrapper for Test262WindowModuleTestHandler defines the following script:

<script type="module">
  test262Setup();
  import {} from "%(path)s";
  test262Done();
</script>

This will not execute in the intended order. In JavaScript ES modules, static import declarations are hoisted. This means that the imported test module (%(path)s) will be fetched and evaluated before test262Setup() is executed. Since test262Setup() initializes the testing harness context (like assert or reporting mechanisms), the test will run without the required setup.

Furthermore, if the imported Test262 module utilizes top-level await, test262Done() will not wait for the module's execution to complete because import inside the same script tag doesn't block the execution of the surrounding synchronous code in the way one might expect when mixing static imports and inline code.

Recommendation: Separate the setup and teardown into their own script blocks, or use dynamic import() to guarantee execution order.

<script type="module">
  test262Setup();
</script>
<script type="module">
  import "%(path)s";
</script>
<script type="module">
  test262Done();
</script>

Note: Using dynamic import("%(path)s").then(test262Done) might be more robust if top-level await is involved.

Optimizations

1. File Read Efficiency in Test262StrictHandler._get_script

In Test262StrictHandler._get_script, the file is loaded into memory entirely:

    def _get_script(self, request):
        path = self._get_filesystem_path(request)
        try:
            with open(path, encoding='utf-8') as f:
                yield f.read()
        except OSError:
            raise HTTPException(404)

Test262 files can occasionally be large. Reading the entire file into memory at once with f.read() before yielding might cause unnecessary memory overhead, especially if the server processes many concurrent requests.

Recommendation: Yield the file in chunks instead of a single string.

            with open(path, encoding='utf-8') as f:
                while True:
                    chunk = f.read(8192)
                    if not chunk:
                        break
                    yield chunk

Thanks for the suggestions @DanielRyanSmith! I have implemented the module execution order fix. However, I left the streaming part alone for now. We would need to change handle_request in WrapperHandler to stream the whole response. Not just the part that calls _get_script

Comment on lines +272 to +278
Test262WindowHandler.__test__ = False # type: ignore[attr-defined]
Test262WindowTestHandler.__test__ = False # type: ignore[attr-defined]
Test262WindowModuleHandler.__test__ = False # type: ignore[attr-defined]
Test262WindowModuleTestHandler.__test__ = False # type: ignore[attr-defined]
Test262StrictWindowHandler.__test__ = False # type: ignore[attr-defined]
Test262StrictWindowTestHandler.__test__ = False # type: ignore[attr-defined]
Test262StrictHandler.__test__ = False # type: ignore[attr-defined]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand what this is for.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because those handlers start with Test, the unit test runner thinks it is a test to run.

Without these lines completely, we get:

ERROR serve/test_serve.py::Test262WindowHandler - pytest.PytestCollectionWarning: cannot collect test class 'Test262WindowHandler' because it has a __init__ constructor...
ERROR serve/test_serve.py::Test262WindowTestHandler - pytest.PytestCollectionWarning: cannot collect test class 'Test262WindowTestHandler' because it has a __init__ constru...
ERROR serve/test_serve.py::Test262WindowModuleHandler - pytest.PytestCollectionWarning: cannot collect test class 'Test262WindowModuleHandler' because it has a __init__ const...
ERROR serve/test_serve.py::Test262WindowModuleTestHandler - pytest.PytestCollectionWarning: cannot collect test class 'Test262WindowModuleTestHandler' because it has a __init__ c...
ERROR serve/test_serve.py::Test262StrictWindowHandler - pytest.PytestCollectionWarning: cannot collect test class 'Test262StrictWindowHandler' because it has a __init__ const...
ERROR serve/test_serve.py::Test262StrictWindowTestHandler - pytest.PytestCollectionWarning: cannot collect test class 'Test262StrictWindowTestHandler' because it has a __init__ c...
ERROR serve/test_serve.py::Test262StrictHandler - pytest.PytestCollectionWarning: cannot collect test class 'Test262StrictHandler' because it has a __init__ constructor...

We ignore the attr-defined rule for these because, we will get these mypy errors otherwise:

tools/serve/test_serve.py:272: error: "type[Test262WindowHandler]" has no attribute "__test__"  [attr-defined]
tools/serve/test_serve.py:273: error: "type[Test262WindowTestHandler]" has no attribute "__test__"  [attr-defined]
tools/serve/test_serve.py:274: error: "type[Test262WindowModuleHandler]" has no attribute "__test__"  [attr-defined]
tools/serve/test_serve.py:275: error: "type[Test262WindowModuleTestHandler]" has no attribute "__test__"  [attr-defined]
tools/serve/test_serve.py:276: error: "type[Test262StrictWindowHandler]" has no attribute "__test__"  [attr-defined]
tools/serve/test_serve.py:277: error: "type[Test262StrictWindowTestHandler]" has no attribute "__test__"  [attr-defined]
tools/serve/test_serve.py:278: error: "type[Test262StrictHandler]" has no attribute "__test__"  [attr-defined]

This is similar to this instance:

TestQueueBuilder.__test__ = None # type: ignore[attr-defined]
TestRoot.__test__ = None # type: ignore[attr-defined]

or

this (but all mypy errors are ignored here)

# mypy: ignore-errors
from io import BytesIO
from unittest import mock
from manifest import manifest as wptmanifest
from manifest.item import TestharnessTest, RefTest
from manifest.utils import to_os_path
from . test_update import tree_and_sourcefile_mocks
from .. import manifestexpected, manifestupdate, wpttest
TestharnessTest.__test__ = False

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I realised what was probably going on right before looking at this review again. I think it's confusing enough to warrant a comment.

@jcscottiii jcscottiii force-pushed the feat/test262-serve-handler branch from adef084 to 956ba8b Compare March 16, 2026 14:04
Copy link
Copy Markdown
Contributor

@jgraham jgraham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r+ with nit

Comment on lines +272 to +278
Test262WindowHandler.__test__ = False # type: ignore[attr-defined]
Test262WindowTestHandler.__test__ = False # type: ignore[attr-defined]
Test262WindowModuleHandler.__test__ = False # type: ignore[attr-defined]
Test262WindowModuleTestHandler.__test__ = False # type: ignore[attr-defined]
Test262StrictWindowHandler.__test__ = False # type: ignore[attr-defined]
Test262StrictWindowTestHandler.__test__ = False # type: ignore[attr-defined]
Test262StrictHandler.__test__ = False # type: ignore[attr-defined]
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I realised what was probably going on right before looking at this review again. I think it's confusing enough to warrant a comment.

Base automatically changed from feat/test262-manifest-tooling to master March 17, 2026 16:23
@wpt-pr-bot wpt-pr-bot added the wpt label Mar 17, 2026
jcscottiii and others added 8 commits March 17, 2026 16:27
This commit introduces the necessary server-side handlers within `wptserve`
to dynamically generate HTML wrappers for Test262 JavaScript tests.
This is needed to enable Test262 execution within WPT.

Key changes and their purpose:
- Introduction of several new `HtmlWrapperHandler` and `WrapperHandler`
  subclasses (e.g., `Test262WindowHandler`, `Test262WindowTestHandler`,
  `Test262StrictHandler`). These handlers are responsible for:
  - Identifying Test262 test requests based on URL patterns.
  - Dynamically constructing an HTML page that loads the Test262
    `.js` test file within an isolated `iframe`.
  - Injecting the required Test262 harness files (`assert.js`, `sta.js`)
    and the WPT-specific `testharness-client.js` and `harness-adapter.js`
    into the generated HTML.
  - Processing Test262-specific metadata (like `includes` and `negative`
    expectations) extracted by the manifest tooling from PR 1.
- Updates to `RoutesBuilder` in `serve.py` to map incoming requests
  for Test262 test URLs to the appropriate new handler.
- Unit tests in `test_serve.py` to validate the correct
  behavior of these new handlers, including URL rewriting, metadata
  processing, and the structure of the generated HTML wrappers.

This work directly supports the integration of Test262 into WPT as detailed
in the RFC: web-platform-tests/rfcs#229

This commit is the second in a series of smaller PRs split from the larger,
original implementation in #55997.
@jcscottiii jcscottiii force-pushed the feat/test262-serve-handler branch from 12f7b3e to 1f27a1e Compare March 17, 2026 16:27
@jcscottiii jcscottiii closed this Mar 17, 2026
@jcscottiii jcscottiii reopened this Mar 17, 2026
@jcscottiii
Copy link
Copy Markdown
Contributor Author

The failing check is this:

=================================== FAILURES ===================================
_______________________ test_list_tests_missing_manifest _______________________

cls = <class '_pytest.runner.CallInfo'>
func = <function call_and_report.<locals>.<lambda> at 0x11a76d2d0>
when = 'call'
reraise = (<class '_pytest.outcomes.Exit'>, <class 'KeyboardInterrupt'>)

    @classmethod
    def from_call(
        cls,
        func: Callable[[], TResult],
        when: Literal["collect", "setup", "call", "teardown"],
        reraise: type[BaseException] | tuple[type[BaseException], ...] | None = None,
    ) -> CallInfo[TResult]:
        """Call func, wrapping the result in a CallInfo.
    
        :param func:
            The function to call. Called without arguments.
        :type func: Callable[[], _pytest.runner.TResult]
        :param when:
            The phase in which the function is called.
        :param reraise:

And it's only failing on Mac os x on a particular python version (whereas other versions passed). And it seems unrelated. Will merge.

@jcscottiii jcscottiii merged commit 2fc4cd1 into master Mar 17, 2026
45 of 47 checks passed
@jcscottiii jcscottiii deleted the feat/test262-serve-handler branch March 17, 2026 20:44
jcscottiii added a commit that referenced this pull request Mar 17, 2026
…e tests

This commit completes the core integration of Test262 into the WPT
runner (`wptrunner`), enabling the execution of Test262 JavaScript
conformance tests within the browser environment. This work builds upon
the manifest tooling (PR 1 #56840) and server-side handling (PR 2 #56841) to provide
a full end-to-end testing solution for Test262.

Key components introduced and integrated:
- **Client-Side Harness:** New JavaScript files (`harness-adapter.js`,
  `testharness-client.js`, `testharness.js`) under `resources/test262/`
  provide the necessary environment for Test262 tests to execute in an
  `iframe` and report results back to the main WPT testharness.
- **Test262 Assertion Libraries (Vendored):** For the purpose of enabling these
  initial smoke tests, the standard Test262 assertion libraries (`assert.js`
  and `sta.js`) are directly placed into `third_party/test262/harness/`.
  This provides the immediate dependencies required by the smoke tests.
  An autoamted vendoring solution, including version tracking via
  `vendored.toml`, will be implemented in a subsequent PR focused on
  automated updates, as described in the RFC.
- **Wptrunner Integration:** Modifications to `tools/wptrunner/wpttest.py`
  and `tools/wptrunner/browsers/*.py` enable `wptrunner` to recognize the
  `test262` test type and execute it using existing testharness executors.
- **Smoke Tests and Metadata:** A suite of basic Test262 smoke tests
  (`infrastructure/test262/`) and their corresponding expected results
  (`infrastructure/metadata/infrastructure/test262/`) are added to verify
  the correct functioning of the entire integration.
- **Linting Exemption:** `lint.ignore` is updated to exclude the vendored
  Test262 harness files from linting, respecting their upstream style.

This integration allows browser vendors to run Test262 directly with `wptrunner`,
streamlining conformance testing efforts as specified in the RFC:
web-platform-tests/rfcs#229

This commit is the third in a series of smaller PRs split from the larger,
original implementation in #55997.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants