Context

The idea is to run tc39/test262 tests suite against Babel.

Running the official spec tests against Babel can make sure we comply to the spec or at least know when we don’t. We’ll want to figure out how to do filtering on things we can’t compile (proxy, TCO, etc) and set up an easy way to check failing tests and file issues and PRs for them.

This issue will be used to track this.

Previous discussions:

• test262 conformance as an line item the compat table compat-table/compat-table#830
• https://github.com/bakkot/test262-web-runner/
• Steps to run test262 with Babel tc39/test262-harness#58
• https://github.com/motiz88/babel-smoke-tests

Possible Solution

• https://github.com/bterlson/test262-harness (it seems pretty straightforward, replace node with babel-node ?)

I think we should use our own test runner. We might want to skip some tests.

There are some restriction in order to run the tests. They are well explained in https://github.com/tc39/test262/blob/master/INTERPRETING.md#test-execution.

The tests could be run automaticly on the CI (they are taking some time).

/cc @kangax

Edit: 11/12

My test is available here: https://github.com/xtuc/babel-test262.

Some observations I made:

• Running all tests takes too long (build log)
• Mocha reporter and only the language built-ins tests (build log) and this time the ouput was too long.
• Mocha dot reporter and only the language built-ins tests seems to work pretty well (build log)