Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

@trylek @steveisok @akoeplinger Is it acceptable to have a job in runtime which is known to have test failures, and thus the job has continueOnError: true?

this means that WASM tests will always run for SetPathVars_libraries.containsChange -> do we have enough to do WASM tests on every library change?

this means that WASM tests will always run for SetPathVars_libraries.containsChange -> do we have enough to do WASM tests on every library change?

This doesn't change the existing state. Wasm does use the libraries, so running tests for those makes sense to me. We already have CI holes due to running fewer tests on PRs, which sometimes causes changes that break wasm to get into main.

@trylek @steveisok @akoeplinger Is it acceptable to have a job in runtime which is known to have test failures, and thus the job has continueOnError: true?

which tests?

@trylek @steveisok @akoeplinger Is it acceptable to have a job in runtime which is known to have test failures, and thus the job has continueOnError: true?

which tests?

Firefox debugger tests

Regarding @radical's question, if the failing job with continueOnError in runtime makes the pipeline report as red, that's something we should avoid as it's the primary health indicator everyone's looking at; if the AzDO pipeline summary ends up showing all crosses in red circles, that's not good even if we convince ourselves that the tests semantically passed.

• In runtime-staging at least test failures make the job show up as orange. I'm guessing that it would be the same case for runtime? But would it show up as red for rolling builds?
• In the orange case too, the failures show up in kusto though. I don't how the CI runs are monitored though

I'm talking specifically about test failures.

• In runtime-staging at least test failures make the job show up as orange. I'm guessing that it would be the same case for runtime? But would it show up as red for rolling builds?
• In the orange case too, the failures show up in kusto though. I don't how the CI runs are monitored though

I'm talking specifically about test failures.

Would it be better to only execute the tests in runtime-extra-platforms?

My thinking is that the errors will only matter to a subset of people, so why not manage it post PR.

• In runtime-staging at least test failures make the job show up as orange. I'm guessing that it would be the same case for runtime? But would it show up as red for rolling builds?
• In the orange case too, the failures show up in kusto though. I don't how the CI runs are monitored though

I'm talking specifically about test failures.

Would it be better to only execute the tests in runtime-extra-platforms?

My thinking is that the errors will only matter to a subset of people, so why not manage it post PR.

That makes sense. But one problem is that the dev will need to run runtime-wasm pipeline, which has lots of other wasm jobs including all the AOT ones, which won't be needed for a PR with only debugger changes.

We could move it there though, if there aren't any good alternatives.

Done. Moved the Firefox debugger tests jobs, which has the failures, to runtime-extra-platforms.

/azp run runtime-wasm

That makes sense. But one problem is that the dev will need to run runtime-wasm pipeline, which has lots of other wasm jobs including all the AOT ones, which won't be needed for a PR with only debugger changes.

FWIW, there's always the possibility to introduce another pipeline and trigger it via an /azp command...

FWIW, there's always the possibility to introduce another pipeline and trigger it via an /azp command...

I'm starting to think we're going to need more pipelines :-).

The failure is #74588 .