Group JitBlue regression tests by runner by tannergooding · Pull Request #133591 · dotnet/runtime

tannergooding · 2026-09-10T12:03:42Z

Summary

Replace the conflict-prone regression source lists with globs over per-runner directories. Keep the existing categories (JitBlue, Dev11, Dev14, and CLR-x86-JIT) at the same level, and group sources beneath them by runner. Single-file tests no longer need a directory of their own; related multi-file tests remain together.

Group project-backed JitBlue tests beneath Regression_2 and Regression_3, preserving all 235 individual projects and their supporting files. Update the wrapper project globs and shared-source references without changing existing runner membership, compilation settings, or process-isolation requirements. Update the test-authoring guidance to distinguish source-glob runners from project-backed runners.

All 1,198 relocated files retain identical contents. The one intentional coverage change is Runtime_131557: it previously had no project reference and is now included in Regression_ro_2.

Validation

Built and ran all 15 affected runners on Windows x64 Checked: 1,013 passed, 81 skipped, and no failures. Confirmed Runtime_131557 executes and passes.
Reconfigured native test discovery, rebuilt Runtime101046Native and CrossplatVirtualAlloc at their new paths, and copied their dependencies through the existing test build flow.
Compared evaluated source/project membership and compiler settings before and after the moves; verified exact-content renames and checked for stale path references.
Linux, ARM, and Wasm were not run locally.

Note

This pull request description was drafted with GitHub Copilot.

azure-pipelines · 2026-09-10T12:04:34Z

Azure Pipelines: Successfully started running 4 pipeline(s). 12 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run.

dotnet-policy-service · 2026-09-10T12:05:14Z

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Co-authored-by: Copilot App <[email protected]>

Copilot

Copilot review overview

🔵 Needs a closer look

The change moves/re-globs a very large test surface area, so a human should validate runner membership and discovery end-to-end (beyond the specific issues flagged in review comments).

Review tier: Lite
Findings: None

What changed in this PR

Refactors CoreCLR JIT regression test organization under src/tests/JIT/Regression by switching from explicit, conflict-prone source lists to directory/glob-based inclusion that groups tests beneath runner-specific directories (and updates the relevant merged-runner wrapper projects accordingly).

Changes:

Replace several explicit <Compile Include="..."> entries with a runner-folder glob (*\$(MSBuildProjectName)\**\*.cs) for merged PdbOnly runners.
Update merged wrapper project globs so Regression_2/Regression_3 only pick up projects from JitBlue/Regression_2 and JitBlue/Regression_3.
Relocate/add regression test sources and project-backed tests under the new runner-grouped layout; update documentation example path accordingly.

File	Description
src/tests/JIT/Regression/Regression_PdbOnly_ro.csproj	Switches JitBlue PdbOnly-ro compilation from explicit includes to a runner-directory glob.
src/tests/JIT/Regression/Regression_PdbOnly_r_3.csproj	Switches JitBlue PdbOnly-r3 compilation from explicit includes to a runner-directory glob.
src/tests/JIT/Regression/Regression_2.csproj	Narrows merged wrapper project discovery to `JitBlue/Regression_2/*/.??proj`.
src/tests/JIT/Regression/Regression_3.csproj	Narrows merged wrapper project discovery to `JitBlue/Regression_3/*/.??proj`.
src/tests/JIT/Regression/JitBlue/Regression_o_3/Runtime_67723.cs	Relocated regression test source under runner grouping.
src/tests/JIT/Regression/JitBlue/Regression_o_2/Runtime_106143.cs	Relocated regression test source under runner grouping.
src/tests/JIT/Regression/JitBlue/Regression_ro_2/Runtime_61359.cs	Relocated regression test source under runner grouping.
docs/workflow/testing/coreclr/disasm-checks.md	Updates example test path to match the new directory layout.

Copilot

Copilot review overview

🔵 Needs a closer look

The change is dominated by large-scale test source/project relocation and build-discovery behavior changes that are difficult to fully validate here (especially cross-platform) from the provided diffs alone.

Review tier: Lite
Findings: None

jkotas · 2026-09-10T14:36:28Z

src/tests/JIT/Regression/JitBlue/Regression_2/Runtime_

If we are moving things around, can we omit some parts of the path? Would src/tests/JIT/Regression_2/Runtime_ make sense?

tannergooding · 2026-09-10T15:04:29Z

If we are moving things around, can we omit some parts of the path? Would src/tests/JIT/Regression_2/Runtime_ make sense?

We can, I had just tried to keep this mostly minimal to start, essentially just taking files/projects "as is" and then moving them under a folder for what globs them.

The general goal here is really just to stop the constant merge conflicts we otherwise are getting from adding new regression tests into the merged runner projects.

Co-authored-by: Copilot App <[email protected]>

Move all 1,198 files without content changes. Update build wiring in the following commit. Co-authored-by: Copilot App <[email protected]>

Preserve project settings through the existing regression props and exclude relocated projects from the outer JIT runners. Update source globs, shared-source references, and test-authoring guidance. Co-authored-by: Copilot App <[email protected]>

Resolve the source-list conflict by placing Runtime_133518 and Runtime_133521 under the Regression_ro_2 glob without changing their contents. Co-authored-by: Copilot App <[email protected]>

tannergooding · 2026-09-10T16:04:18Z

Shorted the paths as suggested and resolved merge conflicts that came in

Copilot

Copilot review overview

🔵 Needs a closer look

It performs a large-scale test layout/build-graph reshaping with broad potential impact, and I can’t fully validate runner membership and cross-platform behavior from the provided diffs alone.

Review tier: Lite
Findings: None

Co-authored-by: Copilot App <[email protected]>

tannergooding · 2026-09-10T19:52:02Z

Had to move the csproj to their respective source folders as there is a pre-existing behavior for included projects that they're expected to be nested compared to the project including them, which is what caused the WASM failures

Copilot

Copilot review overview

🔵 Needs a closer look

There are a few concrete correctness/quality issues in newly added/relocated test sources (type-name typos and minor text/whitespace issues) that should be fixed before merging.

Review tier: Lite
Findings: None

tannergooding · 2026-09-10T23:28:16Z

/ba-g Remaining failures are the same Mono failures reproducing on main

@jkoritzinsky could I get another sign-off so this can be merged?

Resolve the regression test layout conflict from dotnet#133591. Co-authored-by: Copilot <[email protected]> Copilot-Session: be30cab5-d605-4896-b07e-9758803fed4f

Copilot AI lite review requested due to automatic review settings September 10, 2026 12:03

github-actions Bot added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Sep 10, 2026

Copilot started reviewing on behalf of tannergooding September 10, 2026 12:04

View session

dotnet-policy-service Bot assigned tannergooding Sep 10, 2026

tannergooding and others added 2 commits September 10, 2026 05:07

Group regression test files by runner

f259146

Co-authored-by: Copilot App <[email protected]>

Use globs for grouped regression tests

42c785e

Co-authored-by: Copilot App <[email protected]>

tannergooding force-pushed the tannergooding-project-conflict-resolution branch from 894d737 to 42c785e Compare September 10, 2026 12:08

Copilot AI reviewed Sep 10, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings September 10, 2026 12:12

Copilot started reviewing on behalf of tannergooding September 10, 2026 12:12

View session

Copilot AI reviewed Sep 10, 2026

View reviewed changes

tannergooding and others added 4 commits September 10, 2026 08:09

Merge upstream main into JitBlue test regrouping

cb06363

Co-authored-by: Copilot App <[email protected]>

Shorten grouped JIT regression test paths

70ba6b7

Move all 1,198 files without content changes. Update build wiring in the following commit. Co-authored-by: Copilot App <[email protected]>

Merge upstream regression additions into the grouped runner

11b6202

Resolve the source-list conflict by placing Runtime_133518 and Runtime_133521 under the Regression_ro_2 glob without changing their contents. Co-authored-by: Copilot App <[email protected]>

Copilot AI review requested due to automatic review settings September 10, 2026 16:02

Copilot started reviewing on behalf of tannergooding September 10, 2026 16:03

View session

Copilot AI reviewed Sep 10, 2026

View reviewed changes

jkotas requested a review from jkoritzinsky September 10, 2026 16:11

jkoritzinsky approved these changes Sep 10, 2026

View reviewed changes

Colocate regression runners with their test directories

bd78d7f

Co-authored-by: Copilot App <[email protected]>

Copilot AI review requested due to automatic review settings September 10, 2026 19:51

Copilot started reviewing on behalf of tannergooding September 10, 2026 19:51

View session

Copilot AI reviewed Sep 10, 2026

View reviewed changes

tannergooding enabled auto-merge (squash) September 10, 2026 23:28

jkoritzinsky approved these changes Sep 11, 2026

View reviewed changes

tannergooding merged commit ac550b6 into dotnet:main Sep 11, 2026
89 of 91 checks passed

dotnet-maestro Bot mentioned this pull request Sep 11, 2026

[main] Source code updates from dotnet/runtime dotnet/dotnet#9263

Merged

AndyAyersMS added a commit to AndyAyersMS/runtime that referenced this pull request Sep 11, 2026

Merge upstream/main

98f034a

Resolve the regression test layout conflict from dotnet#133591. Co-authored-by: Copilot <[email protected]> Copilot-Session: be30cab5-d605-4896-b07e-9758803fed4f

dotnet-milestone-bot Bot added this to the 12.0-preview1 milestone Sep 14, 2026

github-actions Bot locked and limited conversation to collaborators Oct 15, 2026

Navigation Menu

Repository navigation

Group JitBlue regression tests by runner - #133591

Group JitBlue regression tests by runner#133591
tannergooding merged 7 commits into
dotnet:mainfrom
tannergooding:tannergooding-project-conflict-resolution

tannergooding commented Sep 10, 2026

Uh oh!

azure-pipelines Bot commented Sep 10, 2026

Uh oh!

dotnet-policy-service Bot commented Sep 10, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

jkotas commented Sep 10, 2026

Uh oh!

tannergooding commented Sep 10, 2026

Uh oh!

tannergooding commented Sep 10, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

tannergooding commented Sep 10, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

tannergooding commented Sep 10, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

tannergooding commented Sep 10, 2026

Summary

Validation

Uh oh!

azure-pipelines Bot commented Sep 10, 2026

Uh oh!

dotnet-policy-service Bot commented Sep 10, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🔵 Needs a closer look

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🔵 Needs a closer look

Uh oh!

jkotas commented Sep 10, 2026

Uh oh!

tannergooding commented Sep 10, 2026

Uh oh!

tannergooding commented Sep 10, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🔵 Needs a closer look

Uh oh!

tannergooding commented Sep 10, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🔵 Needs a closer look

Uh oh!

tannergooding commented Sep 10, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants