Considerably speed up building the test tree on Windows by MichalStrehovsky · Pull Request #82080 · dotnet/runtime

MichalStrehovsky · 2023-02-14T09:08:32Z

Building the JIT\Directed tree on my machine:

Before: 5 minutes 7 seconds.
After: 3 minutes 34 seconds.

Not kidding.

We run vswhere and vcvarsall. My bet is on vcvarsall - opening the visual studio tools command prompt always takes an eternity.

Cc @dotnet/ilc-contrib

Building the JIT\Directed tree on my machine before: 5 minutes 7 seconds. After: 3 minutes 34 seconds.

ghost · 2023-02-14T09:08:37Z

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

Issue Details

Building the JIT\Directed tree on my machine:

Before: 5 minutes 7 seconds.
After: 3 minutes 34 seconds.

Not kidding.

We run vswhere and vcvarsall. My bet is on vcvarsall - opening the visual studio tools command prompt always takes an eternity.

Cc @dotnet/ilc-contrib

Author:	MichalStrehovsky
Assignees:	-
Labels:	`area-NativeAOT-coreclr`
Milestone:	-

MichalStrehovsky · 2023-02-15T06:18:04Z

Hm, so one issue I just ran into with this is:

LINK : fatal error LNK1318: Unexpected PDB error; FILE_SYSTEM (3) 'C:\git\runtime\artifacts\tests\coreclr\windows.x64.R
elease\JIT\Performance\CodeQuality\Benchstones\BenchI\Puzzle\Puzzle\native\Puzzle.pdb' [C:\git\runtime\src\tests\JIT\Pe
rformance\CodeQuality\Benchstones\BenchI\Puzzle\Puzzle.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1201: error writing to program database 'C:\git\runtime\artifacts\tests\coreclr\windows.x64.Relea
se\JIT\Performance\CodeQuality\Benchstones\BenchI\Puzzle\Puzzle\native\Puzzle.pdb'; check for insufficient disk space,
invalid path, or insufficient privilege [C:\git\runtime\src\tests\JIT\Performance\CodeQuality\Benchstones\BenchI\Puzzle
\Puzzle.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\BenchI\Puzzle\Puzzle\native\link.rsp"" exited with code 1201. [C:\git\runtime\src\tests\JIT\Performance\C
odeQuality\Benchstones\BenchI\Puzzle\Puzzle.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1318: Unexpected PDB error; FILE_SYSTEM (3) 'C:\git\runtime\artifacts\tests\coreclr\windows.x64.R
elease\JIT\Performance\CodeQuality\Benchstones\BenchI\TreeInsert\TreeInsert\native\TreeInsert.pdb' [C:\git\runtime\src\
tests\JIT\Performance\CodeQuality\Benchstones\BenchI\TreeInsert\TreeInsert.csproj] [C:\git\runtime\src\tests\build.proj
]
LINK : fatal error LNK1201: error writing to program database 'C:\git\runtime\artifacts\tests\coreclr\windows.x64.Relea
se\JIT\Performance\CodeQuality\Benchstones\BenchI\TreeInsert\TreeInsert\native\TreeInsert.pdb'; check for insufficient
disk space, invalid path, or insufficient privilege [C:\git\runtime\src\tests\JIT\Performance\CodeQuality\Benchstones\B
enchI\TreeInsert\TreeInsert.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\BenchI\TreeInsert\TreeInsert\native\link.rsp"" exited with code 1201. [C:\git\runtime\src\tests\JIT\Perfo
rmance\CodeQuality\Benchstones\BenchI\TreeInsert\TreeInsert.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1201: error writing to program database 'C:\git\runtime\artifacts\tests\coreclr\windows.x64.Relea
se\JIT\Performance\CodeQuality\Benchstones\BenchI\QuickSort\QuickSort\native\QuickSort.pdb'; check for insufficient dis
k space, invalid path, or insufficient privilege [C:\git\runtime\src\tests\JIT\Performance\CodeQuality\Benchstones\Benc
hI\QuickSort\QuickSort.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\BenchI\QuickSort\QuickSort\native\link.rsp"" exited with code 1201. [C:\git\runtime\src\tests\JIT\Perform
ance\CodeQuality\Benchstones\BenchI\QuickSort\QuickSort.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1318: Unexpected PDB error; FILE_SYSTEM (3) 'C:\git\runtime\artifacts\tests\coreclr\windows.x64.R
elease\JIT\Performance\CodeQuality\Benchstones\BenchI\XposMatrix\XposMatrix\native\XposMatrix.pdb' [C:\git\runtime\src\
tests\JIT\Performance\CodeQuality\Benchstones\BenchI\XposMatrix\XposMatrix.csproj] [C:\git\runtime\src\tests\build.proj
]
LINK : fatal error LNK1201: error writing to program database 'C:\git\runtime\artifacts\tests\coreclr\windows.x64.Relea
se\JIT\Performance\CodeQuality\Benchstones\BenchI\XposMatrix\XposMatrix\native\XposMatrix.pdb'; check for insufficient
disk space, invalid path, or insufficient privilege [C:\git\runtime\src\tests\JIT\Performance\CodeQuality\Benchstones\B
enchI\XposMatrix\XposMatrix.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\BenchI\XposMatrix\XposMatrix\native\link.rsp"" exited with code 1201. [C:\git\runtime\src\tests\JIT\Perfo
rmance\CodeQuality\Benchstones\BenchI\XposMatrix\XposMatrix.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1201: error writing to program database 'C:\git\runtime\artifacts\tests\coreclr\windows.x64.Relea
se\JIT\Performance\CodeQuality\Benchstones\BenchI\TreeSort\TreeSort\native\TreeSort.pdb'; check for insufficient disk s
pace, invalid path, or insufficient privilege [C:\git\runtime\src\tests\JIT\Performance\CodeQuality\Benchstones\BenchI\
TreeSort\TreeSort.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\BenchI\TreeSort\TreeSort\native\link.rsp"" exited with code 1201. [C:\git\runtime\src\tests\JIT\Performan
ce\CodeQuality\Benchstones\BenchI\TreeSort\TreeSort.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1201: error writing to program database 'C:\git\runtime\artifacts\tests\coreclr\windows.x64.Relea
se\JIT\Performance\CodeQuality\Benchstones\MDBenchF\MDInProd\MDInProd\native\MDInProd.pdb'; check for insufficient disk
 space, invalid path, or insufficient privilege [C:\git\runtime\src\tests\JIT\Performance\CodeQuality\Benchstones\MDBen
chF\MDInProd\MDInProd.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\MDBenchF\MDInProd\MDInProd\native\link.rsp"" exited with code 1201. [C:\git\runtime\src\tests\JIT\Perform
ance\CodeQuality\Benchstones\MDBenchF\MDInProd\MDInProd.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1318: Unexpected PDB error; RPC (23) '(0x000006BE)' [C:\git\runtime\src\tests\JIT\Performance\Cod
eQuality\Benchstones\MDBenchI\MDLogicArray\MDLogicArray.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1318: Unexpected PDB error; RPC (23) '(0x000006BE)' [C:\git\runtime\src\tests\JIT\Performance\Cod
eQuality\Benchstones\MDBenchI\MDMulMatrix\MDMulMatrix.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1318: Unexpected PDB error; RPC (23) '(0x000006BE)' [C:\git\runtime\src\tests\JIT\Performance\Cod
eQuality\Benchstones\MDBenchI\MDGeneralArray\MDGeneralArray.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1318: Unexpected PDB error; RPC (23) '(0x000006BE)' [C:\git\runtime\src\tests\JIT\Performance\Cod
eQuality\Benchstones\MDBenchI\MDMidpoint\MDMidpoint.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1318: Unexpected PDB error; RPC (23) '(0x000006BA)' [C:\git\runtime\src\tests\JIT\Performance\Cod
eQuality\Benchstones\MDBenchF\MDSqMtx\MDSqMtx.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\MDBenchF\MDSqMtx\MDSqMtx\native\link.rsp"" exited with code 1318. [C:\git\runtime\src\tests\JIT\Performan
ce\CodeQuality\Benchstones\MDBenchF\MDSqMtx\MDSqMtx.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\MDBenchI\MDMulMatrix\MDMulMatrix\native\link.rsp"" exited with code 1318. [C:\git\runtime\src\tests\JIT\P
erformance\CodeQuality\Benchstones\MDBenchI\MDMulMatrix\MDMulMatrix.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\MDBenchI\MDGeneralArray\MDGeneralArray\native\link.rsp"" exited with code 1318. [C:\git\runtime\src\tests
\JIT\Performance\CodeQuality\Benchstones\MDBenchI\MDGeneralArray\MDGeneralArray.csproj] [C:\git\runtime\src\tests\build
.proj]
LINK : fatal error LNK1318: Unexpected PDB error; RPC (23) '(0x000006BA)' [C:\git\runtime\src\tests\JIT\Performance\Cod
eQuality\Benchstones\MDBenchF\MDRomber\MDRomber.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\MDBenchI\MDLogicArray\MDLogicArray\native\link.rsp"" exited with code 1318. [C:\git\runtime\src\tests\JIT
\Performance\CodeQuality\Benchstones\MDBenchI\MDLogicArray\MDLogicArray.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\MDBenchI\MDMidpoint\MDMidpoint\native\link.rsp"" exited with code 1318. [C:\git\runtime\src\tests\JIT\Per
formance\CodeQuality\Benchstones\MDBenchI\MDMidpoint\MDMidpoint.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\MDBenchF\MDRomber\MDRomber\native\link.rsp"" exited with code 1318. [C:\git\runtime\src\tests\JIT\Perform
ance\CodeQuality\Benchstones\MDBenchF\MDRomber\MDRomber.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1318: Unexpected PDB error; RPC (23) '(0x000006BA)' [C:\git\runtime\src\tests\JIT\Performance\Cod
eQuality\Benchstones\MDBenchF\MDLLoops\MDLLoops.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\MDBenchF\MDLLoops\MDLLoops\native\link.rsp"" exited with code 1318. [C:\git\runtime\src\tests\JIT\Perform
ance\CodeQuality\Benchstones\MDBenchF\MDLLoops\MDLLoops.csproj] [C:\git\runtime\src\tests\build.proj]
LINK : fatal error LNK1318: Unexpected PDB error; RPC (23) '(0x000006BA)' [C:\git\runtime\src\tests\JIT\Performance\Cod
eQuality\Benchstones\MDBenchI\MDAddArray2\MDAddArray2.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\artifacts\bin\coreclr\windows.x64.Release\build\Microsoft.NETCore.Native.targets(336,5): error MSB3073:
The command ""link" @"C:\git\runtime\artifacts\tests\coreclr/obj/windows.x64.Release/Managed/JIT\Performance\CodeQualit
y\Benchstones\MDBenchI\MDAddArray2\MDAddArray2\native\link.rsp"" exited with code 1318. [C:\git\runtime\src\tests\JIT\P
erformance\CodeQuality\Benchstones\MDBenchI\MDAddArray2\MDAddArray2.csproj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\src\tests\Common\dir.traversal.targets(25,5): error : (No message specified) [C:\git\runtime\src\tests\b
uild.proj] [C:\git\runtime\src\tests\build.proj]
C:\git\runtime\src\tests\build.proj(496,5): error MSB3073: The command ""C:\git\runtime\\dotnet.cmd" msbuild C:\git\run
time\src\tests\build.proj /t:Build "/p:TargetArchitecture=x64" "/p:Configuration=Release" "/p:LibrariesConfiguration=Re
lease" "/p:TargetOS=windows" "/p:RuntimeOS=" "/p:RuntimeFlavor=coreclr" "/p:RuntimeVariant=" "/p:CLRTestBuildAllTargets
=" "/p:UseCodeFlowEnforcement=" "/p:__TestGroupToBuild=2" "/p:__SkipRestorePackages=1" /nodeReuse:false /maxcpucount /b
l:C:\git\runtime\artifacts\/log/Release/InnerManagedTestBuild.2.binlog "/p:DefaultBuildAllTarget=BuildNativeAot"" exite
d with code 1.

The fatal error LNK1318: Unexpected PDB error; RPC (23) is a mystery error that could be related to link.exe running out of memory. Indeed the logs have extra link.exe output where it says it's restarting as x64 because it ran out of memory.

The repo builds with x86_amd64 cross linker (#76516). The linker ILC would set up is the amd64 native linker.

I'm converting this to draft while I meditate on this. It's possible the compile time wins are also due to using the x86 linker. But also if there's stability issues, there's little point in doing this :(

jkotas · 2023-02-15T07:19:02Z

Also, do instructions for running individual tests need updating for this change? https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/nativeaot.md#running-tests

MichalStrehovsky · 2023-02-24T06:19:04Z

We can only do this if #76516 is fixed.

Considerably speed up building the test tree on Windows

f4b9091

Building the JIT\Directed tree on my machine before: 5 minutes 7 seconds. After: 3 minutes 34 seconds.

MichalStrehovsky added the area-NativeAOT-coreclr label Feb 14, 2023

ghost assigned MichalStrehovsky Feb 14, 2023

MichalStrehovsky marked this pull request as draft February 15, 2023 06:18

MichalStrehovsky closed this Feb 24, 2023

MichalStrehovsky deleted the fastbuild branch February 24, 2023 06:19

ghost locked as resolved and limited conversation to collaborators Mar 26, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Considerably speed up building the test tree on Windows#82080

Considerably speed up building the test tree on Windows#82080
MichalStrehovsky wants to merge 1 commit intodotnet:mainfrom
MichalStrehovsky:fastbuild

MichalStrehovsky commented Feb 14, 2023

Uh oh!

ghost commented Feb 14, 2023

Uh oh!

MichalStrehovsky commented Feb 15, 2023

Uh oh!

jkotas commented Feb 15, 2023

Uh oh!

MichalStrehovsky commented Feb 24, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

MichalStrehovsky commented Feb 14, 2023

Uh oh!

ghost commented Feb 14, 2023

Uh oh!

MichalStrehovsky commented Feb 15, 2023

Uh oh!

jkotas commented Feb 15, 2023

Uh oh!

MichalStrehovsky commented Feb 24, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants