Add -CentralizedRestore to avoid NuGet parallel restore race conditions by ilonatommy · Pull Request #65069 · dotnet/aspnetcore

ilonatommy · 2026-01-15T12:39:50Z

Avoid "Cannot create a file when that file already exists" on CI

.dotnet\sdk\11.0.100-alpha.1.25618.104\NuGet.targets(196,5): error : (NETCORE_ENGINEERING_TELEMETRY=Build) Cannot create a file when that file already exists.

parallel-restore-log.txt

This happens very often on our CI, e.g. build. Rerun fixes it but it also prolongs the time from posting PR to merging it and is annoying. The issue is reported here: NuGet/Home#7648.

Description

By default, we are restoring in parallel. BuildInParallel=true is set in dotnet\sdk\*.*.*\Microsoft.Common.CurrentVersion.targets shipped with the sdk:

<BuildInParallel Condition="'$(BuildInParallel)' == ''">true</BuildInParallel>

When BuildInParallel="true", MSBuild invokes restore on multiple projects simultaneously. NuGet's RestoreTask is not thread-safe when:

Multiple projects share the same dependency
NuGet tries to download/extract the same package to the same location concurrently
Two threads race to create the same file in the global packages folder

It is known and reported in NuGet/Home#7648.

The NuGet issue mentions a workaround. If we managed to perform a single coordinated restore of all projects upfront before the build starts, then run the actual build with restore disabled, it would avoid NuGet's internal race condition where parallel restores of projects with shared dependencies can conflict.

Details

There are 5 places with Targets="Restore":

File	What it restores	Fix when CentralizedRestore=true
FunctionalTestWithAssets.targets	Test asset projects (in solution)	Skip - already restored by upfront `dotnet restore AspNetCore.slnx --disable-parallel`
BuildAfterTargetingPack.csproj	`RequiresDelayedBuild` projects (need targeting pack first)	Sequential - can't restore upfront, so use `BuildInParallel=false` to avoid race condition
trimmingTests.targets	Trimming test console apps	No change needed - already sequential (batched with `%`)
SiteExtension.targets	Hosting startup packages	No change needed - already sequential (batched with `%`)
Tools.props	GenerateFiles.csproj (single project)	No change needed - single project, no race condition

CentralizedRestore flag in build.ps1 - When it is set, it:

Restores AspNetCore.slnx with --disable-parallel
Generates build files (GenerateFiles.csproj)
Disables per-project restore during build (/p:Restore=false)
FunctionalTestWithAssets.targets - Skip test asset restore when CentralizedRestore=true (already restored upfront)

BuildAfterTargetingPack.csproj - Use sequential restore (BuildInParallel=false) for delayed projects when CentralizedRestore=true (these projects can't be restored upfront because they depend on the targeting pack being built first)

Performance impact

Running locally (always after ./clean.cmd) it's actually faster but I took only 1 sample:
Originally: 131.6s
CentralizedRestore=true: 111.8s

Does it fix the issue?

It's hard to reproduce the race but we could re-run this PR a few times.

ilonatommy · 2026-01-15T14:17:52Z

/azp run

azure-pipelines · 2026-01-15T14:18:12Z

Azure Pipelines successfully started running 3 pipeline(s).

Copilot

Pull request overview

This pull request introduces a -CentralizedRestore flag to address NuGet parallel restore race conditions that cause "Cannot create a file when that file already exists" errors on CI. The solution performs a single coordinated restore upfront with parallel processing disabled, then runs the build with per-project restore disabled.

Changes:

Adds -CentralizedRestore parameter to build.ps1 that restores the solution sequentially upfront
Modifies BuildAfterTargetingPack.csproj to use sequential restore for delayed build projects when centralized restore is enabled
Updates FunctionalTestWithAssets.targets to skip restore when centralized restore is enabled
Adds CentralizedRestore.proj as an alternative manual approach for centralized restore

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
eng/build.ps1	Adds CentralizedRestore parameter and implements upfront sequential restore of solution and build file generation
src/BuildAfterTargetingPack/BuildAfterTargetingPack.csproj	Adds logic to use sequential restore (_RestoreInParallel=false) for delayed build projects when CentralizedRestore is enabled
eng/targets/FunctionalTestWithAssets.targets	Adds condition to skip test asset restore when CentralizedRestore is enabled, as they're already restored via solution
eng/CentralizedRestore.proj	New file providing an alternative manual approach for centralized restore with RestoreDelayedOnly and RestoreAll targets

Copilot · 2026-01-15T14:20:07Z

eng/build.ps1

 # -NoBuild implies -NoRestore, unless -Restore is explicitly set (as in restore.cmd)
+# -CentralizedRestore performs a single upfront restore then disables per-project restore
 $RunRestore = if ($NoRestore) { $false }
+    elseif ($CentralizedRestore) { $false } # Will do centralized restore separately


The CentralizedRestore property needs to be passed to MSBuild so that the condition checks in BuildAfterTargetingPack.csproj and FunctionalTestWithAssets.targets can work correctly. Without this property being set, the sequential restore logic in BuildAfterTargetingPack.csproj (line 12-13) and the skip restore condition in FunctionalTestWithAssets.targets (line 46) will not activate, defeating the purpose of the centralized restore feature. Add a line after line 280 like: if ($CentralizedRestore) { $MSBuildArguments += "/p:CentralizedRestore=true" }

Copilot · 2026-01-15T14:20:08Z

eng/CentralizedRestore.proj

+    <Message Importance="High" Text="Centralized Restore - Restoring all projects" />
+    <Message Importance="High" Text="======================================" />
+
+    <Exec Command="dotnet restore &quot;$(RepoRoot)AspNetCore.slnx&quot;" />


The dotnet restore command should include the --disable-parallel flag to avoid NuGet's internal race conditions, consistent with the implementation in build.ps1 line 503. The whole purpose of centralized restore is to avoid parallel restore issues. Change to: dotnet restore "$(RepoRoot)AspNetCore.slnx" --disable-parallel

Suggested change

<Exec Command="dotnet restore "$(RepoRoot)AspNetCore.slnx"" />

<Exec Command="dotnet restore "$(RepoRoot)AspNetCore.slnx" --disable-parallel" />

Copilot · 2026-01-15T14:20:08Z

eng/CentralizedRestore.proj

+    Or manually:
+      dotnet restore AspNetCore.slnx
+      dotnet msbuild eng/CentralizedRestore.proj /t:RestoreDelayedOnly /p:Configuration=Release
+      ./eng/build.cmd -ci -NoRestore /p:RestoreForTestAssets=false /p:RestoreForDelayedProjects=false ...


The documentation references properties RestoreForTestAssets and RestoreForDelayedProjects that do not appear to exist in the codebase. This example command would not work as documented. Consider updating the documentation to reflect the actual properties used, which is CentralizedRestore based on the implementation in build.ps1 and the condition checks in BuildAfterTargetingPack.csproj and FunctionalTestWithAssets.targets.

Suggested change

./eng/build.cmd -ci -NoRestore /p:RestoreForTestAssets=false /p:RestoreForDelayedProjects=false ...

./eng/build.cmd -ci -NoRestore /p:CentralizedRestore=false ...

ViktorHofer · 2026-01-15T15:39:38Z

These different approaches/workarounds that result in parallel NuGet restore operations feel hacky. Yes, aspnetcore is special as it needs to support restoring exotic project types (.javaproj / .nodeproj) and so NuGet doesn't work well with that inbox but I think there should be a better solution.

I remember last year when I looked into this a bit, I was exploring if aspnetcore's build could just be a Traversal project like in runtime to avoid the parallel restore nodes. That then resulted in difficulties with those other project types which I hadn't had time to take a closer look at and fix.

Just my 5c

wtgodbe · 2026-01-15T21:34:09Z

src/BuildAfterTargetingPack/BuildAfterTargetingPack.csproj

      Returns="@(TargetPathWithTargetPlatformMoniker)">
    <MSBuild Projects="@(RequiresDelayedBuild)"
-        BuildInParallel="$(BuildInParallel)"
+        BuildInParallel="$(_RestoreInParallel)"


This project is the source of most of the issues - I think the rest of the custom stuff here may be overkill, but I'd bet that just setting this to false would buy us quite a bit of reliability. I actually just merged #65040 earlier today, which is effectively the same thing but less complete. Maybe we revert my PR & just set false here?

Actually, maybe we just pass RestoreDisableParallel as an AdditionalProperty here instead

Thank you for ideas. I was not aware of the other PR, let's maybe first wait for a while and see how good it works, then I can come back here if needed and try to work on improving this attempt.

…ipping/) that does not exist yet during the centralized restore.

Try avoiding NuGet parallel restore race conditions

31e9ddc

github-actions bot added the area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework label Jan 15, 2026

ilonatommy self-assigned this Jan 15, 2026

attribute expects a boolean not a string

18e8558

ilonatommy marked this pull request as ready for review January 15, 2026 14:14

ilonatommy requested review from a team and wtgodbe as code owners January 15, 2026 14:14

ilonatommy requested review from ViktorHofer and Copilot January 15, 2026 14:14

Copilot started reviewing on behalf of ilonatommy January 15, 2026 14:15 View session

Copilot AI reviewed Jan 15, 2026

View reviewed changes

ilonatommy marked this pull request as draft January 15, 2026 15:07

Use -CentralizedRestore when running ./eng/build.cmd.

cac0fcd

wtgodbe reviewed Jan 15, 2026

View reviewed changes

Fix: project depends on local NuGet feed (artifacts/packages/Debug/Sh…

c0860a4

…ipping/) that does not exist yet during the centralized restore.

wtgodbe mentioned this pull request Jan 16, 2026

Disable parallel restore for BuildAfterTargetingPack projects #65107

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add -CentralizedRestore to avoid NuGet parallel restore race conditions#65069

Add -CentralizedRestore to avoid NuGet parallel restore race conditions#65069
ilonatommy wants to merge 4 commits intodotnet:mainfrom
ilonatommy:centralized-restore

ilonatommy commented Jan 15, 2026 •

edited

Loading

Uh oh!

ilonatommy commented Jan 15, 2026

Uh oh!

azure-pipelines bot commented Jan 15, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Jan 15, 2026

Uh oh!

Copilot AI Jan 15, 2026

Uh oh!

Copilot AI Jan 15, 2026

Uh oh!

ViktorHofer commented Jan 15, 2026 •

edited

Loading

Uh oh!

wtgodbe Jan 15, 2026

Uh oh!

wtgodbe Jan 15, 2026

Uh oh!

ilonatommy Jan 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

	<Exec Command="dotnet restore "$(RepoRoot)AspNetCore.slnx"" />
	<Exec Command="dotnet restore "$(RepoRoot)AspNetCore.slnx" --disable-parallel" />

	./eng/build.cmd -ci -NoRestore /p:RestoreForTestAssets=false /p:RestoreForDelayedProjects=false ...
	./eng/build.cmd -ci -NoRestore /p:CentralizedRestore=false ...

Conversation

ilonatommy commented Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Avoid "Cannot create a file when that file already exists" on CI

Description

Details

Performance impact

Does it fix the issue?

Uh oh!

ilonatommy commented Jan 15, 2026

Uh oh!

azure-pipelines bot commented Jan 15, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

ViktorHofer commented Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wtgodbe Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

wtgodbe Jan 15, 2026

Choose a reason for hiding this comment

Uh oh!

ilonatommy Jan 16, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ilonatommy commented Jan 15, 2026 •

edited

Loading

ViktorHofer commented Jan 15, 2026 •

edited

Loading