Ensure that Trimming and AOT scenarios trigger downloading of necessary runtime packs#51765
Conversation
…also bring in the runtime packs when targeting a single RID. This fixes a gap where 'dotnet restore' for a RID followed by 'dotnet publish --no-restore' for that RID in trimming/AOT scenarios wouldn't work.
| <ItemGroup> | ||
| <ProjectCapability Remove="TestContainer" /> | ||
| </ItemGroup> |
There was a problem hiding this comment.
Oh, this is a bugfix from the DevKit folks (thanks @peterwald!) that unblocks the DevKit test explorer experience.
Because this project references xunit (thanks Arcade!) the xunit package adds this capability. Because the project has this capability, DevKit tries to load it as a test. Because this project is a library and doesn't have all of the test dependencies, it fails to load and crashes the test-discovery process.
There was a problem hiding this comment.
@baronfel xunit isn't brought by Arcade here. It's added by https://github.com/baronfel/sdk/blob/6948352a75a507040529c6eaf9a39bd6d95ac35b/test/Microsoft.NET.TestFramework/Microsoft.NET.TestFramework.csproj#L54
Instead of removing TestContainer, the project must not reference xunit in the first place. Instead, it should reference the pieces it needs from this list:
- xunit.assert
- xunit.extensibility.core
- xunit.extensibility.execution
There was a problem hiding this comment.
Pull Request Overview
This pull request fixes a gap in runtime pack resolution where projects using PublishTrimmed or PublishAot without SelfContained=true would fail during publish with --no-restore because required runtime packs weren't downloaded during restore.
Key Changes:
- Introduced
DeploymentModelRequiresRuntimeComponentsproperty that consolidates logic for determining when runtime packs are needed, now includingRequiresILLinkPack(set when trimming is enabled) alongsideSelfContained,ReadyToRunEnabled, andPublishAot - Refactored
KnownRuntimePackstruct to use C# 12 primary constructor and readonly properties for better maintainability - Added comprehensive unit tests covering various publish scenarios including trimming and AOT without self-contained deployment
- Created documentation specification detailing expected behavior of
ProcessFrameworkReferencestask across different deployment scenarios
Reviewed Changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| src/Tasks/Microsoft.NET.Build.Tasks/ProcessFrameworkReferences.cs | Core fix to runtime pack resolution logic; refactored struct to use modern C# features; introduced constants and helper properties for better code clarity |
| src/Tasks/Microsoft.NET.Build.Tasks.UnitTests/ProcessFrameworkReferencesTests.cs | Added comprehensive test coverage for PublishTrimmed, PublishAot, and ReadyToRun scenarios with and without SelfContained; added support for ReadyToRunEnabled in test configuration |
| documentation/specs/ProcessFrameworkReferences-outputs.md | New specification document describing expected outputs for different deployment scenarios and documenting the fix for issue #51667 |
| test/Microsoft.NET.TestFramework/Microsoft.NET.TestFramework.csproj | Removed TestContainer project capability to prevent VS from treating the framework project as a test container |
| tasks.slnf | New solution filter for convenient development focused on build tasks projects |
src/Tasks/Microsoft.NET.Build.Tasks/ProcessFrameworkReferences.cs
Outdated
Show resolved
Hide resolved
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
| /// <summary> | ||
| /// We have several deployment models that require the use of runtime assets of various kinds. | ||
| /// This member helps identify when any of those models are in use, because we've had bugs in the past | ||
| /// where we didn't properly account for all of them. | ||
| /// </summary> | ||
| private bool DeploymentModelRequiresRuntimeComponents => | ||
| SelfContained || ReadyToRunEnabled || PublishAot || RequiresILLinkPack; |
There was a problem hiding this comment.
This member is the actual fix. Every other change is tests, infra, debugging utility, clarity, etc.
|
Nit: per #51766 (comment) |
| <SelfContained>true</SelfContained> | ||
| <RuntimeIdentifier>linux-x64</RuntimeIdentifier> | ||
| <ReadyToRunEnabled>true</ReadyToRunEnabled> | ||
| <ReadyToRunUseCrossgen2>true</ReadyToRunUseCrossgen2> |
There was a problem hiding this comment.
Is there a minimum runtime pack version that the current .NET SDK is compatible with? Does the current .NET SDK still support publishing for .NET 5 in all flavors?
UseCrossgen2 property was a thing in .NET 5 where crossgen2 was in preview and setting this property to true was used to opt-in into the preview. It has no purpose (it is always true) in .NET 6+.
There was a problem hiding this comment.
The SDK does still support publishing net5 apps - though you get a warning that that runtime is out of support. The property is mostly mentioned here for completeness given that it does play into the runtime pack selection in the processframeworkreferences task today - I didn't want a reader to read this and then see an entirely new concept in the implementation and wonder if we missed something critical.
There was a problem hiding this comment.
The SDK does still support publishing net5 apps
Do you have a deprecation schedule for when .NET SDK is going to drop support for runtimes that are out of support for a long time?
I do not think it is a good use of resources to carry this baggage around, and the support is likely broken in number of ways anyway.
There was a problem hiding this comment.
Why currently have no timeline for this. The general approach has been to allow everything, but only explicitly test 2 LTSs back. So for example net 6 is the lower bound on a lot of our tests.
There was a problem hiding this comment.
Why currently have no timeline for this. The general approach has been to allow everything, but only explicitly test 2 LTSs back. So for example net 6 is the lower bound on a lot of our tests.
There was a problem hiding this comment.
If we remove testing, we have to assume that the functionality for that targeting will degrade from release to release and we're ok with that.
It's not supported today but we've held off taking any action as we didn't want to actively break customers.
What value does this provide? It builds false trust with customer that we will eventually break. In theory we could make a change and unintentionally break a "major" scenario for an EOL RID. This is saying we are ok with that. Let's be proactive in keeping the codebase clean and free of legacy baggage in support of quicker innovation.
There was a problem hiding this comment.
It’s worth spelling out what the alternative looks like in these cases: users will need to download old SDKs to publish for old frameworks. I think that’s a desirable state to be in. If a customer were to ask me how they should continue building for netcoreapp3.1, knowing it is long out of support, that is what I would recommend. In the case where no testing is run and nothing is being supported, I would prioritize no changes over all else. As a netcoreapp3.1 user, no changes being made for net10 would be worth the risk.
There was a problem hiding this comment.
Yes. That's the proposal. I gather this is what we're going to do. Retain the ability to acquire and build not test it.
Proposal for .NET 11 SDK:
net10.0+ -- supportednet8.0+ -- testednet1.0+ -- mechanical enabled (will download TP and should work)
There was a problem hiding this comment.
Andy is pointing out that he would not ever recommend to a customer to depend on net1.0+ mechanical enablement in .NET 11 SDK. I would do the same. It does not form a coherent story.
There was a problem hiding this comment.
Sorry. I missed a key point in his first sentence. Yes, that sounds best. In any case, we agree on the proposed end point.
…n't explicitly set to true until ILLink targets are updated
| <!-- The ILLink Targets expect projects to set SelfContained specifically, but the real condition they're trying to detect is | ||
| 'does this publish scenario require runtime components'. SelfContained is just the 'legacy' signal for that. | ||
| See https://github.com/dotnet/sdk/pull/51765#issuecomment-3537080089 for more context. | ||
|
|
||
| To work around this current constraint, we can detect the scenarios where runtime components are required | ||
| and set SelfContained accordingly before ILLink runs. It's ok to set SelfContained this 'late' because | ||
| the ProcessFrameworkReferences Task (which is where runtime packs are resolved) has already run by this | ||
| point and it has been updated to use this 'expanded' definition of when runtime components are required | ||
| already (e.g. the ProcessFrameworkReferences.DeploymentModelRequiresRuntimeComponents member). | ||
|
|
||
| The _RequiresILLinkPack property is set based on this expanded definition, so we can reuse it as our signal here. | ||
| It's possible that this property is what the ILLink targets should be using directly instead of SelfContained, but | ||
| we may want to have more 'abstract', non-private signal. | ||
| --> | ||
| <Target Name="_AlignSelfContainedBeforeILLink" BeforeTargets="PrepareForPublish" Condition=""> | ||
| <PropertyGroup> | ||
| <SelfContained Condition="'$(SelfContained)' != 'true' and '$(_RequiresILLinkPack)' == 'true'">true</SelfContained> | ||
| </PropertyGroup> | ||
| </Target> |
There was a problem hiding this comment.
I've added a target to the SDK that 'patches' the SelfContained status so that we can demonstrate end-to-ends - we should have a conversation on what the ILLink Targets should be conditioning on instead of SelfContained directly.
|
Is that dotnet/runtime issue from 2021 what you intended to link to? I do expect that the extra care you are providing them 4 years later is much appeciated. |
|
@jjonescz @baronfel There are a few test failures remaining here with dotnet run file.cs, probably because we are setting a PublishRuntimeIdentifier by default in some cases where we weren't before. I will probably need some help understanding how to handle this and whether this is a breaking change we need to fix or just a test issue. |
|
I figured out what the issue with the file run tests was. Now there are just a few failures on Linux, maybe because NatievAOT cross-compilation isn't supported there. |
@dsplaisted I'm curious what the failure was. I don't see any run-file tests failing in the build run https://dev.azure.com/dnceng-public/public/_build/results?buildId=1239391&view=ms.vss-test-web.build-test-results-tab for 1cc4b24 |
Looking back, I guess it wasn't actually a run file test that was failing, it was a single-file publish test and I misinterpreted the test name. The test was |
|
@agocke Under what OS/Architecture combinations is NativeAOT cross-publishing supported? Based on the test results, it looks like cross-architecture NativeAOT works on Windows but not Linux. |
|
@dsplaisted There is no super-simple answer to that question. In general, cross-OS publishing is not supported. But there are exceptions. For example, publishing for Android will be supported from Windows, and publishing for iOS is supported from MacOS. The constraint is not on our side, it is on the native toolchain side. If you have a compatible native toolchain installed, all options are supported. |
|
/azp run |
|
Azure Pipelines successfully started running 2 pipeline(s). |
| <PropertyGroup Condition="'$(PublishRuntimeIdentifier)' == '' And | ||
| '$(RuntimeIdentifier)' == '' And | ||
| '$(_IsExecutable)' == 'true' And | ||
| ( | ||
| '$(PublishSelfContained)' == 'true' or | ||
| '$(PublishReadyToRun)' == 'true' or | ||
| '$(PublishTrimmed)' == 'true' or | ||
| '$(PublishSingleFile)' == 'true' or | ||
| '$(PublishAot)' == 'true' | ||
| ) And | ||
| !('$(PackAsTool)' == 'true' And '$(_IsPacking)' == 'true')"> |
| } | ||
|
|
||
| private struct KnownRuntimePack | ||
| [DebuggerDisplay("{Name}@{LatestRuntimeFrameworkVersion} for {TargetFramework}")] |
There was a problem hiding this comment.
I've never used this attribute before. What does it do?
There was a problem hiding this comment.
it's awesome - it lets you customize the way an object is rendered in the debugger only. You can still expand it to view properties, but often a quick DebuggerDisplay annotation can give you everything you need at a glance.
src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.RuntimeIdentifierInference.targets
Outdated
Show resolved
Hide resolved
…imeIdentifierInference.targets Co-authored-by: Michael Yanni <[email protected]>
|
/backport to release/10.0.2xx |
|
Started backporting to |
|
@dsplaisted backporting to git am output$ git am --3way --empty=keep --ignore-whitespace --keep-non-patch changes.patch
Applying: create MSBuild-Task-specific slnf to make it easier to work on tasks in VSCode
Using index info to reconstruct a base tree...
Falling back to patching base and 3-way merge...
No changes -- Patch already applied.
Applying: scaffold out test scenarios to demonstrate gaps
.git/rebase-apply/patch:75: trailing whitespace.
.git/rebase-apply/patch:77: trailing whitespace.
"Microsoft.NETCore.App.Runtime.**RID**",
.git/rebase-apply/patch:79: trailing whitespace.
.git/rebase-apply/patch:83: trailing whitespace.
.git/rebase-apply/patch:89: trailing whitespace.
warning: squelched 28 whitespace errors
warning: 33 lines add whitespace errors.
Using index info to reconstruct a base tree...
M test/Microsoft.NET.TestFramework/Microsoft.NET.TestFramework.csproj
Falling back to patching base and 3-way merge...
Auto-merging test/Microsoft.NET.TestFramework/Microsoft.NET.TestFramework.csproj
Applying: extract out spec for behavior of ProcessFrameworkReferences
Applying: Update ProcessFrameworkReferences so that trimming and AOT scenarios also bring in the runtime packs when targeting a single RID.
Applying: Update src/Tasks/Microsoft.NET.Build.Tasks/ProcessFrameworkReferences.cs
Applying: Update documentation/specs/ProcessFrameworkReferences-outputs.md
Applying: Update documentation/specs/ProcessFrameworkReferences-outputs.md
Applying: patch gap in publish-for-specific-rid scenario where SelfContained isn't explicitly set to true until ILLink targets are updated
Applying: fix some tests by updating expectations for the new world
Using index info to reconstruct a base tree...
M test/Microsoft.NET.TestFramework/ProjectConstruction/TestProject.cs
Falling back to patching base and 3-way merge...
Applying: reduce the blast-radius of the selfcontained-assignment
Applying: make the SelfContained-fixup task correctly override earlier SDK assumptions
Applying: remove obsolete tests
Applying: Update test/Microsoft.NET.TestFramework/Microsoft.NET.TestFramework.csproj
Using index info to reconstruct a base tree...
M test/Microsoft.NET.TestFramework/Microsoft.NET.TestFramework.csproj
Falling back to patching base and 3-way merge...
Auto-merging test/Microsoft.NET.TestFramework/Microsoft.NET.TestFramework.csproj
CONFLICT (content): Merge conflict in test/Microsoft.NET.TestFramework/Microsoft.NET.TestFramework.csproj
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
hint: When you have resolved this problem, run "git am --continue".
hint: If you prefer to skip this patch, run "git am --skip" instead.
hint: To restore the original branch and stop patching, run "git am --abort".
hint: Disable this message with "git config set advice.mergeConflict false"
Patch failed at 0013 Update test/Microsoft.NET.TestFramework/Microsoft.NET.TestFramework.csproj
Error: The process '/usr/bin/git' failed with exit code 128 |
…alue.
This adds an escape hatch (setting 'UseDefaultPublishRuntimeIdentifier=false') when trying to set a default PublishRuntimeIdentifier value.
This is necessary when publishing with RuntimeIdentifiers (plural), but without a RuntimeIdentifier, which is valid when building universal apps for macOS and Mac Catalyst (the netX-macos and netX-maccatalyst target frameworks).
When building such an app, the project file will set RuntimeIdentifiers (plural):
<TargetFramework>net11.0-macos</TargetFramework>
<RuntimeIdentifiers>osx-x64;osx-arm</RuntimeIdentifiers>
and then during build/publish, the macOS SDK will run two inner builds, with RuntimeIdentifiers unset, and RuntimeIdentifier set to each of the rids (and at the end merge the result into a single app).
The problem is that the outer build, where RuntimeIdentifiers is set, but RuntimeIdentifier isn't, PublishRuntimeIdentifier will now get a default value (after PR dotnet#51765), and that will set RuntimeIdentifier, which will confuse our outer build.
Also note that we can't set PublishRuntimeIdentifier to the desired runtime identifiers (plural), because PublishRuntimeIdentifier is only valid for a single runtime identifier.
…alue.
This adds an escape hatch (setting 'UseDefaultPublishRuntimeIdentifier=false') when trying to set a default PublishRuntimeIdentifier value.
This is necessary when publishing with RuntimeIdentifiers (plural), but without a RuntimeIdentifier, which is valid when building universal apps for macOS and Mac Catalyst (the netX-macos and netX-maccatalyst target frameworks).
When building such an app, the project file will set RuntimeIdentifiers (plural):
<TargetFramework>net11.0-macos</TargetFramework>
<RuntimeIdentifiers>osx-x64;osx-arm</RuntimeIdentifiers>
and then during build/publish, the macOS SDK will run two inner builds, with RuntimeIdentifiers unset, and RuntimeIdentifier set to each of the rids (and at the end merge the result into a single app).
The problem is that the outer build, where RuntimeIdentifiers is set, but RuntimeIdentifier isn't, PublishRuntimeIdentifier will now get a default value (after PR #51765), and that will set RuntimeIdentifier, which will confuse our outer build.
Also note that we can't set PublishRuntimeIdentifier to the desired runtime identifiers (plural), because PublishRuntimeIdentifier is only valid for a single runtime identifier.

Fixes #51667, #51766, #50093 and likely many others.
There was a gap when restoring a project for a RID and then publishing it for that same RID with
--no-restorewhere if that project only set PublishTrimmed or PublishAOT, the runtime packs for that RID would not be restored. This would lead to the publish command failing saying that required runtime packs weren't available. There were a couple workarounds:Along the way I added some tests to cover the new scenarios, and extracted out a documentation spec to capture how we think ProcessFrameworkReferences should behave, modulo any bugs. In the future we should be able to reference this when answering questions about what the expected behavior is.
I also made a
tasks.slnfto make it easier to iterate on the SDK's MSBuild Tasks and their tests in isolation. This massively helps the VSCode user experiences, especially on the testing inner loop.