Skip to content

Remove usages of AllRepoVersions.props#19317

Merged
jkoritzinsky merged 6 commits intodotnet:mainfrom
jkoritzinsky:no-all-repo-versions
Apr 6, 2024
Merged

Remove usages of AllRepoVersions.props#19317
jkoritzinsky merged 6 commits intodotnet:mainfrom
jkoritzinsky:no-all-repo-versions

Conversation

@jkoritzinsky
Copy link
Member

Fixes dotnet/source-build#4036

Remove usages in Versions.props patching by using an alternative mechanism that's already in use for Microsoft.NETCore.App.

Remove usages for Arcade SDK resolution through some NuGet package inspection to discover the SDK version.

@jkoritzinsky jkoritzinsky requested review from a team as code owners April 4, 2024 23:10
@ViktorHofer
Copy link
Member

ViktorHofer commented Apr 5, 2024

<_BuiltArcadeSdkPackageVersion>$([System.String]::new('%(_BuiltArcadeSdkPackage.FileName)').Substring($([MSBuild]::Add($(_ArcadeSdkName.Length), 1))))</_BuiltArcadeSdkPackageVersion>

I think it would be better to retrieve that information from Arcade's build manifest which is already available at that time. The XmlPeek msbuild task allows that. The manifest(s) are stored here:

<RepoAssetManifestsDir>$([MSBuild]::NormalizeDirectory('$(AssetManifestsIntermediateDir)', '$(RepositoryName)'))</RepoAssetManifestsDir>

We will do the same for baseline testing the produced nuget packages but with the merged vertical manifest as an input. Here we just need the Arcade build manifest(s).

EDIT: I also just filed dotnet/source-build#4297. It would be great if you could fix the NoTargets and Traversal versions as part of this. They will need the same treatment in the new target.

All this doesn't need to be perfect as we will eventually get rid of all this when we have the necessary NuGet support.


<!-- Discover the Arcade SDKs built from the live Arcade source and set them as overrides here. -->
<Target Name="DiscoverArcadeSourceBuiltSdkOverrides"
DependsOnTargets="GetRepositoryReferenceInfo"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This target also depends on the dependent repositories being built which is a parent of GetRepositoryReferenceInfo.

Suggested change
DependsOnTargets="GetRepositoryReferenceInfo"
DependsOnTargets="BuildRepoReferences"

</Target>

<Target Name="SetSourceBuiltSdkOverrides"
Condition="'@(SourceBuiltSdkOverride)' != ''">
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would also add a DependsOnTargets on the new defined target here in case someone tests this locally and directly invokes the SetSourceBuiltSdkOverrides target.

@mmitche
Copy link
Member

mmitche commented Apr 5, 2024

<_BuiltArcadeSdkPackageVersion>$([System.String]::new('%(_BuiltArcadeSdkPackage.FileName)').Substring($([MSBuild]::Add($(_ArcadeSdkName.Length), 1))))</_BuiltArcadeSdkPackageVersion>

I think it would be better to retrieve that information from Arcade's build manifest which is already available at that time. The XmlPeek msbuild task allows that. The manifest(s) are stored here:

<RepoAssetManifestsDir>$([MSBuild]::NormalizeDirectory('$(AssetManifestsIntermediateDir)', '$(RepositoryName)'))</RepoAssetManifestsDir>

We will do the same for baseline testing the produced nuget packages but with the merged vertical manifest as an input. Here we just need the Arcade build manifest(s).

EDIT: I also just filed dotnet/source-build#4297. It would be great if you could fix the NoTargets and Traversal versions as part of this. They will need the same treatment in the new target.

All this doesn't need to be perfect as we will eventually get rid of all this when we have the necessary NuGet support.

I agree with using the merged manifests eventually but we could do it in one fell swoop and change the other PackageVersion*.props generation to work off the manifests at the same time. That said, there are still two scenarios and we unfortunately can't drive everything off of the merged manifests:

  • Generation of new PVPs or extra version props based on the output of previous repo builds - This can probably be driven by the merged manifests. One exception here may be source-build-reference-packages. /cc @dotnet/source-build-internal
  • Generation of previously SB props files based on an input directory of packages - We can't rely on the exist of merged manifests for this case.

I don't love the current path manipulation approach above but I'm fine with it as a temporary measure (please include a comment and issue link)

@jkoritzinsky
Copy link
Member Author

I have an idea of how to make this all work cleanly though asset manifest inspection (and in general update how we look at dependencies), but I'll do that in a follow-up PR.

@ViktorHofer
Copy link
Member

I have an idea of how to make this all work cleanly though asset manifest inspection (and in general update how we look at dependencies), but I'll do that in a follow-up PR.

Sounds good. Please file an issue so that we can discuss that.

Comment on lines +13 to +15
<BuiltSdkPackageOverride Include="Microsoft.DotNet.Arcade.Sdk" Version="$(OutputPackageVersion)" />
<BuiltSdkPackageOverride Include="Microsoft.DotNet.SharedFramework.Sdk" Version="$(OutputPackageVersion)" />
<BuiltSdkPackageOverride Include="Microsoft.DotNet.CMake.Sdk" Version="$(OutputPackageVersion)" />
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is OutputPackageVerison defined?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OutputPackageVersion is currently defined in the git-info/arcade.props file that's generated by the VMR. So this doesn't fully get us off the git-info folder, but it at least gets us down to the point that we don't have any cross-project usage.

I plan to remove the dependency on the git-info folder for any and all package versions when I switch to the asset manifest model.

@jkoritzinsky
Copy link
Member Author

I have an idea of how to make this all work cleanly though asset manifest inspection (and in general update how we look at dependencies), but I'll do that in a follow-up PR.

Sounds good. Please file an issue so that we can discuss that.

Filed dotnet/source-build#4302

@jkoritzinsky jkoritzinsky enabled auto-merge (squash) April 5, 2024 22:27
@jkoritzinsky jkoritzinsky merged commit 94edbe8 into dotnet:main Apr 6, 2024
@jkoritzinsky jkoritzinsky deleted the no-all-repo-versions branch April 11, 2024 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Orchestrator should pull tooling package versions from outputs of the repos rather than the AllRepoVersions.props files

3 participants