Mono toolchain workload #51327

steveisok · 2021-04-15T19:25:58Z

First pass at a more comprehensive mono workload that includes wasm, the aot compilers, and the runtime config task.

We were originally publishing the MonoAOTCompiler nuget and having Blazor, iOS, and Android include it as part of their workload. Unfortunately, the workload spec does not allow multiple pack references from different manifests. To get around this 1-1 relationship, we can supply a containing workload that can be consumed via the extends key. Resolves dotnet/sdk#16700

…load

lewing

Added some thoughts. More to come

src/mono/nuget/Microsoft.NET.Sdk.Mono.Toolchain.Manifest/WorkloadManifest.targets

src/mono/nuget/Microsoft.NET.Sdk.Mono.Toolchain.Manifest/WorkloadManifest.json.in

marek-safar · 2021-04-16T07:53:05Z

src/mono/nuget/Directory.Build.props

  <Import Project="$(NuGetPackageRoot)\microsoft.dotnet.build.tasks.packaging\$(MicrosoftDotNetBuildTasksPackagingVersion)\build\Microsoft.DotNet.Build.Tasks.Packaging.props" />

+  <PropertyGroup>
+    <WorkloadTasksAssemblyPath>$([MSBuild]::NormalizeDirectory('$(ArtifactsBinDir)', 'WorkloadBuildTasks', 'Debug', '$(NetCoreAppToolCurrent)'))WorkloadBuildTasks.dll</WorkloadTasksAssemblyPath>


Why Debug ?

@akoeplinger Do you recall why it's always debug?

We always build all tasks in debug. I believe the reason for that was that one of the subsets modified the configuration property which caused the tasks to not build correctly in mixed configurations. Also hardcoding to one configuration avoids a rebuild when you switch configurations when building the repo.

src/mono/nuget/Microsoft.NET.Sdk.Mono.Toolchain.Manifest/WorkloadManifest.targets

.../Microsoft.NET.Sdk.Mono.Toolchain.Manifest/Microsoft.NET.Sdk.Mono.Toolchain.Manifest.pkgproj

SamMonoRT · 2021-04-19T18:51:08Z

cc @grendello from Android pov

grendello · 2021-04-19T19:22:16Z

cc @jonathanpeppers, @dellis1972

src/mono/nuget/Microsoft.NET.Sdk.Mono.Toolchain.Manifest/WorkloadManifest.json.in

… blazor

src/mono/nuget/Microsoft.NET.Sdk.Mono.Toolchain.Manifest/WorkloadManifest.json.in

mhutch · 2021-04-29T02:54:31Z

Regarding b40c49b, are you assuming that the installer/resolver will fall back to osx-x64 packs on osx-arm64 hosts?

mhutch · 2021-04-29T02:56:06Z

I would recommend marking all these workloads as abstract so they're only able to be installed via the concrete platform SDK workloads that extend them.

steveisok · 2021-04-29T10:22:34Z

Regarding b40c49b, are you assuming that the installer/resolver will fall back to osx-x64 packs on osx-arm64 hosts?

After a bunch of back and forth, we'll be explicit about what happens on osx-arm64.

lewing

Looks good enough to start producing packs.

lewing · 2021-05-03T17:52:29Z

Regarding b40c49b, are you assuming that the installer/resolver will fall back to osx-x64 packs on osx-arm64 hosts?

After a bunch of back and forth, we'll be explicit about what happens on osx-arm64.

I tested the aliasing we're using now with an arm64 runtime on an m1 and produced functional AOT builds

ViktorHofer · 2021-05-04T08:41:17Z

This change broke our official builds:

D:\workspace\_work\1\s\src\installer\prepare-artifacts.proj(126,5): error : Item to sign 'Microsoft.NET.Sdk.Mono.Toolchain.Manifest.6.0.0-preview.5.21253.8.nupkg' was not found in the artifacts
##[error]src\installer\prepare-artifacts.proj(126,5): error : (NETCORE_ENGINEERING_TELEMETRY=Build) Item to sign 'Microsoft.NET.Sdk.Mono.Toolchain.Manifest.6.0.0-preview.5.21253.8.nupkg' was not found in the artifacts

https://dev.azure.com/dnceng/internal/_build/results?buildId=1119221&view=logs&j=4d50a8bf-a143-51c7-5cc8-defff437e23b&t=0b0b242f-bbcb-57b5-fe9f-26dc042642ec&l=200

akoeplinger · 2021-05-04T14:38:09Z

#52247

ViktorHofer · 2021-05-04T19:10:38Z

eng/Versions.props

    -->
    <NETStandardPatchVersion>0</NETStandardPatchVersion>
+    <MicrosoftNETRuntimeEmscripten2012Nodewinx64Version>6.0.0-preview.4.21212.1</MicrosoftNETRuntimeEmscripten2012Nodewinx64Version>
+    <MicrosoftNETRuntimeEmscriptenVersion>$(MicrosoftNETRuntimeEmscripten2012Nodewinx64Version)</MicrosoftNETRuntimeEmscriptenVersion>


Can you please move these two properties down to the other property group and add a comment to indicate which repo these properties are targeting? Similar to all the other properties in this file (see below).

ViktorHofer · 2021-05-04T19:11:33Z

src/tasks/WorkloadBuildTasks/WorkloadBuildTasks.csproj

+<Project Sdk="Microsoft.NET.Sdk">
+  <PropertyGroup>
+    <TargetFramework>$(NetCoreAppToolCurrent)</TargetFramework>
+    <OutputType>Library</OutputType>


nit: you can remove this property. It's the default.

ViktorHofer · 2021-05-04T19:12:47Z

src/tasks/WorkloadBuildTasks/WorkloadBuildTasks.csproj

+  <Target Name="PublishBuilder"
+          AfterTargets="Build"
+          DependsOnTargets="Publish" />


Why do you need to publish the task?

ViktorHofer · 2021-05-04T19:14:22Z

src/mono/nuget/Microsoft.NET.Sdk.Mono.Toolchain.Manifest/WorkloadManifest.targets

@@ -0,0 +1,46 @@
+<Project>


If this is a shipping package then it needs the copyright section: https://github.com/dotnet/arcade/blob/cc512d28c9368336a2893172cbd45dc27960e304/src/Microsoft.DotNet.Build.Tasks.TargetFramework.Sdk/src/build/Microsoft.DotNet.Build.Tasks.TargetFramework.Sdk.props#L1

ViktorHofer · 2021-05-04T19:16:59Z

.../Microsoft.NET.Sdk.Mono.Toolchain.Manifest/Microsoft.NET.Sdk.Mono.Toolchain.Manifest.pkgproj

+  <Import Project="$([MSBuild]::GetPathOfFileAbove(Directory.Build.props))" />
+
+  <PropertyGroup>
+    <PackageDescription>Internal toolchain package not meant for direct consumption. Please do not reference directly.</PackageDescription>


Consider setting <UseRuntimePackageDisclaimer>true</UseRuntimePackageDisclaimer> which adds the standard description for internal packages that we use everywhere else:

runtime/Directory.Build.targets

Line 54 in cd271c1

<PackageDescription Condition="'$(PackageDescription)' == '' and '$(UseRuntimePackageDisclaimer)' == 'true'">$(RuntimePackageDisclaimer)</PackageDescription>

.

ViktorHofer · 2021-05-04T19:18:12Z

src/mono/nuget/Directory.Build.props

  <Import Project="$(NuGetPackageRoot)\microsoft.dotnet.build.tasks.packaging\$(MicrosoftDotNetBuildTasksPackagingVersion)\build\Microsoft.DotNet.Build.Tasks.Packaging.props" />

+  <PropertyGroup>
+    <WorkloadTasksAssemblyPath>$([MSBuild]::NormalizeDirectory('$(ArtifactsBinDir)', 'WorkloadBuildTasks', 'Debug', '$(NetCoreAppToolCurrent)'))WorkloadBuildTasks.dll</WorkloadTasksAssemblyPath>


Use NormalizePath instead. NormalizeDirectory is intended to be used for paths to directories where-as NormalizePath is for file paths.

ViktorHofer · 2021-05-04T19:21:29Z

src/mono/nuget/Microsoft.NET.Sdk.Mono.Toolchain.Manifest/WorkloadManifest.targets

+        <Import Project="Sdk.props" Sdk="Microsoft.NET.Runtime.Emscripten.Node" />
+        <Import Project="Sdk.props" Sdk="Microsoft.NET.Runtime.Emscripten.Sdk" />
+        <Import Project="Sdk.props" Sdk="Microsoft.NETcore.App.Runtime.AOT.Cross.browser-wasm" />
+    </ImportGroup>


nit: indentations in this file are off. We use two spaces in all our msbuild files in the repo.

Steve Pfister added 5 commits April 7, 2021 17:48

Make sure the pack versions correctly

8ec3266

Merge remote-tracking branch 'upstream/main' into mono-toolchain-work…

d090f3b

…load

Merge remote-tracking branch 'upstream/main' into mono-toolchain-work…

2c0d6ab

…load

More comprehensive mono workload

0be9877

steveisok requested a review from marek-safar as a code owner April 15, 2021 19:25

ghost added the area-Build-mono label Apr 15, 2021

steveisok requested review from akoeplinger and lewing April 15, 2021 19:26

lewing reviewed Apr 15, 2021

View reviewed changes

src/mono/nuget/Microsoft.NET.Sdk.Mono.Toolchain.Manifest/WorkloadManifest.targets Outdated Show resolved Hide resolved

src/mono/nuget/Microsoft.NET.Sdk.Mono.Toolchain.Manifest/WorkloadManifest.targets Outdated Show resolved Hide resolved

lewing force-pushed the mono-toolchain-workload branch from b273dbd to 13827a7 Compare April 16, 2021 00:12

alias the simulators by host platform

70cc8bc

spouliot reviewed Apr 16, 2021

View reviewed changes

src/mono/nuget/Microsoft.NET.Sdk.Mono.Toolchain.Manifest/WorkloadManifest.json.in Outdated Show resolved Hide resolved

Simplify the workload definitions and remove the linux python alias

d1e0623

lewing force-pushed the mono-toolchain-workload branch from 13827a7 to d1e0623 Compare April 16, 2021 00:36

lewing added 2 commits April 15, 2021 19:47

Remove old simulator package and fix json

8b0ee55

Remove tvos-x64

5c03c7d

marek-safar reviewed Apr 16, 2021

View reviewed changes

Steve Pfister added 4 commits April 16, 2021 10:04

Separate wasm imports from wasm aot imports

c87b5cb

Add browser runtime pack

0df4d96

Added android, ios/tvos runtime packs

a7428da

Switched runtime pack kind to framework

0ab081c

fanyang-mono mentioned this pull request Apr 19, 2021

Add RuntimeConfigParserTask to the Mono workload pack #50906

Closed

jonathanpeppers reviewed Apr 19, 2021

View reviewed changes

src/mono/nuget/Microsoft.NET.Sdk.Mono.Toolchain.Manifest/WorkloadManifest.json.in Outdated Show resolved Hide resolved

lewing added 3 commits April 19, 2021 16:29

Attempt to resolve some more issues

eb87e8f

Enable browser workload by default when it is installed

8540db7

Require UsingBrowserRuntimeWorkload when targeting browser outside of…

8578a15

… blazor

lewing reviewed Apr 20, 2021

View reviewed changes

src/mono/nuget/Microsoft.NET.Sdk.Mono.Toolchain.Manifest/WorkloadManifest.json.in Outdated Show resolved Hide resolved

lewing reviewed Apr 22, 2021

View reviewed changes

src/mono/nuget/Microsoft.NET.Sdk.Mono.Toolchain.Manifest/WorkloadManifest.json.in Outdated Show resolved Hide resolved

src/mono/nuget/Microsoft.NET.Sdk.Mono.Toolchain.Manifest/WorkloadManifest.json.in Outdated Show resolved Hide resolved

Steve Pfister and others added 4 commits April 22, 2021 16:09

Add back arm64 cross simulator alias

01f42a6

x86 too

953d64d

Remove more aliases and require cross compiler for browser

da98134

Fix trailing commas

24f2bca

lambdageek reviewed Apr 29, 2021

View reviewed changes

src/mono/nuget/Microsoft.NET.Sdk.Mono.Toolchain.Manifest/WorkloadManifest.json.in Outdated Show resolved Hide resolved

mhutch reviewed Apr 29, 2021

View reviewed changes

src/mono/nuget/Microsoft.NET.Sdk.Mono.Toolchain.Manifest/WorkloadManifest.json.in Outdated Show resolved Hide resolved

lewing force-pushed the mono-toolchain-workload branch from 587c552 to 2b7d364 Compare April 29, 2021 03:28

add osx-arm64 aliasing back

921f982

lewing force-pushed the mono-toolchain-workload branch from 2b7d364 to 921f982 Compare April 29, 2021 03:38

Steve Pfister added 2 commits April 29, 2021 06:27

Tweak workload name and make a proper ios x86 simulator pack

8fd3827

Split out iossimulator x86 on its own

c4d1118

lewing approved these changes May 3, 2021

View reviewed changes

steveisok merged commit b31bcc3 into dotnet:main May 3, 2021

akoeplinger mentioned this pull request May 4, 2021

Download Mono AOT workload .nupkg in prepare-artifacts.proj #52247

Merged

ViktorHofer reviewed May 4, 2021

View reviewed changes

karelz added this to the 6.0.0 milestone May 20, 2021

ghost locked as resolved and limited conversation to collaborators Jun 19, 2021

Mono toolchain workload #51327

Mono toolchain workload #51327

Uh oh!

Conversation

steveisok commented Apr 15, 2021

Uh oh!

lewing left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

SamMonoRT commented Apr 19, 2021

Uh oh!

grendello commented Apr 19, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mhutch commented Apr 29, 2021

Uh oh!

mhutch commented Apr 29, 2021

Uh oh!

steveisok commented Apr 29, 2021

Uh oh!

lewing left a comment

Choose a reason for hiding this comment

Uh oh!

lewing commented May 3, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ViktorHofer commented May 4, 2021

Uh oh!

akoeplinger commented May 4, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ViktorHofer May 4, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

13 participants

lewing commented May 3, 2021 •

edited

Loading

ViktorHofer May 4, 2021 •

edited

Loading