Updated patch to automatically disable crossgen2 when targeting s390x. This enables the source-build method to work as well (there's no simple way to pass the /p:CrossgenOutput=false option when using the source-build framework). In any case, crossgen2 is completely unsupported on s390x, so it makes sense to disable it.

FYI @adityamandaleeka - in addition to the big-endian patches, this PR is also required to get a successful build on s390x.

@dougbu @wtgodbe Can one of you also review this change before we merge it?

@wtgodbe I guess I was confused by this comment in eng/Dependencies.props then:

    <!-- Runtime packages required for crossgen -->
    <LatestPackageReference Include="Microsoft.NETCore.App.Runtime.win-x64" />
    <LatestPackageReference Include="Microsoft.NETCore.App.Runtime.win-x86" />
    <LatestPackageReference Include="Microsoft.NETCore.App.Runtime.win-arm" />
    <LatestPackageReference Include="Microsoft.NETCore.App.Runtime.win-arm64" />
    <LatestPackageReference Include="Microsoft.NETCore.App.Runtime.osx-x64" />
    <LatestPackageReference Include="Microsoft.NETCore.App.Runtime.osx-arm64" />
    <LatestPackageReference Include="Microsoft.NETCore.App.Runtime.linux-x64" />
    <LatestPackageReference Include="Microsoft.NETCore.App.Runtime.linux-arm" />
    <LatestPackageReference Include="Microsoft.NETCore.App.Runtime.linux-arm64" />
    <LatestPackageReference Include="Microsoft.NETCore.App.Runtime.linux-musl-x64" />
    <LatestPackageReference Include="Microsoft.NETCore.App.Runtime.linux-musl-arm" />
    <LatestPackageReference Include="Microsoft.NETCore.App.Runtime.linux-musl-arm64" />

If the App.Runtime package is indeed used for something else besides crossgen, then we can simply add the s390x version to that list and leave the reference unconditional. This means if you build aspnetcore on s390x, you'll have to provide the package via a local repository, but I think this is OK (with source-build it should work correctly anyway). Does this look reasonable to you?

"Microsoft.NETCore.App.Runtime.$(RuntimeIdentifier)" - the linux-s390x runtime is not currently built by default and therefore not available in the default repos

I'm generally concerned that this change indicates a significant gap, not just a missing crossgen2 tool.

1. Why isn't this runtime produced❔
2. Can another runtime be used when building for s390❔

"Microsoft.NETCore.App.Runtime.$(RuntimeIdentifier)" - the linux-s390x runtime is not currently built by default and therefore not available in the default repos

I'm generally concerned that this change indicates a significant gap, not just a missing crossgen2 tool.

1. Why isn't this runtime produced

2. Can another runtime be used when building for s390

We are currently working on enabling .NET support for the s390x processor architecture (IBM Z mainframe). This is a new platform which needs support throughout the various runtime & SDK components. We currently have an internal build of all those components working, and are in the process of contributing the necessary source code changes back upstream. (E.g. the "runtime" package and most of the SDK already builds out of the box.)

However, while source code changes have been merged to various dotnet components as community contributions, at this point there is no plan for Microsoft to officially support this architecture. We are working with Red Hat to enable support for s390x via the Red Hat .NET product.

Therefore, Microsoft is likewise not doing any automated binary builds of .NET components, and so they're not available in the Azure repos (like "dotnet-public" etc.). This means that if the aspnetcore build looks there for e.g. a platform-specific App.Runtime package for linux-s390x, it won't find it there.

That doesn't mean you can't build it - you just have to build the runtime source tree first, install the package in some local nuget repo, and add that when building aspnetcore. (Or use the source-build mechanism, where this should work automatically, and which is what Red Hat is using anyway.)

As to the second question, if native code out of the runtime package is needed, then no, you cannot use any other runtime instead - s390x is a different processor architecture, so native code for other runtimes will not run.

We are currently working on enabling .NET support for the s390x processor architecture (IBM Z mainframe).

…

there is no plan for Microsoft to officially support this architecture.

These two bits seem to contradict each other. If we don't need to produce an ASP.NET Core runtime for s390x, we shouldn't check anything into this repo, especially because the special cases complicate an already-complicated project.

you just have to build the runtime source tree first, install the package in some local nuget repo, and add that when building aspnetcore. (Or use the source-build mechanism, where this should work automatically, and which is what Red Hat is using anyway.)

Please why we don't either

1. use the regular "source-build mechanism" i.e. conditionalize crossgen2 for the s390x platform and add a single line to recognize and use the x390x runtime when Red Hat builds for that platform, or
2. leave the whole mess for a source-build patch that Red Hat owns.

As to the second question, if native code out of the runtime package is needed, then no, you cannot use any other runtime instead - s390x is a different processor architecture, so native code for other runtimes will not run.

Pretty sure we don't need native code from the runtime package.

@dougbu - there is the concept of "community maintained" ports. Tizen is another example of this, and we're generally supportive of this effort, so unless there are compelling technical reasons to reject this PR, I think we should take it.

Thanks, @Pilchie ! I'm happy to work on finding whatever solution is the least maintenance burden to the aspnetcore project while still allowing it to be built for s390x as a community maintained port.

@dougbu This patch was intended to address three problems I was running into:

• Disable crossgen by default when targeting s390x. This doesn't necessarily need to be handled in the main aspnetcore repo, and can instead be done via the source-build project, if this would be preferable to you. (I just thought it better to add it here given that there is no support for crossgen on s390x, so everybody building aspnetcore on the platform would have to remember to disable it anyway.)
• Do not install Microsoft.NETCore.App.Runtime.linux-s390x. The main reason why I added this is because of the comment in eng/Dependencies.props that says this package is only needed because of crossgen, so I thought it would be fine, across platforms, to not reference it when not doing crossgen. However, if the package is actually needed even without crossgen, then an alternative solution for s390x would be to simply add LatestPackageReference statements to eng/Dependencies.prop instead.
• Do not install Microsoft.NETCore.App.Crossgen2.linux-s390x. This really doesn't exist on s390x, so it will have to be disabled somehow. I'd hoped making that reference conditional on CrossgenOutput would be OK in any case.

As far as I can see, the last two points really need (small) changes to the aspnetcore sources. The only alternative would be to make those changes via patches carried permanently in source-build, but that doesn't really seem preferable overall ...

Pretty sure we don't need native code from the runtime package.

I guess I'm still confused what the platform-specific runtime package is used for then (in the absence of crossgen)? @wtgodbe you mentioned the SharedFx framework, and I do indeed see target runtime code there, but unless I'm missing something this is installed via _DownloadAndExtractDotNetRuntime, which uses the DotNetRuntimeDownloadUrl tarball and not the nuget package?

unless there are compelling technical reasons to reject this PR, I think we should take it

👍

if the package is actually needed

I suspect it's not actually needed except when using crossgen2. But, the code added at https://github.com/dotnet/aspnetcore/pull/35916/files#diff-eb8fd31a7971d712606a187ea074d0bd5d6ea85aae5a707107fd998216986ea5R46 looks iffy at best. Probably need to disable $(CrossgenOutput) based only on the s390x platform. If that works together w/ the @(Reference) conditions, we're good.

unless there are compelling technical reasons to reject this PR, I think we should take it

+1

Thanks!

if the package is actually needed

I suspect it's not actually needed except when using crossgen2. But, the code added at https://github.com/dotnet/aspnetcore/pull/35916/files#diff-eb8fd31a7971d712606a187ea074d0bd5d6ea85aae5a707107fd998216986ea5R46 looks iffy at best. Probably need to disable $(CrossgenOutput) based only on the s390x platform.

I don't quite understand what you're asking me to do here, sorry. I thought the code I'm adding in line 46 there does disable $(CrossgenOutput) on s390x only (by default, can still be overridden manually). Should I be doing something else here?

If that works together w/ the @(Reference) conditions, we're good.

The two @(Reference) conditions are definitely the critical part.

Either your suggested change, or my original patch, will build fine with the other two changes (conditional references) included. Neither version will build without the changes to the references lines.

As far as I can see, the only difference with your suggestion is that it removes the possibility to override the CrossgenOutput setting manually on the command line. I guess this doesn't really matter either way to me -- if you prefer the version with the s390x check on a separate line, I'm happy with that.

I'm not sure if you're still interested in the specific errors that occur when removing the changes to the references lines, but just in case, the first error is:

/home/uweigand/aspnetcore/eng/targets/ResolveReferences.targets(220,5): error MSB3245: Could not resolve this reference. Could not locate the package or project for "Microsoft.NETCore.App.Runtime.linux-s390x". Did you update baselines and dependencies lists? See docs/ReferenceResolution.md for more details. [/home/uweigand/aspnetcore/src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj]

This error goes away when adding the following to eng/Dependencies.props:

    <LatestPackageReference Include="Microsoft.NETCore.App.Runtime.linux-s390x" />

and then I immediately get instead:

/home/uweigand/aspnetcore/eng/targets/ResolveReferences.targets(220,5): error MSB3245: Could not resolve this reference. Could not locate the package or project for "Microsoft.NETCore.App.Crossgen2.linux-s390x". Did you update baselines and dependencies lists? See docs/ReferenceResolution.md for more details. [/home/uweigand/aspnetcore/src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj]                                                                                                                                                

which I can similarly fix by adding to eng/Dependencies.props:

    <LatestPackageReference Include="Microsoft.NETCore.App.Crossgen2.linux-s390x" />

However, then I get the following errors:

/home/uweigand/aspnetcore/src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj : error NU1101: Unable to find package Microsoft.NETCore.App.Runtime.linux-s390x. No packages exist with this id in source(s): dotnet-eng, dotnet-experimental, dotnet-public, dotnet-tools, dotnet31-transport, dotnet5, dotnet5-transport, dotnet6, dotnet6-transport, richnav [/home/uweigand/dotnet-net6/sdk/6.0.100-preview.7.21379.14/NuGet.targets]                                                                                                              
/home/uweigand/aspnetcore/src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj : error NU1101: Unable to find package Microsoft.NETCore.App.Crossgen2.linux-s390x. No packages exist with this id in source(s): dotnet-eng, dotnet-experimental, dotnet-public, dotnet-tools, dotnet31-transport, dotnet5, dotnet5-transport, dotnet6, dotnet6-transport, richnav [/home/uweigand/dotnet-net6/sdk/6.0.100-preview.7.21379.14/NuGet.targets]                                                                                                            

The first of those I can fix by adding a local repo holding the Microsoft.NETCore.App.Runtime.linux-s390x package to NuGet.config. But the second I cannot fix because the Crossgen2.linux-s390x package really does not exist anywhere.

Of course, making the References lines conditional as in the PR will fix both those issues.

Updated PR as per the above suggestion.

I wasn't clear and missed something in your earlier comments @uweigand, sorry. I thought making the two @(Reference) items conditional caused other problems w/ the s390x runtime package. Since that's not the case, this is fine as-is.

Thanks!

Strongly suggest running the Microsoft.AspNetCore.App.UnitTests.csproj tests on x390x to make sure the runtime built matches expectations.

I did run this, and IMO it looks basically good. There are a couple of issues causing test failures, but those seem to be test case problems -- I'm seeing the exact same failures also on linux-x64. Specifically, I get:

1. Failures because artifacts/obj/SharedFx.Layout/Release/linux-s390x/shared/Microsoft.AspNetCore.App/6.0.0-dev/RuntimeList.xml does not exist - the file exists, but at artifacts/obj/RuntimeList.xml, and is never copied under SharedFx.Layout when running the test locally via "build --test". I assume the test works when running in Helix because there the SharedFx folder is unpacked from the nuget package, which does contain the file. Manually copying the file before running the test makes those failures go away.

2. Failures because of a missing System.Diagnostics.EventLog.Messages assembly. This assembly seems to be used only on Windows and not on Linux, so it looks like the expected list needs to be updated.

With the two manual changes mentioned above, there is still one final failing test, Microsoft.AspNetCore.SharedFxTests.RuntimeListListsContainsCorrectPaths, which fails here:

            var sharedFxPath = Path.Combine(Environment.GetEnvironmentVariable("HELIX_WORKITEM_ROOT"), ("Microsoft.AspNetCore.App.Runtime.win-x64." + TestData.GetSharedFxVersion() + ".nupkg"));

because that environment variable is not set when running outside of Helix. (And also the test seems to hardcode win-x64 anyway.)

Thanks @uweigand❕

/backport to release/6.0

Started backporting to release/6.0: https://github.com/dotnet/aspnetcore/actions/runs/1215205405

@Pilchie backporting to release/6.0 failed, the patch most likely resulted in conflicts:

$ git am --3way --ignore-whitespace --keep-non-patch changes.patch

Applying: Support platforms where crossgen2 is not available
Using index info to reconstruct a base tree...
M	src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj
Falling back to patching base and 3-way merge...
Auto-merging src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj
CONFLICT (content): Merge conflict in src/Framework/App.Runtime/src/Microsoft.AspNetCore.App.Runtime.csproj
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 Support platforms where crossgen2 is not available
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".
Error: The process '/usr/bin/git' failed with exit code 128

Please backport manually!

@dougbu or @uweigand - can you take a look at the conflicts and put up a release/6.0 version?

@dougbu or @uweigand - can you take a look at the conflicts and put up a release/6.0 version?

Submitted as #36325

Hi @uweigand. It looks like you just commented on a closed PR. The team will most probably miss it. If you'd like to bring something important up to their attention, consider filing a new issue and add enough details to build context.