Add local build for System.Globalization shim#72896
Add local build for System.Globalization shim#72896sbomer merged 28 commits intodotnet:mainfrom smhmhmd:build
Conversation
Bundle the System.Globalization.Native shim sources into the native AOT toolchain package and compile them automatically using C/C++ compiler on the target machine. Compiling of the shim sources using C/C++ compiler is done during dotnet publish as follows: dotnet publish /p:LocalSystemGlobalizationNative=1 -bl -r linux-x64 This change is 'opt-in', it is enabled only by using /p:LocalSystemGlobalizationNative=1
|
Tagging subscribers to this area: @dotnet/area-system-globalization Issue DetailsBundle the System.Globalization.Native shim sources into the native AOT toolchain package and compile them automatically using C/C++ compiler on the target machine. Compiling of the shim sources using C/C++ compiler is done during dotnet publish as follows: This change is 'opt-in', it is enabled only by using /p:LocalSystemGlobalizationNative=1
|
src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets
Outdated
Show resolved
Hide resolved
|
CC @jkotas |
src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets
Outdated
Show resolved
Hide resolved
src/native/libs/System.Globalization.Native/pal_icushim_native_static.c
Outdated
Show resolved
Hide resolved
src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets
Outdated
Show resolved
Hide resolved
src/native/libs/System.Globalization.Native/pal_icushim_native_static.c
Outdated
Show resolved
Hide resolved
src/native/libs/System.Globalization.Native/config-standalone.h
Outdated
Show resolved
Hide resolved
|
So far we have not required cmake dependency for |
I do not think that cmake dependency is a problem for this special mode of NativeAOT publishing. This mode can grow into optionally building all C/C++ that the runtime is composed from. |
|
We can construct the same arguments in our existing msbuild targets which invokes the compiler, without requiring a fully fledged build system from user. It just feels heavy (even for this optional mode) as there is no system introspection going on only a few paths and macro definitions which can be passed with rest of the command line arguments. |
* Copy shim source code into users Obj directory at IntermediateOutputPath and compile ** $ ls -al obj/Debug/net7.0/linux-x64/shim_source_code/libs/System.Globalization.Native/build/libSystem.Globalization.Native-Static.a * Add shell script for readability * Add copyright headers
Add change that was part of previous commit
This does not seem to be addressed. |
src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets
Outdated
Show resolved
Hide resolved
* Include existing pal_icushim_static.c by using STATIC_ICU define * Use IlcSdkPath to copy shim source code
src/installer/pkg/projects/Microsoft.DotNet.ILCompiler/Microsoft.DotNet.ILCompiler.pkgproj
Outdated
Show resolved
Hide resolved
src/native/libs/System.Globalization.Native/config-standalone.h
Outdated
Show resolved
Hide resolved
Removed |
…pile defs
* Remove IlcHostPackagePath check
* StaticICULinking property is true/false
* Remove -g and -fPIC from definitions
* Use minimal warning level
* Remove -DHOST_AMD64
* Remove -O0
* Remove -fms-extensions -fwrapv
* Standardize on "local build"
* Rename build-nativeaot-shim-standalone.sh to build.sh
* Script name in the error message should match the actual script name
* Rename source_code to native/src
Test code:
$ cat Program.cs
using System.Globalization;
namespace CurrentCulture
{
class Program
{
static void Main(string[] args)
{
double val = 1235.56;
Console.WriteLine($"Current culture: {CultureInfo.CurrentCulture.Name}");
Console.WriteLine(val);
CultureInfo.DefaultThreadCurrentCulture = new CultureInfo("en-US");
Console.WriteLine($"Current culture: {CultureInfo.CurrentCulture.Name}");
Console.WriteLine(val);
}
}
}
$ cat hello-4.csproj
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net7.0</TargetFramework>
<RootNamespace>hello_4</RootNamespace>
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
</PropertyGroup>
$ cat nuget.config
<?xml version="1.0" encoding="utf-8"?>
<configuration>
<packageSources>
<!--To inherit the global NuGet package sources remove the <clear/> line below -->
<clear />
<add key="repositoryPath" value="/home/ubuntu/runtime/artifacts/packages/Debug/Shipping" />
<add key="dotnet7" value="https://pkgs.dev.azure.com/dnceng/public/_packaging/dotnet7/nuget/v3/index.json" />
</packageSources>
</configuration>
Test code build:
rm ./bin/Debug/net7.0/linux-arm64/native/hello-4
~/dotnet-sdk/dotnet clean
~/dotnet-sdk/dotnet publish /p:PublishAot=true /p:StaticICULinking=true -bl -r linux-arm64 --self-contained -v d 2>&1 > log
./bin/Debug/net7.0/linux-arm64/native/hello-4
Tested on x86-64 and arm64
|
@smhmhmd and I both tested the recent commit across ARM64 and x64 with the below sample code, and were able to see that ulocdata_getCLDRVersion_66 was statically linked in the binary. As expected the resulting binary is much bigger with this new static linking. Up from 20MB unstripped to 50MB unstripped. Published with |
src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets
Outdated
Show resolved
Hide resolved
src/native/libs/System.Globalization.Native/local-build/CMakeLists.txt
Outdated
Show resolved
Hide resolved
src/native/libs/System.Globalization.Native/local-build/CMakeLists.txt
Outdated
Show resolved
Hide resolved
src/installer/pkg/projects/Microsoft.DotNet.ILCompiler/Microsoft.DotNet.ILCompiler.pkgproj
Outdated
Show resolved
Hide resolved
* Incorporated change to use only src/native/libs/System.Globalization.Native/CMakeLists.txt * Conditionalized CMakeLists.txt for local build * Moved the build.sh into parent as local_build.sh and removed local-build directory. * Rename STATIC_SHIM_COMPILE to LOCAL_BUILD
Move the includes in Microsoft.DotNet.ILCompiler.pkgproj to <ItemGroup Condition="'$(PackageTargetRuntime)' != ''">
Move the includes in Microsoft.DotNet.ILCompiler.pkgproj to <ItemGroup Condition="'$(PackageTargetRuntime)' != ''">
|
One last thing: Could you please add a section "Using statically linked ICU" to https://github.com/dotnet/runtime/blob/main/src/coreclr/nativeaot/docs/compiling.md that documents this feature? (what it does and why one may want to use it, additional pre-requisites to install in addition to https://docs.microsoft.com/en-us/dotnet/core/deploying/native-aot/#prerequisites, etc.) |
Thanks, I will add in next patch |
* Add "Using statically linked ICU" to compiling.md * Remove define check for LOCAL_BUILD in CMakeLists.txt
jkotas
left a comment
There was a problem hiding this comment.
LGTM! Thank you for working on this!
@dotnet/ilc-contrib @dotnet/area-system-globalization Any additional feedback?
MichalStrehovsky
left a comment
There was a problem hiding this comment.
Looks great otherwise.
If you would like to make sure we're testing this configuration and it doesn't regress, add a test under src/tests/nativeaot/SmokeTests.
src/installer/pkg/projects/Microsoft.DotNet.ILCompiler/Microsoft.DotNet.ILCompiler.pkgproj
Show resolved
Hide resolved
Co-authored-by: Michal Strehovský <[email protected]>
Thanks, I will add a test under src/tests/nativeaot/SmokeTests |
…ft.DotNet.ILCompiler.pkgproj Co-authored-by: Michal Strehovský <[email protected]>
Bundle the System.Globalization.Native shim sources into the native AOT toolchain package and compile them automatically using C/C++ compiler on the target machine.
Compiling of the shim sources using C/C++ compiler is done during dotnet publish as follows:
dotnet publish /p:LocalSystemGlobalizationNative=1 -bl -r linux-x64
This change is 'opt-in', it is enabled only by using /p:LocalSystemGlobalizationNative=1