You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Some assertions are failing at build-time in AsmOffset* and llvm-libunwind due to the sizes of various structs. llvm-libunwind is getting 96 bytes size difference in UnwindCursor<A, R> vs. unw_cursor_t which is a mystery. See the TODO added in src/native/external/llvm-libunwind/src/UnwindCursor.hpp, that template function is there temporarily to print the sizes on stderr during the build.
Also, I've used the register names we have in coreclr/pal. Some of them might not be needed, but I'm not sure since llvm-libunwind uses different names X1,X2,X3.. while coreclr/pal is using T1,T2,T3.. so I used the latter and handled the mapping in src/coreclr/nativeaot/Runtime/unix/UnwindHelpers.cpp.
# on macOS arm64
$ cd runtime
$ docker run -e ROOTFS_DIR=/crossrootfs/riscv64 -v$(pwd):/runtime -w /runtime --rm \
--platform linux/arm64 -it ubuntu
$ eng/common/native/install-dependencies.sh
$ apt install -y debootstrap
$ eng/common/cross/build-rootfs.sh riscv64 noble --skipunmount --rootfsdir /crossrootfs/riscv64
$ src/coreclr/build-runtime.sh -cross -riscv64
# once nativeaot runtime ^ starts building in silo, we can try a full build# rm -rf artifacts# ./build.sh clr+libs+packs -cross --arch riscv64
currently it is not getting too far 😅
...
[ 45%] Building CXX object nativeaot/Runtime/Full/CMakeFiles/Runtime.WorkstationGC.dir/__/__/__/gc/unix/events.cpp.o
[ 45%] Building CXX object classlibnative/bcltype/CMakeFiles/bcltype.dir/system.cpp.o
In file included from /runtime/src/coreclr/nativeaot/Runtime/AsmOffsetsVerify.cpp:43:
In file included from /runtime/src/coreclr/nativeaot/Runtime/AsmOffsets.h:79:
/runtime/src/coreclr/nativeaot/Runtime/riscv64/AsmOffsetsCpu.h:10:1: error: static assertion failed due to requirement '(sizeof(ExInfo) == 2120) || (sizeof(ExInfo) > 2120)': Bad asm size for 'ExInfo', the actual size is smaller than 0x848.
10 | PLAT_ASM_SIZEOF(848, ExInfo)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/runtime/src/coreclr/nativeaot/Runtime/AsmOffsetsVerify.cpp:36:19: note: expanded from macro 'PLAT_ASM_SIZEOF'
36 | static_assert((sizeof(cls) == 0x##size) || (sizeof(cls) > 0x##size), "Bad asm size for '" #cls "', the actual size is smaller than 0x" #size "."); \
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /runtime/src/coreclr/nativeaot/Runtime/AsmOffsetsVerify.cpp:43:
In file included from /runtime/src/coreclr/nativeaot/Runtime/AsmOffsets.h:79:
/runtime/src/coreclr/nativeaot/Runtime/riscv64/AsmOffsetsCpu.h:15:1: error: static assertion failed due to requirement '(__builtin_offsetof(ExInfo, m_passNumber) == 28) || (__builtin_offsetof(ExInfo, m_passNumber) > 28)': Bad asm offset for 'ExInfo.m_passNumber', the actual offset is smaller than 0x1C.
15 | PLAT_ASM_OFFSET(1C, ExInfo, m_passNumber)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/runtime/src/coreclr/nativeaot/Runtime/AsmOffsetsVerify.cpp:32:19: note: expanded from macro 'PLAT_ASM_OFFSET'
32 | static_assert((offsetof(cls, member) == 0x##offset) || (offsetof(cls, member) > 0x##offset), "Bad asm offset for '" #cls "." #member "', the actual offset is smaller than 0x" #offset "."); \
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
...
@filipnavara, have you had a chance to check out the assertion failures? No rush—just wondering if there are any blockers or details we need to sort out. Thanks! 👍
Unfortunately no, still tackling some things for .NET 9 release and few other hack projects that I want to get done for the .NET 10 timeframe. I hope to eventually look into it but it’s currently not a priority, sorry.
Ok, no worries, I'll also take a look when I get a chance. Was hoping to get some feedback from @viewizard et al. on register naming since it's not immediately clear to which convention is preferred at present day (and given llvm-libunwind has its own which is obviously different than HP-libunwind, coreclr VM and PAL). 😅
Here are some of my ideas, and I'm not sure if they're helpful.
I have also encountered the same problem when adding the LoongArch architecture, and I think the assertion failures should come from the incorrect offsets and sizes in src/coreclr/nativeaot/Runtime/riscv64/AsmOffsetsCpu.h. The offsets and sizes should be defined according to the actual definition of the structures.
Before rewriting src/coreclr/nativeaot/Runtime/riscv64/AsmOffsetsCpu.h, I think we should first determine the register names defined in the structures, for example, in src/coreclr/nativeaot/Runtime/regdisplay.h. Now the register names in regdisplay.h and AsmOffsetsCpu.h are still conflicting. And then determine the order of variables in the structures. After determining the definition of the structures, we can easily rewrite AsmOffsetsCpu.h all at once.
Updated workbench with libs.tests step #106223 (comment). It seems to be crossbuilding most of the tests, but ilc gets sigkill on these two (perhps due to the size or some other kind of overflow):
$ /runtime/build.sh -arch riscv64 -cross -s libs.tests -c Debug --keepnativesymbols true -p:UseLocalAppHostPack=true -p:TestNativeAot=true -p:LinkerFlavor=lld
...
/runtime/artifacts/bin/coreclr/linux.riscv64.Debug/build/Microsoft.NETCore.Native.targets(327,5): error MSB3073: The command""/runtime/artifacts/bin/coreclr/linux.riscv64.Debug/arm64/ilc/ilc" @"/runtime/artifacts/obj/System.Diagnostics.DiagnosticSource.Tests/Debug/net10.0/native/System.Diagnostics.DiagnosticSource.Tests.ilc.rsp"" exited with code 137. [/runtime/src/libraries/System.Diagnostics.DiagnosticSource/tests/System.Diagnostics.DiagnosticSource.Tests.csproj::TargetFramework=net10.0]
/runtime/artifacts/bin/coreclr/linux.riscv64.Debug/build/Microsoft.NETCore.Native.targets(327,5): error MSB3073: The command""/runtime/artifacts/bin/coreclr/linux.riscv64.Debug/arm64/ilc/ilc" @"/runtime/artifacts/obj/Microsoft.Extensions.DependencyModel.Tests/Debug/net10.0/native/Microsoft.Extensions.DependencyModel.Tests.ilc.rsp"" exited with code 137. [/runtime/src/libraries/Microsoft.Extensions.DependencyModel/tests/Microsoft.Extensions.DependencyModel.Tests.csproj::TargetFramework=net10.0]
The six which were built haven't reported any failures on the machine:
With release build, DynamicGeneric smoke is failing at "Running Test: ThreadLocalStatics.TLSTesting.ThreadLocalStatics_Test" on both the device and qemu; with DOTNET_gcConservative=1 it passes on both.
Thread 4"DynamicGenerics" received signal SIGABRT, Aborted.
[Switching to Thread 0x3fa59fe080 (LWP 314432)]
0x0000003ff7e28b14 in __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:43
warning: 43 ./nptl/pthread_kill.c: No such file or directory
(gdb) bt
#0 0x0000003ff7e28b14 in __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:43#1 0x0000003ff7e28b68 in __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78#2 0x0000003ff7deee74 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26#3 0x0000003ff7dde372 in __GI_abort () at ./stdlib/abort.c:79#4 0x0000002aaad199fc in RaiseFailFastException (arg1=<optimized out>, arg2=<optimized out>, arg3=<optimized out>) at /runtime/src/coreclr/nativeaot/Runtime/unix/PalRedhawkUnix.cpp:97#5 0x0000002aaace1c14 in StackFrameIterator::NextInternal (this=0x3fa59fce10) at /runtime/src/coreclr/nativeaot/Runtime/StackFrameIterator.cpp:1758#6 0x0000002aaace1b42 in StackFrameIterator::Next (this=0x3fa59fce10) at /runtime/src/coreclr/nativeaot/Runtime/StackFrameIterator.cpp:1716#7 0x0000002aaace3506 in Thread::GcScanRootsWorker (this=0x3fa51fd800, pfnEnumCallback=0x2aaacfe9b6 <WKS::GCHeap::Promote(Object**, ScanContext*, unsigned int)>, pvCallbackData=0x3fa59fd208, frameIterator=...)
at /runtime/src/coreclr/nativeaot/Runtime/thread.cpp:549#8 0x0000002aaace342e in Thread::GcScanRoots (this=0x3fa51fd800, pfnEnumCallback=0x2aaacfe9b6 <WKS::GCHeap::Promote(Object**, ScanContext*, unsigned int)>, pvCallbackData=0x3fa59fd208) at /runtime/src/coreclr/nativeaot/Runtime/thread.cpp:419#9 0x0000002aaacdeb84 in GCToEEInterface::GcScanRoots (fn=0x2aaacfe9b6 <WKS::GCHeap::Promote(Object**, ScanContext*, unsigned int)>, condemned=<optimized out>, max_gen=<optimized out>, sc=0x3fa59fd208)
at /runtime/src/coreclr/nativeaot/Runtime/gcenv.ee.cpp:122#10 0x0000002aaacf56ae in WKS::gc_heap::mark_phase (condemned_gen_number=2) at /runtime/src/coreclr/gc/gc.cpp:30255#11 0x0000002aaacf2c6e in WKS::gc_heap::gc1 () at /runtime/src/coreclr/gc/gc.cpp:22694#12 0x0000002aaacfb608 in WKS::gc_heap::garbage_collect (n=<optimized out>) at /runtime/src/coreclr/gc/gc.cpp:39995#13 0x0000002aaacefa06 in WKS::GCHeap::GarbageCollectGeneration (this=<optimized out>, gen=2, reason=reason_induced) at /runtime/src/coreclr/gc/gc.cpp:51505#14 0x0000002aaad0e21e in WKS::GCHeap::GarbageCollectTry (generation=<optimized out>, this=<optimized out>, low_memory_p=<optimized out>, mode=<optimized out>) at /runtime/src/coreclr/gc/gc.cpp:50699#15 WKS::GCHeap::GarbageCollect (this=<optimized out>, generation=<optimized out>, low_memory_p=<optimized out>, mode=<optimized out>) at /runtime/src/coreclr/gc/gc.cpp:50629#16 0x0000002aaacddfbc in RhpCollect (uGeneration=314432, uMode=4159865678, lowMemoryP=0) at /runtime/src/coreclr/nativeaot/Runtime/GCHelpers.cpp:114#17 0x0000002aaae67f1c in S_P_CoreLib_System_Runtime_InternalCalls__RhCollect (generation=0, mode=(unknown: 0x4cc40), lowMemoryP=6) at /runtime/src/coreclr/nativeaot/Runtime.Base/src/System/Runtime/InternalCalls.cs:65#18 0x0000002aaad52d14 in DynamicGenerics_ThreadLocalStatics_TLSTesting___c__DisplayClass3_0___MultiThreaded_Test_b__0 (this=...) at /runtime/src/tests/nativeaot/SmokeTests/DynamicGenerics/threadstatics.cs:464#19 0x0000002aaae38a1c in S_P_CoreLib_System_Threading_ExecutionContext__RunFromThreadPoolDispatchLoop (threadPoolThread=..., executionContext=..., callback=..., state=...)
at /runtime/src/libraries/System.Private.CoreLib/src/System/Threading/ExecutionContext.cs:264#20 0x0000002aaae42de0 in S_P_CoreLib_System_Threading_Tasks_Task__ExecuteWithThreadLocal (this=..., currentTaskSlot=..., threadPoolThread=...) at /runtime/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2347#21 0x0000002aaae3ddcc in S_P_CoreLib_System_Threading_ThreadPoolWorkQueue__Dispatch () at /runtime/src/libraries/System.Private.CoreLib/src/System/Threading/ThreadPoolWorkQueue.cs:1154#22 0x0000002aaaec0608 in S_P_CoreLib_System_Threading_PortableThreadPool_WorkerThread__WorkerThreadStart () at /runtime/src/libraries/System.Private.CoreLib/src/System/Threading/PortableThreadPool.WorkerThread.cs:127#23 0x0000002aaaeba120 in S_P_CoreLib_System_Threading_Thread_StartHelper__RunWorker (this=...) at /runtime/src/libraries/System.Private.CoreLib/src/System/Threading/Thread.cs:77#24 0x0000002aaae35e54 in S_P_CoreLib_System_Threading_Thread__StartThread (parameter=0) at /runtime/src/coreclr/nativeaot/System.Private.CoreLib/src/System/Threading/Thread.NativeAot.cs:459#25 0x0000002aaae366a8 in S_P_CoreLib_System_Threading_Thread__ThreadEntryPoint (parameter=<optimized out>) at /runtime/src/coreclr/nativeaot/System.Private.CoreLib/src/System/Threading/Thread.NativeAot.Unix.cs:114#26 0x0000003ff7e2719c in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447#27 0x0000003ff7e792ac in __thread_start_clone3 () at ../sysdeps/unix/sysv/linux/riscv/clone3.S:71
(gdb) f 5#5 0x0000002aaace1c14 in StackFrameIterator::NextInternal (this=0x3fa59fce10) at /runtime/src/coreclr/nativeaot/Runtime/StackFrameIterator.cpp:17581758 FAILFAST_OR_DAC_FAIL(GetCodeManager()->UnwindStackFrame(&m_methodInfo, unwindFlags, &m_RegDisplay,
(gdb) list
1753 if ((m_dwFlags & GcStackWalkFlags) == GcStackWalkFlags)
1754 {
1755 unwindFlags |= USFF_GcUnwind;
1756 }
17571758 FAILFAST_OR_DAC_FAIL(GetCodeManager()->UnwindStackFrame(&m_methodInfo, unwindFlags, &m_RegDisplay,
1759 &m_pPreviousTransitionFrame));
17601761 if (m_pPreviousTransitionFrame != NULL)
1762 {
In 1/10 runs, UnitTests also failed (exit code 0), but it seems quite rare (and individually running it doesn't seem to fail ever).
With release build, DynamicGeneric smoke is failing at "Running Test: ThreadLocalStatics.TLSTesting.ThreadLocalStatics_Test" on both the device and qemu; with DOTNET_gcConservative=1 it passes on both.
I'll have a look sometime next week unless you beat me to it.
With release build, DynamicGeneric smoke is failing at "Running Test: ThreadLocalStatics.TLSTesting.ThreadLocalStatics_Test" on both the device and qemu
I submitted PR to fix this. I didn't look into UnitTests in Release builds yet.
The tests seem to be passing consistently. It would be a good idea to add AOT smoke tests to PR runs (e.g. #112736 (comment)), so it doesn't regress over time. cc @gbalykov, @clamp03
The tests seem to be passing consistently. It would be a good idea to add AOT smoke tests to PR runs (e.g. #112736 (comment)), so it doesn't regress over time.
You will notice that running the tests without riscv64 SDK requires some custom scripting, we should include it in src/tests/Common/scripts/bringup_runtest.sh under --nativeaot or similar flag at some point. It will help not only riscv64/LA64, but any platform bringup in the future when the working SDK is not there.
With the matching SDK version in runtime/global.json, it's just a matter of src/tests/build.sh -nativeaot -tree:nativeaot -Debug -p:LibrariesConfiguration=Debug -keepnativesymbols -skipnative -skiprestorepackages, but currently infra is in the process of transition, so it will take some time to stabilize that route.
@sirntar, fyi, steps are updated above after #113243 merged, now we no longer need to do a separate host build. If your infra already has StageOneBuild and StageTwoBuild, all you need is src/tests/build.sh -nativeaot ... step shown earlier.
Activity
am11 commentedon Aug 10, 2024
cc @dotnet/samsung, @filipnavara
If someone wants take over, let me know. I can also give ssh access to riscv computer (bianbu OS, Debian derivative) if needed for testing. 👍
am11 commentedon Aug 12, 2024
Some assertions are failing at build-time in AsmOffset* and llvm-libunwind due to the sizes of various structs. llvm-libunwind is getting 96 bytes size difference in
UnwindCursor<A, R>vs.unw_cursor_twhich is a mystery. See the TODO added insrc/native/external/llvm-libunwind/src/UnwindCursor.hpp, that template function is there temporarily to print the sizes on stderr during the build.Also, I've used the register names we have in coreclr/pal. Some of them might not be needed, but I'm not sure since llvm-libunwind uses different names X1,X2,X3.. while coreclr/pal is using T1,T2,T3.. so I used the latter and handled the mapping in
src/coreclr/nativeaot/Runtime/unix/UnwindHelpers.cpp.am11 commentedon Aug 23, 2024
Applied changes from #106225.
FWIW, this is my workflow:
currently it is not getting too far 😅
filipnavara commentedon Aug 23, 2024
I tried to compile your branch and I ended up pretty much at the same spot. Currently away from computer, but this should be easy to fix…
am11 commentedon Sep 29, 2024
@filipnavara, have you had a chance to check out the assertion failures? No rush—just wondering if there are any blockers or details we need to sort out. Thanks! 👍
filipnavara commentedon Sep 29, 2024
Unfortunately no, still tackling some things for .NET 9 release and few other hack projects that I want to get done for the .NET 10 timeframe. I hope to eventually look into it but it’s currently not a priority, sorry.
am11 commentedon Sep 29, 2024
Ok, no worries, I'll also take a look when I get a chance. Was hoping to get some feedback from @viewizard et al. on register naming since it's not immediately clear to which convention is preferred at present day (and given llvm-libunwind has its own which is obviously different than HP-libunwind, coreclr VM and PAL). 😅
sunlijun-610 commentedon Sep 30, 2024
Here are some of my ideas, and I'm not sure if they're helpful.
I have also encountered the same problem when adding the LoongArch architecture, and I think the assertion failures should come from the incorrect offsets and sizes in
src/coreclr/nativeaot/Runtime/riscv64/AsmOffsetsCpu.h. The offsets and sizes should be defined according to the actual definition of the structures.Before rewriting
src/coreclr/nativeaot/Runtime/riscv64/AsmOffsetsCpu.h, I think we should first determine the register names defined in the structures, for example, insrc/coreclr/nativeaot/Runtime/regdisplay.h. Now the register names inregdisplay.handAsmOffsetsCpu.hare still conflicting. And then determine the order of variables in the structures. After determining the definition of the structures, we can easily rewriteAsmOffsetsCpu.hall at once.am11 commentedon Sep 30, 2024
Thanks @sunlijun-610, https://github.com/am11/runtime/tree/feature/nativeaot/riscv64-port has the changes. Currently, the size of
UnwindCursor<A, R>is 864 andunw_context_tis 768, so thedoes_fitassertion fails in llvm-libunwind UnwindCursor.hpp https://github.com/am11/runtime/blob/feature/nativeaot/riscv64-port/src/native/external/llvm-libunwind/src/UnwindCursor.hpp#L1349-L1357 (the existing assertions only tell us one operand's value not both sides that's why I added this template one to see the computed sizes at built time). If you have ideas how to satisfy AsmOffsetsCpu and UnwindCursor assertions simultaneously, feel free to push commits to the branch when you have time. I've sent you the colab invite. :)179 remaining items
am11 commentedon Feb 27, 2025
Updated workbench with libs.tests step #106223 (comment). It seems to be crossbuilding most of the tests, but ilc gets sigkill on these two (perhps due to the size or some other kind of overflow):
The six which were built haven't reported any failures on the machine:
am11 commentedon Feb 28, 2025
With release build, DynamicGeneric smoke is failing at "Running Test: ThreadLocalStatics.TLSTesting.ThreadLocalStatics_Test" on both the device and qemu; with
DOTNET_gcConservative=1it passes on both.In 1/10 runs, UnitTests also failed (exit code 0), but it seems quite rare (and individually running it doesn't seem to fail ever).
filipnavara commentedon Feb 28, 2025
I'll have a look sometime next week unless you beat me to it.
filipnavara commentedon Mar 2, 2025
I submitted PR to fix this. I didn't look into UnitTests in Release builds yet.
am11 commentedon Mar 2, 2025
Thanks! It's fixed on device and emulator.
I wasn't able to repro that UnitTests failure either after that one time. Will keep an eye out.
am11 commentedon Mar 4, 2025
The tests seem to be passing consistently. It would be a good idea to add AOT smoke tests to PR runs (e.g. #112736 (comment)), so it doesn't regress over time. cc @gbalykov, @clamp03
tomeksowi commentedon Mar 4, 2025
cc @sirntar
sirntar commentedon Mar 5, 2025
@am11 I will add them asap.
am11 commentedon Mar 6, 2025
@sirntar thank you! The steps for cross-building are same as #106223 (comment).
This github action cross-builds runtime and AOT smoke tests, then uses qemu to execute them (
docker run --platform linux/riscv64):https://github.com/am11/CrossRepoCITesting/actions/runs/13694911386/workflow
The(merged) Other than that, you can adapt it in your infra. If your infra is open-source, I can lend hand as well.git applyline will go away once #113179 is merged.You will notice that running the tests without riscv64 SDK requires some custom scripting, we should include it in
src/tests/Common/scripts/bringup_runtest.shunder--nativeaotor similar flag at some point. It will help not only riscv64/LA64, but any platform bringup in the future when the working SDK is not there.With the matching SDK version in
runtime/global.json, it's just a matter ofsrc/tests/build.sh -nativeaot -tree:nativeaot -Debug -p:LibrariesConfiguration=Debug -keepnativesymbols -skipnative -skiprestorepackages, but currently infra is in the process of transition, so it will take some time to stabilize that route.am11 commentedon Mar 7, 2025
With #113243, we don't need to build for host and it cuts time almost in half (36m vs. 56m) am11/CrossRepoCITesting@d4edd95. Run https://github.com/am11/CrossRepoCITesting/actions/runs/13730444237
am11 commentedon Mar 10, 2025
@sirntar, fyi, steps are updated above after #113243 merged, now we no longer need to do a separate host build. If your infra already has StageOneBuild and StageTwoBuild, all you need is
src/tests/build.sh -nativeaot ...step shown earlier.am11 commentedon Apr 3, 2025
Tests are also passing on Alpine with musl-libc build:
linux-musl-riscv64: https://github.com/am11/CrossRepoCITesting/blob/master/.github/workflows/linux-riscv64-aot-alpine.yml
linux-riscv64 glibc: https://github.com/am11/CrossRepoCITesting/blob/master/.github/workflows/linux-riscv64-aot.yml
am11 commentedon Jun 24, 2026
Found one issue during the libs tests in AOT mode; reported here: #129792.
Closing as done.