Skip to content

RISC-V NativeAOT port #106223

Closed
Closed

Description

@am11

This is to track the progress of nativeaot port on riscv64 architecture.

WIP initial translation (based on @sunlijun-610's LA64 port):
main...am11:runtime:feature/nativeaot/riscv64-port

Activity

  1. am11 commented on Aug 10, 2024

    @am11
    MemberAuthor

    cc @dotnet/samsung, @filipnavara

    If someone wants take over, let me know. I can also give ssh access to riscv computer (bianbu OS, Debian derivative) if needed for testing. 👍

  2. am11 commented on Aug 12, 2024

    @am11
    MemberAuthor

    Some assertions are failing at build-time in AsmOffset* and llvm-libunwind due to the sizes of various structs. llvm-libunwind is getting 96 bytes size difference in UnwindCursor<A, R> vs. unw_cursor_t which is a mystery. See the TODO added in src/native/external/llvm-libunwind/src/UnwindCursor.hpp, that template function is there temporarily to print the sizes on stderr during the build.

    Also, I've used the register names we have in coreclr/pal. Some of them might not be needed, but I'm not sure since llvm-libunwind uses different names X1,X2,X3.. while coreclr/pal is using T1,T2,T3.. so I used the latter and handled the mapping in src/coreclr/nativeaot/Runtime/unix/UnwindHelpers.cpp.

  3. added this to the Future milestone on Aug 12, 2024
  4. am11 commented on Aug 23, 2024

    @am11
    MemberAuthor

    Applied changes from #106225.

    FWIW, this is my workflow:

    # on macOS arm64
    $ cd runtime
    $ docker run -e ROOTFS_DIR=/crossrootfs/riscv64 -v$(pwd):/runtime -w /runtime --rm \
        --platform linux/arm64 -it ubuntu
    
    $ eng/common/native/install-dependencies.sh 
    $ apt install -y debootstrap
    $ eng/common/cross/build-rootfs.sh riscv64 noble --skipunmount --rootfsdir /crossrootfs/riscv64
    $ src/coreclr/build-runtime.sh -cross -riscv64
    
    # once nativeaot runtime ^ starts building in silo, we can try a full build
    # rm -rf artifacts
    # ./build.sh clr+libs+packs -cross --arch riscv64

    currently it is not getting too far 😅

    ...
    [ 45%] Building CXX object nativeaot/Runtime/Full/CMakeFiles/Runtime.WorkstationGC.dir/__/__/__/gc/unix/events.cpp.o
    [ 45%] Building CXX object classlibnative/bcltype/CMakeFiles/bcltype.dir/system.cpp.o
    In file included from /runtime/src/coreclr/nativeaot/Runtime/AsmOffsetsVerify.cpp:43:
    In file included from /runtime/src/coreclr/nativeaot/Runtime/AsmOffsets.h:79:
    /runtime/src/coreclr/nativeaot/Runtime/riscv64/AsmOffsetsCpu.h:10:1: error: static assertion failed due to requirement '(sizeof(ExInfo) == 2120) || (sizeof(ExInfo) > 2120)': Bad asm size for 'ExInfo', the actual size is smaller than 0x848.
       10 | PLAT_ASM_SIZEOF(848, ExInfo)
          | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
    /runtime/src/coreclr/nativeaot/Runtime/AsmOffsetsVerify.cpp:36:19: note: expanded from macro 'PLAT_ASM_SIZEOF'
       36 |     static_assert((sizeof(cls) == 0x##size) || (sizeof(cls) > 0x##size), "Bad asm size for '" #cls "', the actual size is smaller than 0x" #size "."); \
          |                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    In file included from /runtime/src/coreclr/nativeaot/Runtime/AsmOffsetsVerify.cpp:43:
    In file included from /runtime/src/coreclr/nativeaot/Runtime/AsmOffsets.h:79:
    /runtime/src/coreclr/nativeaot/Runtime/riscv64/AsmOffsetsCpu.h:15:1: error: static assertion failed due to requirement '(__builtin_offsetof(ExInfo, m_passNumber) == 28) || (__builtin_offsetof(ExInfo, m_passNumber) > 28)': Bad asm offset for 'ExInfo.m_passNumber', the actual offset is smaller than 0x1C.
       15 | PLAT_ASM_OFFSET(1C, ExInfo, m_passNumber)
          | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    /runtime/src/coreclr/nativeaot/Runtime/AsmOffsetsVerify.cpp:32:19: note: expanded from macro 'PLAT_ASM_OFFSET'
       32 |     static_assert((offsetof(cls, member) == 0x##offset) || (offsetof(cls, member) > 0x##offset), "Bad asm offset for '" #cls "." #member "', the actual offset is smaller than 0x" #offset "."); \
          |                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    ...
    
  5. filipnavara commented on Aug 23, 2024

    @filipnavara
    Member

    I tried to compile your branch and I ended up pretty much at the same spot. Currently away from computer, but this should be easy to fix…

  6. am11 commented on Sep 29, 2024

    @am11
    MemberAuthor

    @filipnavara, have you had a chance to check out the assertion failures? No rush—just wondering if there are any blockers or details we need to sort out. Thanks! 👍

  7. filipnavara commented on Sep 29, 2024

    @filipnavara
    Member

    Unfortunately no, still tackling some things for .NET 9 release and few other hack projects that I want to get done for the .NET 10 timeframe. I hope to eventually look into it but it’s currently not a priority, sorry.

  8. am11 commented on Sep 29, 2024

    @am11
    MemberAuthor

    Ok, no worries, I'll also take a look when I get a chance. Was hoping to get some feedback from @viewizard et al. on register naming since it's not immediately clear to which convention is preferred at present day (and given llvm-libunwind has its own which is obviously different than HP-libunwind, coreclr VM and PAL). 😅

  9. sunlijun-610 commented on Sep 30, 2024

    @sunlijun-610
    Contributor

    Here are some of my ideas, and I'm not sure if they're helpful.
    I have also encountered the same problem when adding the LoongArch architecture, and I think the assertion failures should come from the incorrect offsets and sizes in src/coreclr/nativeaot/Runtime/riscv64/AsmOffsetsCpu.h. The offsets and sizes should be defined according to the actual definition of the structures.
    Before rewriting src/coreclr/nativeaot/Runtime/riscv64/AsmOffsetsCpu.h, I think we should first determine the register names defined in the structures, for example, in src/coreclr/nativeaot/Runtime/regdisplay.h. Now the register names in regdisplay.h and AsmOffsetsCpu.h are still conflicting. And then determine the order of variables in the structures. After determining the definition of the structures, we can easily rewrite AsmOffsetsCpu.h all at once.

  10. am11 commented on Sep 30, 2024

    @am11
    MemberAuthor

    Thanks @sunlijun-610, https://github.com/am11/runtime/tree/feature/nativeaot/riscv64-port has the changes. Currently, the size of UnwindCursor<A, R> is 864 and unw_context_t is 768, so the does_fit assertion fails in llvm-libunwind UnwindCursor.hpp https://github.com/am11/runtime/blob/feature/nativeaot/riscv64-port/src/native/external/llvm-libunwind/src/UnwindCursor.hpp#L1349-L1357 (the existing assertions only tell us one operand's value not both sides that's why I added this template one to see the computed sizes at built time). If you have ideas how to satisfy AsmOffsetsCpu and UnwindCursor assertions simultaneously, feel free to push commits to the branch when you have time. I've sent you the colab invite. :)

  11. 179 remaining items

  12. am11 commented on Feb 27, 2025

    @am11
    MemberAuthor

    Updated workbench with libs.tests step #106223 (comment). It seems to be crossbuilding most of the tests, but ilc gets sigkill on these two (perhps due to the size or some other kind of overflow):

    $ /runtime/build.sh -arch riscv64 -cross -s libs.tests -c Debug --keepnativesymbols true -p:UseLocalAppHostPack=true -p:TestNativeAot=true -p:LinkerFlavor=lld
    ...
    /runtime/artifacts/bin/coreclr/linux.riscv64.Debug/build/Microsoft.NETCore.Native.targets(327,5): error MSB3073: The command ""/runtime/artifacts/bin/coreclr/linux.riscv64.Debug/arm64/ilc/ilc" @"/runtime/artifacts/obj/System.Diagnostics.DiagnosticSource.Tests/Debug/net10.0/native/System.Diagnostics.DiagnosticSource.Tests.ilc.rsp"" exited with code 137. [/runtime/src/libraries/System.Diagnostics.DiagnosticSource/tests/System.Diagnostics.DiagnosticSource.Tests.csproj::TargetFramework=net10.0]
    /runtime/artifacts/bin/coreclr/linux.riscv64.Debug/build/Microsoft.NETCore.Native.targets(327,5): error MSB3073: The command ""/runtime/artifacts/bin/coreclr/linux.riscv64.Debug/arm64/ilc/ilc" @"/runtime/artifacts/obj/Microsoft.Extensions.DependencyModel.Tests/Debug/net10.0/native/Microsoft.Extensions.DependencyModel.Tests.ilc.rsp"" exited with code 137. [/runtime/src/libraries/Microsoft.Extensions.DependencyModel/tests/Microsoft.Extensions.DependencyModel.Tests.csproj::TargetFramework=net10.0]

    The six which were built haven't reported any failures on the machine:

    $ find ~/projects/runtime/artifacts/bin -path '*net10.0-linux/publish*' -type f -exec file {} \; |
        grep ELF.*executable |
        while IFS=: read f _; do
          pushd "$(dirname $f)"
          ./$(basename $f) -notrait category=OuterLoop -notrait category=failing
          popd
        done
    
    Running assembly:System.Runtime.Serialization.Formatters.Disabled.Tests, Version=10.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
    ...
    Tests run: 3, Errors: 0, Failures: 0, Skipped: 2. Time: 0.0425735s
    
    Running assembly:System.IO.FileSystem.Watcher.Tests, Version=10.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
    ...
    Tests run: 307, Errors: 0, Failures: 0, Skipped: 0. Time: 3.9406643s
    
    Running assembly:System.DirectoryServices.Protocols.Tests, Version=10.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
    ...
    Tests run: 630, Errors: 0, Failures: 0, Skipped: 22. Time: 1.208248s
    
    Running assembly:Common.Tests, Version=10.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
    ...
    Tests run: 1444, Errors: 0, Failures: 0, Skipped: 1. Time: 186.2769032s
    
    Running assembly:System.Net.Http.Functional.Tests, Version=10.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
    ...
    Tests run: 0, Errors: 0, Failures: 0, Skipped: 0. Time: 0.0080169s
    
    Running assembly:System.Net.Quic.Functional.Tests, Version=10.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51
    ...
    Tests run: 5, Errors: 0, Failures: 0, Skipped: 1. Time: 3.2699584s
  13. am11 commented on Feb 28, 2025

    @am11
    MemberAuthor

    With release build, DynamicGeneric smoke is failing at "Running Test: ThreadLocalStatics.TLSTesting.ThreadLocalStatics_Test" on both the device and qemu; with DOTNET_gcConservative=1 it passes on both.

    Thread 4 "DynamicGenerics" received signal SIGABRT, Aborted.
    [Switching to Thread 0x3fa59fe080 (LWP 314432)]
    0x0000003ff7e28b14 in __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:43
    warning: 43	./nptl/pthread_kill.c: No such file or directory
    (gdb) bt
    #0  0x0000003ff7e28b14 in __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:43
    #1  0x0000003ff7e28b68 in __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78
    #2  0x0000003ff7deee74 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
    #3  0x0000003ff7dde372 in __GI_abort () at ./stdlib/abort.c:79
    #4  0x0000002aaad199fc in RaiseFailFastException (arg1=<optimized out>, arg2=<optimized out>, arg3=<optimized out>) at /runtime/src/coreclr/nativeaot/Runtime/unix/PalRedhawkUnix.cpp:97
    #5  0x0000002aaace1c14 in StackFrameIterator::NextInternal (this=0x3fa59fce10) at /runtime/src/coreclr/nativeaot/Runtime/StackFrameIterator.cpp:1758
    #6  0x0000002aaace1b42 in StackFrameIterator::Next (this=0x3fa59fce10) at /runtime/src/coreclr/nativeaot/Runtime/StackFrameIterator.cpp:1716
    #7  0x0000002aaace3506 in Thread::GcScanRootsWorker (this=0x3fa51fd800, pfnEnumCallback=0x2aaacfe9b6 <WKS::GCHeap::Promote(Object**, ScanContext*, unsigned int)>, pvCallbackData=0x3fa59fd208, frameIterator=...)
        at /runtime/src/coreclr/nativeaot/Runtime/thread.cpp:549
    #8  0x0000002aaace342e in Thread::GcScanRoots (this=0x3fa51fd800, pfnEnumCallback=0x2aaacfe9b6 <WKS::GCHeap::Promote(Object**, ScanContext*, unsigned int)>, pvCallbackData=0x3fa59fd208) at /runtime/src/coreclr/nativeaot/Runtime/thread.cpp:419
    #9  0x0000002aaacdeb84 in GCToEEInterface::GcScanRoots (fn=0x2aaacfe9b6 <WKS::GCHeap::Promote(Object**, ScanContext*, unsigned int)>, condemned=<optimized out>, max_gen=<optimized out>, sc=0x3fa59fd208)
        at /runtime/src/coreclr/nativeaot/Runtime/gcenv.ee.cpp:122
    #10 0x0000002aaacf56ae in WKS::gc_heap::mark_phase (condemned_gen_number=2) at /runtime/src/coreclr/gc/gc.cpp:30255
    #11 0x0000002aaacf2c6e in WKS::gc_heap::gc1 () at /runtime/src/coreclr/gc/gc.cpp:22694
    #12 0x0000002aaacfb608 in WKS::gc_heap::garbage_collect (n=<optimized out>) at /runtime/src/coreclr/gc/gc.cpp:39995
    #13 0x0000002aaacefa06 in WKS::GCHeap::GarbageCollectGeneration (this=<optimized out>, gen=2, reason=reason_induced) at /runtime/src/coreclr/gc/gc.cpp:51505
    #14 0x0000002aaad0e21e in WKS::GCHeap::GarbageCollectTry (generation=<optimized out>, this=<optimized out>, low_memory_p=<optimized out>, mode=<optimized out>) at /runtime/src/coreclr/gc/gc.cpp:50699
    #15 WKS::GCHeap::GarbageCollect (this=<optimized out>, generation=<optimized out>, low_memory_p=<optimized out>, mode=<optimized out>) at /runtime/src/coreclr/gc/gc.cpp:50629
    #16 0x0000002aaacddfbc in RhpCollect (uGeneration=314432, uMode=4159865678, lowMemoryP=0) at /runtime/src/coreclr/nativeaot/Runtime/GCHelpers.cpp:114
    #17 0x0000002aaae67f1c in S_P_CoreLib_System_Runtime_InternalCalls__RhCollect (generation=0, mode=(unknown: 0x4cc40), lowMemoryP=6) at /runtime/src/coreclr/nativeaot/Runtime.Base/src/System/Runtime/InternalCalls.cs:65
    #18 0x0000002aaad52d14 in DynamicGenerics_ThreadLocalStatics_TLSTesting___c__DisplayClass3_0___MultiThreaded_Test_b__0 (this=...) at /runtime/src/tests/nativeaot/SmokeTests/DynamicGenerics/threadstatics.cs:464
    #19 0x0000002aaae38a1c in S_P_CoreLib_System_Threading_ExecutionContext__RunFromThreadPoolDispatchLoop (threadPoolThread=..., executionContext=..., callback=..., state=...)
        at /runtime/src/libraries/System.Private.CoreLib/src/System/Threading/ExecutionContext.cs:264
    #20 0x0000002aaae42de0 in S_P_CoreLib_System_Threading_Tasks_Task__ExecuteWithThreadLocal (this=..., currentTaskSlot=..., threadPoolThread=...) at /runtime/src/libraries/System.Private.CoreLib/src/System/Threading/Tasks/Task.cs:2347
    #21 0x0000002aaae3ddcc in S_P_CoreLib_System_Threading_ThreadPoolWorkQueue__Dispatch () at /runtime/src/libraries/System.Private.CoreLib/src/System/Threading/ThreadPoolWorkQueue.cs:1154
    #22 0x0000002aaaec0608 in S_P_CoreLib_System_Threading_PortableThreadPool_WorkerThread__WorkerThreadStart () at /runtime/src/libraries/System.Private.CoreLib/src/System/Threading/PortableThreadPool.WorkerThread.cs:127
    #23 0x0000002aaaeba120 in S_P_CoreLib_System_Threading_Thread_StartHelper__RunWorker (this=...) at /runtime/src/libraries/System.Private.CoreLib/src/System/Threading/Thread.cs:77
    #24 0x0000002aaae35e54 in S_P_CoreLib_System_Threading_Thread__StartThread (parameter=0) at /runtime/src/coreclr/nativeaot/System.Private.CoreLib/src/System/Threading/Thread.NativeAot.cs:459
    #25 0x0000002aaae366a8 in S_P_CoreLib_System_Threading_Thread__ThreadEntryPoint (parameter=<optimized out>) at /runtime/src/coreclr/nativeaot/System.Private.CoreLib/src/System/Threading/Thread.NativeAot.Unix.cs:114
    #26 0x0000003ff7e2719c in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:447
    #27 0x0000003ff7e792ac in __thread_start_clone3 () at ../sysdeps/unix/sysv/linux/riscv/clone3.S:71
    (gdb) f 5
    #5  0x0000002aaace1c14 in StackFrameIterator::NextInternal (this=0x3fa59fce10) at /runtime/src/coreclr/nativeaot/Runtime/StackFrameIterator.cpp:1758
    1758	    FAILFAST_OR_DAC_FAIL(GetCodeManager()->UnwindStackFrame(&m_methodInfo, unwindFlags, &m_RegDisplay,
    (gdb) list
    1753	    if ((m_dwFlags & GcStackWalkFlags) == GcStackWalkFlags)
    1754	    {
    1755	        unwindFlags |= USFF_GcUnwind;
    1756	    }
    1757	
    1758	    FAILFAST_OR_DAC_FAIL(GetCodeManager()->UnwindStackFrame(&m_methodInfo, unwindFlags, &m_RegDisplay,
    1759	                                                            &m_pPreviousTransitionFrame));
    1760	
    1761	    if (m_pPreviousTransitionFrame != NULL)
    1762	    {

    In 1/10 runs, UnitTests also failed (exit code 0), but it seems quite rare (and individually running it doesn't seem to fail ever).

  14. filipnavara commented on Feb 28, 2025

    @filipnavara
    Member

    With release build, DynamicGeneric smoke is failing at "Running Test: ThreadLocalStatics.TLSTesting.ThreadLocalStatics_Test" on both the device and qemu; with DOTNET_gcConservative=1 it passes on both.

    I'll have a look sometime next week unless you beat me to it.

  15. filipnavara commented on Mar 2, 2025

    @filipnavara
    Member

    With release build, DynamicGeneric smoke is failing at "Running Test: ThreadLocalStatics.TLSTesting.ThreadLocalStatics_Test" on both the device and qemu

    I submitted PR to fix this. I didn't look into UnitTests in Release builds yet.

  16. am11 commented on Mar 2, 2025

    @am11
    MemberAuthor

    Thanks! It's fixed on device and emulator.

    I didn't look into UnitTests in Release builds yet.

    I wasn't able to repro that UnitTests failure either after that one time. Will keep an eye out.

  17. am11 commented on Mar 4, 2025

    @am11
    MemberAuthor

    The tests seem to be passing consistently. It would be a good idea to add AOT smoke tests to PR runs (e.g. #112736 (comment)), so it doesn't regress over time. cc @gbalykov, @clamp03

  18. tomeksowi commented on Mar 4, 2025

    @tomeksowi
    Member

    The tests seem to be passing consistently. It would be a good idea to add AOT smoke tests to PR runs (e.g. #112736 (comment)), so it doesn't regress over time.

    cc @sirntar

  19. sirntar commented on Mar 5, 2025

    @sirntar
    Member

    @am11 I will add them asap.

  20. am11 commented on Mar 6, 2025

    @am11
    MemberAuthor

    @sirntar thank you! The steps for cross-building are same as #106223 (comment).

    This github action cross-builds runtime and AOT smoke tests, then uses qemu to execute them (docker run --platform linux/riscv64):
    https://github.com/am11/CrossRepoCITesting/actions/runs/13694911386/workflow The git apply line will go away once #113179 is merged. (merged) Other than that, you can adapt it in your infra. If your infra is open-source, I can lend hand as well.

    You will notice that running the tests without riscv64 SDK requires some custom scripting, we should include it in src/tests/Common/scripts/bringup_runtest.sh under --nativeaot or similar flag at some point. It will help not only riscv64/LA64, but any platform bringup in the future when the working SDK is not there.

    With the matching SDK version in runtime/global.json, it's just a matter of src/tests/build.sh -nativeaot -tree:nativeaot -Debug -p:LibrariesConfiguration=Debug -keepnativesymbols -skipnative -skiprestorepackages, but currently infra is in the process of transition, so it will take some time to stabilize that route.

  21. am11 commented on Mar 7, 2025

    @am11
    MemberAuthor
  22. am11 commented on Mar 10, 2025

    @am11
    MemberAuthor

    @sirntar, fyi, steps are updated above after #113243 merged, now we no longer need to do a separate host build. If your infra already has StageOneBuild and StageTwoBuild, all you need is src/tests/build.sh -nativeaot ... step shown earlier.

  23. am11 commented on Apr 3, 2025

    @am11
    MemberAuthor
  24. am11 commented on Jun 24, 2026

    @am11
    MemberAuthor

    Found one issue during the libs tests in AOT mode; reported here: #129792.

    Closing as done.

  25. locked and limited conversation to collaborators on Jul 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    • Status
      No status

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions