[Design] RISC-V: NativeAOT soft-float target (riscv64-lp64) and builds without the A extension #134558

The complete implementation is in draft PR #134456. It is there to show the whole
series and to run CI; please discuss the design here first. Once the direction is
agreed, we will split it into the smaller PRs listed under Plan.

Background

RISC-V defines a standard soft-float calling convention, lp64: the base integer
ISA with no F/D extensions, and floating-point values passed and returned in
integer registers. .NET cannot target it today. TARGET_RISCV64 in RyuJIT
assumes lp64d: FP values live in fa*, the emitter may emit fadd.d, and
ElfObjectWriter writes EF_RISCV_FLOAT_ABI_DOUBLE into e_flags for every
object it produces. The native runtime is compiled with -march=rv64gc -mabi=lp64d unconditionally, and its hand-written assembly uses F/D and A
instructions unconditionally.

The consumer is zkVM guests. A zkVM executes a fixed RISC-V subset. The zkEVM
target standard is rv64im, with no A, C, F or D, because every executed
instruction has to be proven and floating-point circuits are not part of the
standard. NativeAOT is a natural fit for such guests: one static image, no JIT at
run time.

Everything specific to that environment stays outside this repo: libc, the zkVM
system interface, the soft-float libm, the link step, the sysroot. The compiler
and the runtime build cannot. The calling convention and the register
class of TYP_FLOAT/TYP_DOUBLE are decisions RyuJIT makes internally, and they
are not pluggable.

#132204 started this work: it builds the runtime's own assembly without F/D,
A and C. #134456 contains a reworked version of those changes, following the
review feedback there, together with the code generation.

Two different precedents apply. The ABI part follows armel
(CONFIGURABLE_ARM_ABI, compUseSoftFP, CORJIT_FLAG_SOFTFP_ABI). The
arithmetic part does not: armel is a soft-float ABI on hardware with VFP, so
it still emits FP instructions. For decomposing arithmetic into helper calls the
precedents are 64-bit integer arithmetic on 32-bit targets
(CORINFO_HELP_LMUL/LDIV, USE_HELPER_FOR_ARITH) and the x86 FP casts
(fgMorphCastIntoHelper). Soft-float decomposition itself exists in this repo in
Mono's RISC-V port (MONO_ARCH_SOFT_FLOAT_FALLBACK, mono_decompose_soft_float).

Goals

• A soft-float RISC-V target that NativeAOT selects explicitly, producing lp64
  objects with no F/D instructions.
• A native runtime build for an ISA without F/D, A or C, selected at
  configure time.
• No change for any existing target. Defaults, codegen and object output stay the
  same when the new options are not selected.
• No floating-point implementation added to the runtime or the libraries.

Non-goals

• No software floating-point implementation. The arithmetic goes to the C
  toolchain's compiler-rt/libgcc builtins (__adddf3, __ledf2, …), the same way
  CORINFO_HELP_{FLT,DBL}REM already goes to the C library's fmodf/fmod.
  System.Private.CoreLib is not touched and no #if paths are added to managed
  code.
• No change to the baseline. rv64gc stays the default. A hard-float target
  missing F or D is rejected with a diagnostic. lp64f is not supported.
• Soft-float code generation is NativeAOT only. No CoreCLR VM support: the VM
  never selects this ABI, so the helper table carries NULL entries, as the
  64-bit VM does for CORINFO_HELP_LLSH. No crossgen2/ReadyToRun: crossgen2
  rejects the target on the command line. No interpreter.
• No public API change.
• No multi-hart atomicity without A. Without the A extension there is no
  atomic memory operation to emit. Interlocked and CmpXchg then lower to a
  plain read/modify/write, and a build has to opt into that explicitly with
  --assume-no-concurrency.

Proposal

0. Model F, D, C, A as instruction sets

riscv64 has no InstructionSet entries for its base extensions, so there is no
way to express "this target has no F/D" or to gate the emitter on it. Add the
four extensions to InstructionSetDesc.txt (D implies F) and seed the
rv64gc baseline from them in the VM and in the AOT compilers. Every riscv64
target still gets all four unless something removes them. A reduced target is
spelled the way the C toolchain spells it: --instruction-set=-a,-c,-d,-f.

Only C gets a runtime config opt-out (DOTNET_EnableRiscV64Compressed), because
turning it off only stops the JIT from emitting compressed encodings. F, D and
A get none. A JIT that silently lost one of them would either still need FP
instructions under the lp64d ABI or make Interlocked non-atomic on multi-hart
hardware. A target without them is selected only through the AOT compiler's
instruction set, which checks the ABI (§2) and the execution environment (§5).

The emitter then asks before it emits:

• tryEmitCompressedIns_R_R_R, the only place the JIT emits a compressed
  encoding, returns false without C;
• a checked JIT asserts in emitOutput_Instr that no F/D major opcode is
  emitted without F, so an unexpanded FP node fails on the method that
  produced it and not later on the target.

1. Build the native runtime for a reduced ISA

• -march and -mabi become the cmake cache variables CLR_CMAKE_RISCV64_MARCH
  and CLR_CMAKE_RISCV64_MABI. Their defaults are today's values, so an
  unconfigured build does not change. The cross toolchain file applies the same
  variables to cmake's own probes, which otherwise fail to link against an lp64
  sysroot.
• The bundled libunwind builds with __riscv_flen == 0. There are no FP
  registers to unwind, so dwarf_getfp/dwarf_putfp return -UNW_EBADREG.
• The hand-written .S sources guard their F/D and A sequences on the
  __riscv_flen and __riscv_atomic predefined macros, which are what -march
  sets. The assembler is the consumer here, so it cannot use the InstructionSet
  model. With the extensions present, the same instructions are emitted.
• The hwprobe assertion in cpufeatures.c checks what the build requires, not the
  full rv64gc baseline.
• The build detects A from the single-letter part of the ISA string, with g
  implying it, and links libatomic only when A is absent.

2. Target ABI

TargetAbi.NativeAotRiscV64SoftFloat, spelled riscv64-lp64 on the ilc command
line. It is selected rather than inferred, following armel. Deriving it from the
instruction sets would be wrong: the ABI of the native objects we link against is
not a function of what our own codegen happens to emit.

The ABI drives four things:

• CORJIT_FLAG_SOFTFP_ABI for the JIT — the existing armel flag, now also
  meaningful on riscv64;
• the RISC-V float-ABI field in the ELF e_flags: EF_RISCV_FLOAT_ABI_SOFT
  instead of the unconditional EF_RISCV_FLOAT_ABI_DOUBLE. This is what lets the
  linker reject a mix of lp64 and lp64d objects. In the same place, the
  EF_RISCV_RVC bit follows the C instruction set instead of being always set;
• the instruction-set defaults: no F/D for the soft target;
• a two-way check in RyuJitCompilation: lp64d requires F and D, lp64
  must not have them.

crossgen2 rejects riscv64-lp64 right after GetTargetSpec, because the new
helpers have no ReadyToRun encoding.

For dotnet publish, the property IlcRiscV64SoftFloat=true selects
riscv64-lp64. The RID cannot imply it, because linux-musl-riscv64 covers both
lp64d and lp64 userspaces; this differs from linux-bionic on ARM, where
armel follows from the RID. The same property passes -mabi=lp64 to the native
link and to the compile of the aggregate-executable shim.

3. JIT/EE: 14 helpers

CORINFO_HELP_{FLT,DBL}{ADD,SUB,MUL,DIV}, CORINFO_HELP_{FLT,DBL}CMP_{LE,GE},
CORINFO_HELP_FLT2DBL, CORINFO_HELP_DBL2FLT.

The compare helpers are three-way. They differ only for unordered operands: LE
returns a positive value and GE a negative one. This is the __le*f2/__ge*f2
convention; compiler-rt returns 1/-1 and libgcc's soft-fp 2/-2, and the
expansion only tests the sign. As a result, each of the ten comparison forms that
IL can express is one call plus one integer compare against zero:

oeq: LE == 0    une: LE != 0
olt: LE <  0    ult: GE <  0
ole: LE <= 0    ule: GE <= 0
ogt: GE >  0    ugt: LE >  0
oge: GE >= 0    uge: LE >= 0

The JIT's IR has two more forms that IL cannot express: ueq (EQ with
GTF_RELOP_NAN_UN) and one (ordered NE). They need both helpers:

ueq = (LE >= 0) && (GE <= 0)        one = (LE < 0) || (GE > 0)

Conversions between FP and integers reuse the existing
DBL2LNG/DBL2ULNG/*_OVF/LNG2*/ULNG2* helpers. The non-overflow 32-bit
results are saturated from the 64-bit ones. ilc binds the new helpers to the
compiler-rt symbols (__addsf3, __adddf3, __lesf2, __gedf2,
__extendsfdf2, __truncdfsf2, …). All 14 are isNoGC, in the
CORINFO_HELP_LLSH group.

Value numbering models them as the operations they implement: VNFunc(GT_ADD)
and friends, as CORINFO_HELP_LMUL is modelled on 32-bit targets, and the
existing cast model for FLT2DBL/DBL2FLT. Only the two three-way compares need
VNFuncs of their own.

Adding helpers changes the JIT/EE contract, so jiteeversionguid.h is bumped per
docs/project/updating-jitinterface.md. As for FLTREM/DBLREM, each helper also
gets a ReadyToRunHelper entry, which is how ilc names a helper internally. The
values are appended at the end of the enum, and since crossgen2 rejects this
target no R2R image can contain them.

On the count

The JIT/EE interface has no way to name an arbitrary native symbol as a call
target. A helper call is a CORINFO_HELP_* constant resolved through
getHelperFtn, which is JitHelper.GetEntryPoint in ilc. None of the existing FP
helpers covers arithmetic, comparison or float↔double; the set is int↔FP
conversions plus FLTREM/DBLREM. The mechanism is not new:
CORINFO_HELP_DBLREM → ReadyToRunHelper.DblRem → fmod is already a JIT helper
bound to a C library symbol. These are fourteen more instances of it.

With fourteen, every FP operation that IL can express is exactly one call. The
count can be reduced:

Three things cannot be reduced. float arithmetic must not be done in double,
because double rounding changes results. DIV has no correctly-rounded reciprocal
substitute. FLT2DBL/DBL2FLT have no existing helper.

Two compare helpers per type is the minimum. With only __le*f2 you can express
oeq, une, olt, ole, ugt and uge. The other four IL forms would need a
second call to __unord*f2: still two entries per type, and two calls for each of
those forms.

4. Code generation: FP types in integer registers

The IR keeps TYP_FLOAT/TYP_DOUBLE. What changes under compUseSoftFP is the
register class of those two types: varTypeRegister[TYP_FLOAT/TYP_DOUBLE] becomes
VTR_INT.

Almost the whole backend already decides by register class rather than by type:
ins_Load/Store/Copy, inst_Mov, genHomeRegisterParams, genPutArgReg, LSRA's
regType, ReturnTypeDesc::GetABIReturnReg. So FP values are allocated, spilled,
moved, passed and returned in integer registers with no further work. The places
that decide by varTypeIsFloating need adjusting, and there are few:

• riscv64-only code: 4-byte integer load/store selection, FP constant
  materialization (the IEEE 754 bit pattern into a GPR), LSRA's per-type register
  sets and float returns, varTypeUsesFloatArgReg (which on riscv64 now follows
  varTypeUsesFloatReg, as on arm64), and BITCAST(int <- float), which must
  sign-extend because the psABI leaves the upper bits of a float in an integer
  register undefined;
• shared code (importer.cpp, flowgraph.cpp, lclvars.cpp, lsrabuild.cpp):
  the checks that mean "an FP value in an FP register" become
  varTypeIsFloating(x) && varTypeUsesFloatReg(x). On every existing target this
  is exactly varTypeIsFloating(x);
• shared code (lower.cpp, FIELD_LIST lowering): the float-to-int bitcast
  condition also accepts FP-typed values that already live in an integer
  register. The existing condition is kept unchanged next to it.

The operations are expanded during global morph through the existing hooks, so no
FP node reaches the backend:

One implementation note. USE_HELPER_FOR_ARITH and fgMorphCastIntoHelper turn
the node into a call in place, but the importer allocates only
mul/div/rem/conv as large nodes; add/sub and impImplicitR4orR8Cast
are small. The soft-float paths therefore build a new call node instead of
ChangeOper(GT_CALL).

varTypeRegister is a process-wide table, so one process cannot compile both
hard-float and soft-float code. The first compilation claims the configuration and
publishes it. A later compilation that requests the other mode fails with
NO_WAY, as GlobalJitOptions::compFeatureHfa does on armel. ilc compiles one
target per process. Open question 2 asks whether this is the form you want.

5. Atomics without A

genLockedInstructions and genCodeForCmpXchg emit amo* and lr/sc
unconditionally today, so a target without A gets instructions it cannot
execute on any Interlocked call. Both are gated on InstructionSet_A. Without
it, Interlocked lowers to a plain read/modify/write, and CmpXchg to a compare
and store with no reservation pair. BuildNode in lsrariscv64 extends the
address and data lifetimes for that sequence and gives the arithmetic and bitwise
forms one scratch register. The runtime's .S sources use the same lowering
(§1).

This is the only lowering the ISA allows. It is correct when the target runs on
one hart and is never preempted. Neither the ISA nor the ABI says anything about
that: a soft-float lp64 image is normal on rv64imac hardware with threads. So
the property has to be stated, not inferred. ilc gains --assume-no-concurrency,
whose name states both conditions, and it rejects an instruction set without A
unless that option is passed. Open question 6 asks whether this is the form you
want.

6. Runtime

In src/coreclr/nativeaot/Runtime/MathHelpers.cpp, RhpDbl2Lng and RhpDbl2ULng
spell out the .NET conversion semantics (NaN to 0, saturation) instead of relying
on the C cast. The cast has those semantics only where the hardware instruction
does; a soft-float build lowers it to a compiler-rt routine that has neither. The
explicit form gives the same results on arm64 and loongarch64, so it is
unconditional.

The same file also compiles the existing RhpLng2Dbl, RhpULng2Dbl, RhpLng2Flt
and RhpULng2Flt on riscv64, because the soft-float conversions from integers
call them. Today they are built only for 32-bit hosts. This is the one new host
condition (!defined(HOST_64BIT) || defined(HOST_RISCV64)). A hard-float riscv64
build compiles them but never calls them.

There is no new managed code.

Usage

ilc @rsp --targetos linux --targetarch riscv64-lp64 ...

$ readelf -h app.o | grep Flags
  Flags: 0x1                       # EF_RISCV_FLOAT_ABI_SOFT, RVC

The target sets the ABI, not the ISA. riscv64-lp64 has no F/D by
definition, but still defaults to C and A, like the hard-float target. A
target that also lacks those removes them explicitly. Removing A requires
--assume-no-concurrency (§5):

ilc @rsp --targetos linux --targetarch riscv64-lp64 \
    --instruction-set=-a,-c --assume-no-concurrency ...

$ readelf -h app.o | grep Flags
  Flags: 0x0                       # EF_RISCV_FLOAT_ABI_SOFT, no RVC

Without --assume-no-concurrency the same command is rejected with:

Building without the A extension requires --assume-no-concurrency (one hart, no
preemption): Interlocked operations are not atomic without it.

--targetarch riscv64 does not change and still means lp64d.

$ crossgen2 --targetarch riscv64-lp64 ...
error: Target architecture 'riscv64-lp64' is not supported by ReadyToRun

The native runtime for such a target is configured with:

build.sh --arch riscv64 --os linux-musl \
    -cmakeargs "-DCLR_CMAKE_RISCV64_MABI=lp64" \
    -cmakeargs "-DCLR_CMAKE_RISCV64_MARCH=rv64im"

Size

#134456 against main, by part:

Testing

In #134456:

• src/tests/JIT/Directed/softfloat (new) has bit-exact expectations for
  arithmetic rounding, NaN comparison semantics including the unordered branch
  forms, saturating and checked conversions for every integer width, conversions
  from every integer width, float ↔ double, negation, remainder, the Math
  intrinsics that become calls under this target, CSE and value numbering of
  repeated helper calls, FP values across call boundaries including stack-passed
  arguments, and float-bits-as-int sign extension. Every expected value is the
  exact IEEE 754 result, so the test is valid on every target and passes on
  hard-float. It is currently a standalone priority-1 project and is not yet in
  the JIT/Directed merged test groups, so upstream CI does not run it. We will
  wire it in at the priority you prefer.
• ILCompiler.ReadyToRun.Tests checks that crossgen2 rejects riscv64-lp64.
• CI on [RISC-V] NativeAOT soft-float target (riscv64-lp64) and builds without the A extension #134456 builds every existing leg, including
  linux-riscv64 Debug CoreCLR_Bootstrapped. That shows the series builds and
  does not break the existing targets. It does not execute the soft path.

There is no unit test for the e_flags computation yet. It had one in
ILCompiler.Compiler.Tests, but that project was removed in #133474, and
ILCompiler.TypeSystem.Tests does not reference ILCompiler.Compiler, where
ElfObjectWriter lives. Open question 7 asks where such a test belongs.

Running the soft path needs an lp64 sysroot: runtime, libc and compiler-rt built
with -mabi=lp64. This repo's CI does not have one, and it belongs to the
environment-specific layer that stays outside this repo.

Outside this repo, we have run the same soft-float code generation for riscv64
with an lp64 sysroot. SoftFloat.cs was NativeAOT-compiled for the soft-float
ABI and run on rv64gc under qemu-user (the compiler-rt builtins are integer
code, so a soft-float image runs on any RISC-V machine), with e_flags and the
absence of F/D opcodes checked. Alongside it ran 19 generated test programs,
about 7k assertions against goldens from an independent implementation, and a
real guest application validated byte-for-byte against its reference fixtures
on a zkVM emulator. All 21 programs pass under a checked JIT, and the guest
matches all 8 reference pairs.

Not yet executed anywhere: the A-less lowering of §5 and the exact commit
sequence of #134456. The ABI and helper commits were rebased onto the instruction
set work of §0, and that version still has to go through the validation above. We
will report the results here.

Compatibility

• corinfo.h and jithelpers.h are the internal JIT/EE contract, and the GUID
  is bumped. No public API surface changes.
• New code paths are behind TargetAbi.NativeAotRiscV64SoftFloat,
  opts.compUseSoftFP or InstructionSet_A, and the cmake variables default to
  today's values. None of these is set in any configuration that exists today.
  The shared-code checks in §4 are written so that they equal the existing checks
  on every existing target. The MathHelpers.cpp change gives the same results on
  the platforms that ship.
• The helpers resolve to compiler-rt/libgcc, which NativeAOT already links
  against. A toolchain without them fails at link time, not at run time.
• An lp64 image linked against lp64d native objects would break P/Invoke with
  double arguments. Emitting the correct e_flags is what makes the linker
  reject that combination, which is why the ELF writer change is part of the
  proposal.

Alternatives considered

• A fork. The ABI classifier and the register-class model are internal RyuJIT
  decisions, so a fork has to be rebased against every change to morph, LSRA and
  lowering. Nothing else about the environment needs the runtime's cooperation.
• Software FP in the runtime or CoreLib. A large body of correctness-critical
  IEEE 754 code and #if paths in managed code, to reimplement what the C
  toolchain already ships.
• A separate var_types for soft FP, or TYP_INT in the importer. Spreads
  through the importer, VN, assertion prop and inlining, and loses constant
  folding and the FP type information the rest of the JIT relies on. Changing the
  register class of the existing types touches less and keeps VN and CSE working.
• A dedicated lowering pass after morph. Duplicates the cast-expansion and
  constant-folding logic that fgMorphExpandCast and gtFoldExpr already
  implement, and produces nodes that morph never sees.
• Inferring the ABI from the instruction sets. The ABI of the objects being
  linked is not derivable from our own codegen's ISA usage.

Open questions

1. Is an explicit soft-float ABI for riscv64 acceptable as a NativeAOT-only target
   (no VM, no R2R, no interpreter), and is riscv64-lp64 the spelling you want?
2. Is the process-global register-class table the right form, following armel, or
   should the register class be per-Compiler, with the corresponding cost in the
   places that read varTypeRegister?
3. Is fourteen the right granularity? The table in §3 gives the two ways down to
   twelve or ten and what each costs.
4. PerfMapAbiToken has no value for the new ABI and is currently mapped to
   Default. Is a new token wanted?
5. What validation in this repo would be useful, given that an lp64 sysroot
   cannot live here? At minimum we can wire SoftFloat.cs into the merged
   JIT/Directed tests as a hard-float baseline.
6. Is --assume-no-concurrency the right way to state "one hart, no preemption"? The
   compiler cannot verify it. The alternative is a distinct target rather than a
   flag.
7. Where should a unit test for ElfObjectWriter output such as e_flags live,
   now that ILCompiler.Compiler.Tests is gone?

Plan

We understand that a new target is a long-term cost for the repo. If the
direction is agreed, we will do the work of splitting and landing this, and we
will keep the target working afterwards: fixing breaks in it, keeping its tests
passing and answering issues filed against it. We are open to whatever form of
support you think is appropriate. #134456 would be split into PRs in this order:

1. §0 instruction sets and emitter gating;
2. §1 native build and assembly for a reduced ISA;
3. §5 atomics without A and --assume-no-concurrency;
4. §2–3 ABI, helpers and publish integration;
5. §4 and §6 code generation, runtime and the test.

cc @dotnet/jit-contrib @dotnet/ilc-contrib @JulieLeeMSFT @agocke @jakobbotsch
@jkotas @MichalStrehovsky @tomeksowi @am11 @tannergooding