ggml-cpu : select the NEON fp32->fp16 conversion - #30505

ggml-cpu: select the NEON fp32->fp16 conversion

The __ARM_NEON branch of simd-mappings.h defines both GGML_CPU_COMPUTE_FP16_TO_FP32 and
GGML_CPU_COMPUTE_FP32_TO_FP16, but selects only the first. The second therefore falls through to
GGML_COMPUTE_FP32_TO_FP16 at the end of that section -- ggml-base's portable software bit
manipulation, the one ggml.c uses everywhere because ggml-base may not assume CPU features --
while neon_compute_fp32_to_fp16, a single fcvt, stays defined and referenced by nothing. The
GGML_CPU_* pair exists to be the accelerated variant; on ARM it is defined to be the
unaccelerated one.

__POWER9_VECTOR__ and __riscv_zfhmin select both macros. The comment on that fallback already
said NEON does too:

so we define GGML_CPU_FP16_TO_FP32 and GGML_CPU_FP32_TO_FP16 elsewhere for NEON. This is also
true for POWER9.

Same shape of omission #30270 fixed for F16C, in the other direction. Not a regression -- the
NEON selection is absent in all 25 most recent commits touching the file, back to 2025-04-10.

Why it is felt on ARM and nowhere else: ggml_cpu_fp32_to_fp16 -- the from_float of
GGML_TYPE_F16 -- has vector paths for F16C, RISC-V and s390x (the last added in #30157) and
none for NEON, so there the whole function is the scalar loop through the macro.

It is in the released binaries

Disassembling ggml_cpu_fp32_to_fp16 and searching for any fcvt of any form finds none in
llama-b11576-bin-ubuntu-arm64 (all 8 CPU variants) or llama-b11576-bin-macos-arm64, nor in the
17 ARM artifacts a downstream project ships across Linux, Android, Windows-on-ARM and macOS -- 26
binaries, zero conversion instructions. In the same object file the other direction carries 9 of
them, which is the asymmetry in one picture.

The x86 side is the comparison: of this project's 14 x86-64 CPU variant modules, the 11 with F16C
emit 5-6 vcvtps2ph each and the three without (x64, sse42, sandybridge) emit none -- exactly
matching the hardware. Only ARM has the instruction everywhere and uses it nowhere.

What it costs

In the shipped libggml-cpu-armv8.0_1.so the loop is 25 instructions per element, with a
six-operation serial float dependency chain (fabs, fmul, fmov, fmadd, fmov) and a
data-dependent branch for the inf/NaN case. With the selection it is a load, one fcvt and a store,
which the compiler then vectorises to fcvtn v0.4h, v0.4s -- four elements per instruction.

Measured on five ARM core types across Linux, macOS and FreeBSD: a 16M-element bulk conversion, best
of 7, with the opposite direction measured in the same binary and the same run as a control the
change cannot touch.

The factor narrows on wider cores and never vanishes -- 2.45x to 13.3x -- across every ARM
server core generation from 2018 to Graviton 4, an in-order little core, Apple Silicon, and three
operating systems. In all eighteen runs the control direction stays within 3.2 %, against an effect
of 2.45x and up; on the Cortex-A55 it is identical to the digit.

It is bit-exact

Both paths applied to every one of the 2^32 float32 bit patterns: 0 differences for every
finite value, zero, denormal and infinity. They differ only on NaN inputs (16 744 448), where the
software encoder emits a canonical quiet NaN and the instruction propagates the truncated payload.
The identical comparison on x86 with F16C gives the same two numbers, so this brings ARM into
agreement with the behaviour x86 already ships rather than splitting them. The 16M-element output
checksum is byte-identical on all five core types and all nine compiler builds, and the exhaustive
run was repeated on three architectures (Ampere eMAG, Graviton 4, x86-64) with identical counts.

It does not make inference faster

perf on an F16 pp512 run attributes 83.9 % of cycles to tinyBLAS::gemm, which converts
inside its own microkernel, and 0.03 % to ggml_cpu_fp32_to_fp16. End to end the change is
+0.13 % against a run-to-run spread of 0.09 %, i.e. nothing; quantized models never reach the
function at all, since their activations go through a different from_float. This is a correctness
and consistency fix, and the 3x-13x belongs to the function, not to a model.

Also in this change: the guard

Two of the four branches in that chain had this defect at the same time -- F16C until #30270, NEON
until this PR -- and neither was caught by a build, a warning or a test, because the fallback
silently supplies a correct software implementation. The only symptom is a hardware instruction that
is never used. The #error pair checks exactly that state.

The FP16->FP32 half carries a && !defined(__F16C__) carve-out, with the issue link on the
condition line. #30343 landed while this PR was open and puts that direction back on the lookup
table on purpose, so without the carve-out the guard would fire on every x86 F16C build. The
FP32->FP16 half needs no exception: F16C, NEON (with this change), POWER9 and riscv_zfhmin all
select it. If #30342 is ever resolved and the F16C selection comes back, the carve-out becomes dead
weight and should be deleted -- which is what the link on that line is for.

Measured against current master with clang -E:

So 4 of 4 correctness cases are caught on every architecture without the documented exception;
on x86 with F16C the FP16->FP32 direction is unchecked by construction, which is exactly the state
#30343 intends and nothing beyond it.

Its position -- immediately before the fallback rather than directly after the chain -- was chosen
by mutation, not by taste, and #30343 sharpened the argument: it leaves the F16C selection below
the lookup-table declarations, i.e. after the end of the chain, so the same guard placed directly
after the chain now fails every x86 F16C build. Before the fallback it both compiles and catches
a COMPUTE macro defined after the chain -- the same silent-fallback class this PR is about -- which
a guard directly after the chain would miss.

Also in this change: the fallback comment

The comment above the fallback singled out NEON and POWER9 and pointed at a selection that, after
this change, happens in the branch itself. It now states the fallback plainly, per direction
rather than per branch, because since #30343 the F16C branch selects one of the two and "when no
accelerated branch selected" would no longer be accurate.

Build and test

Built in the configuration used for the ARM releases -- -DGGML_BACKEND_DL=ON -DGGML_CPU_ALL_VARIANTS=ON -DGGML_NATIVE=OFF, clang 21.1.8 -- i.e. all eight ARM CPU variant
modules, each compiling this header with a different feature set (dotprod, fp16, SVE, i8mm, SVE2,
SME). All eight build, the guard fires nowhere in the whole build, and ggml_cpu_fp32_to_fp16
contains fcvt, fcvtn and fcvtn2 -- six occurrences -- in every one of them. Reverting the
patch in the same build tree and rebuilding puts that count back to 0.

ctest: 52 of 54 pass. The two failures, test-tokenizers-ggml-vocabs and test-jinja-py, are
the same two on unpatched master in that same tree -- they need test vocabularies and a Python
jinja2 the machine does not have. The suite is unchanged by the patch.

That build was made before the rebase. Both ggml_cpu_fp32_to_fp16 and the __ARM_NEON branch of
this header are byte-identical between that base and current master, so the result still describes
the tree as it stands.

Testing summary

• clang -E resolution matrix over five configurations (x86 +F16C, x86 +F16C+AVX512,
  x86 -F16C, x86 plain, NEON) for master, master+guard and the final tree.
• Mutation of the guard: four ways the invariant can be violated, two ways it can only look
  violated, at both candidate positions.
• Full build with all 8 ARM CPU variants plus ctest, and the revert round trip in the same tree.
• Microbenchmark on five ARM core types and nine compiler builds, with an in-run control.
• llama-bench (f16 and Q4_K_M, interleaved, -r 5) plus perf record.
• Exhaustive 2^32 equivalence, on ARM and on x86.
• Disassembly of 26 released ARM binaries.

Not claimed

• No inference speed-up, as measured above.
• Five ARM core types were measured (Ampere eMAG, Cortex-A55, Neoverse-N1, Neoverse-V2, Apple M1)
  on Linux, macOS and FreeBSD with nine compiler builds. Neoverse N2 and Apple M2 and later are
  untested, which is why the factor is given as a range.
• MSVC on Windows-on-ARM is untested as a compiler, although the ggml-cpu.dll built with it shows
  the same absence of fcvt.
• No explicit NEON vector path is added to ggml_cpu_fp32_to_fp16; see the thread.

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: AI-assisted (Claude Code Opus 5.5), manually reviewed and manually committed. I take full responsibility for the changes.