Summary

• Implements FP16 MatMul using ARM SME2 instructions

• SME2 kernel is only selected if:

  • The compiler supports SME2
  • The CPU supports SME2 (runtime check)
  • M >= 4 (minimum SME2 tile size)

• Adds a separate compilation target for SME2

  • SME2 requires architecture-specific flags (e.g. -march=armv9.2-a+sme2), different from the normal build
  • To avoid double-compilation and impacting global build flags, all SME2 code is isolated to kernel_sme2.cpp
  • For future kernels, an "SME2 version" of each kernel file can be created, and all SME2 files can be added to the compilation target

• Introduces CMake option ENABLE_SME2

  • AUTO (default): build SME2 only if supported by the compiler, else build normally
  • OFF: Force-disable SME2, build normally

• Existing AMX/Neon implementations remain unchanged and serve as the fallback paths

Contributes to #299

Benchmark (M4, run with cactus test --precision FP16)

NEON vs SME2

AMX vs SME2

Further Work

• There is most likely not much room for further optimization for this kernel
  • I can do some experimentation with prefetching and preprocessing, but gains would be minimal
• Next steps would be to accelerate other kernels to leverage SME2, such as INT8/INT4