Remove unsafe code from "Vectorization Guidelines" examples.

The vectorization guidelines heavily relies on unsafe code in its examples. Recently, we landed a few improvements in the JIT that allows to use safe code with SIMD without extra checks. It might not be enough for all shapes currently used with SIMD, but should be OK to stop promoting the unsafe code, e.g. the existing example from the guidelines:

int Sum(Span<int> buffer)
{
    Debug.Assert(Vector128.IsHardwareAccelerated && buffer.Length >= Vector128<int>.Count);
    Vector128<int> sum = Vector128<int>.Zero;
    ref int searchSpace = ref MemoryMarshal.GetReference(buffer);
    nuint elementOffset = 0;
    nuint oneVectorAwayFromEnd = (nuint)(buffer.Length - Vector128<int>.Count);
    for (; elementOffset <= oneVectorAwayFromEnd; elementOffset += (nuint)Vector128<int>.Count)
    {
        Vector128<int> loaded = Vector128.LoadUnsafe(ref searchSpace, elementOffset);
        sum += loaded;
    }
    int result = Vector128.Sum(sum);
    while (elementOffset < (nuint)buffer.Length)
    {
        result += buffer[(int)elementOffset];
        elementOffset++;
    }
    return result;
}

can be rewritten into 100% safe code:

int Sum(Span<int> data)
{
    Vector128<int> sum = default;
    while (data.Length >= Vector128<int>.Count)
    {
        sum += Vector128.Create(data);
        data = data.Slice(Vector128<int>.Count);
    }
    int result = Vector128.Sum(sum);
    foreach (var t in data)
        result += t;
    return result;
}

after #127117 was merged.

See the benchmark below. The results imply there is no perf difference between the safe and the unsafe versions.

@JulieLeeMSFT

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

@EgorBot

@EgorBot -arm -amd -commits main

using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
using System.Runtime.Intrinsics;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

BenchmarkSwitcher.FromAssembly(typeof(Bench).Assembly).Run(args);

public class Bench
{
    private int[] _data = null!;

    [Params(16, 64, 1024)]
    public int Length { get; set; }

    [GlobalSetup]
    public void Setup() => _data = Enumerable.Range(0, Length).ToArray();

    [Benchmark]
    public int Unsafe() => Sum_unsafe(_data);

    [Benchmark]
    public int Safe() => Sum_safe(_data);

    [MethodImpl(MethodImplOptions.NoInlining)]
    static int Sum_unsafe(Span<int> buffer)
    {
        Vector128<int> sum = Vector128<int>.Zero;
        ref int searchSpace = ref MemoryMarshal.GetReference(buffer);
        nuint elementOffset = 0;
        nuint oneVectorAwayFromEnd = (nuint)(buffer.Length - Vector128<int>.Count);
        for (; elementOffset <= oneVectorAwayFromEnd; elementOffset += (nuint)Vector128<int>.Count)
        {
            Vector128<int> loaded = Vector128.LoadUnsafe(ref searchSpace, elementOffset);
            sum += loaded;
        }
        int result = Vector128.Sum(sum);
        while (elementOffset < (nuint)buffer.Length)
        {
            result += buffer[(int)elementOffset];
            elementOffset++;
        }
        return result;
    }

    [MethodImpl(MethodImplOptions.NoInlining)]
    static int Sum_safe(Span<int> data)
    {
        Vector128<int> sum = default;
        while (data.Length >= Vector128<int>.Count)
        {
            sum += Vector128.Create(data);
            data = data.Slice(Vector128<int>.Count);
        }
        int result = Vector128.Sum(sum);
        foreach (var t in data)
            result += t;
        return result;
    }
}

@JulieLeeMSFT

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

@EgorBo

@EgorBo Tried to rewrite some internal code. I don't get the perf benefit of the safe way on .net11

    [MethodImpl(MethodImplOptions.NoInlining)]
    private static ulong DotUnsafeImpl(ReadOnlySpan<ulong> x, ReadOnlySpan<ulong> y)
    {
        ref ulong xr = ref MemoryMarshal.GetReference(x);
        ref ulong yr = ref MemoryMarshal.GetReference(y);
        int len = x.Length;
        int i = 0;
        ulong sum = 0UL;

        if (Vector256.IsHardwareAccelerated && len >= Vector256<ulong>.Count)
        {
            Vector256<ulong> acc = Vector256<ulong>.Zero;
            int upper = len - Vector256<ulong>.Count;
            for (; i <= upper; i += Vector256<ulong>.Count)
            {
                acc += Vector256.LoadUnsafe(ref xr, (nuint)i) * Vector256.LoadUnsafe(ref yr, (nuint)i);
            }

            sum += Vector256.Sum(acc);
        }
        else if (Vector128.IsHardwareAccelerated && len >= Vector128<ulong>.Count)
        {
            Vector128<ulong> acc = Vector128<ulong>.Zero;
            int upper = len - Vector128<ulong>.Count;
            for (; i <= upper; i += Vector128<ulong>.Count)
            {
                acc += Vector128.LoadUnsafe(ref xr, (nuint)i) * Vector128.LoadUnsafe(ref yr, (nuint)i);
            }

            sum += Vector128.Sum(acc);
        }

        for (; i < len; i++)
        {
            sum += Unsafe.Add(ref xr, i) * Unsafe.Add(ref yr, i);
        }

        return sum;
    }

    [MethodImpl(MethodImplOptions.NoInlining)]
    private static ulong DotSafeImpl(ReadOnlySpan<ulong> x, ReadOnlySpan<ulong> y)
    {
        int len = x.Length;
        int i = 0;
        ulong sum = 0UL;

        if (Vector256.IsHardwareAccelerated && len >= Vector256<ulong>.Count)
        {
            Vector256<ulong> acc = Vector256<ulong>.Zero;
            int upper = len - Vector256<ulong>.Count;
            for (; i <= upper; i += Vector256<ulong>.Count)
            {
                acc += Vector256.Create(x.Slice(i, Vector256<ulong>.Count)) * Vector256.Create(y.Slice(i, Vector256<ulong>.Count));
            }

            sum += Vector256.Sum(acc);
        }
        else if (Vector128.IsHardwareAccelerated && len >= Vector128<ulong>.Count)
        {
            Vector128<ulong> acc = Vector128<ulong>.Zero;
            int upper = len - Vector128<ulong>.Count;
            for (; i <= upper; i += Vector128<ulong>.Count)
            {
                acc += Vector128.Create(x.Slice(i, Vector128<ulong>.Count)) * Vector128.Create(y.Slice(i, Vector128<ulong>.Count));
            }

            sum += Vector128.Sum(acc);
        }

        for (; i < len; i++)
        {
            sum += x[i] * y[i];
        }

        return sum;
    }

G_M000_IG04:
       movsxd   r9, r8d
       vmovups  ymm1, ymmword ptr [rax+8*r9]
       vpmullq  ymm1, ymm1, ymmword ptr [rdx+8*r9]
       vpaddq   ymm0, ymm1, ymm0
       add      r8d, 4
       cmp      r8d, r10d
       jle      SHORT G_M000_IG04
; Total bytes of code 126

G_M000_IG04:
       mov      ebx, r9d
       lea      rsi, [rbx+0x04]
       cmp      rsi, r11              ; (i+4) > x.Length ?
       ja       G_M000_IG12          ;  -> ThrowArgumentOutOfRangeException  (x.Slice)
       shl      rbx, 3
       lea      rdi, bword ptr [r8+rbx]
       vmovups  ymm1, ymmword ptr [rdi]
       mov      edi, edx
       cmp      rsi, rdi              ; (i+4) > y.Length ?
       ja       G_M000_IG12          ;  -> ThrowArgumentOutOfRangeException  (y.Slice)
       add      rbx, rax
       vpmullq  ymm1, ymm1, ymmword ptr [rbx]
       vpaddq   ymm0, ymm1, ymm0
       add      r9d, 4
       lea      ebx, [r9+0x04]
       cmp      ebx, ecx
       jle      SHORT G_M000_IG04
; ... G_M000_IG12: call ThrowArgumentOutOfRangeException ; G_M000_IG13: call CORINFO_HELP_RNGCHKFAIL
; Total bytes of code 251

The guidelines (https://github.com/dotnet/runtime/blob/main/docs/coding-guidelines/vectorization-guidelines.md) have since been rewritten and now largely defer to the public docs (https://learn.microsoft.com/en-us/dotnet/standard/simd)

The examples all center around the safe patterns and require no unsafe. There is still an explicit callout (under https://learn.microsoft.com/en-us/dotnet/standard/simd#load-and-store-vectors-safely) that if code absolutely must use the unsafe APIs, there are some patterns that can help mitigate failures. But that most code can and should be using the span based APIs instead.

@unsafePtr

@unsafePtr, you'll want to follow what is more akin to the while loop pattern given. This also allows you to trivial handle the trailing data efficiently.

You are not prevented from using the for pattern you have, but the general flow and checks still need to generally match how the while pattern is setup.

@unsafePtr

Yes, it's being worked on still (#111309), I do plan to optimize shapes like for (; i <= upper; i += Vector256<ulong>.Count) in fact I already fixed it locally, but it didn't make it to .NET 11.0 unfortunately.

For now I recommend using

while (span.Length > CNS)
{
    // jit knows span is >= CNS long here

    span = span.Slice(CNS); // advance to the next chunk
}

@unsafePtr I've rewritten your safe impl and kicked off benchmarks here: EgorBot/Benchmarks#401
As you can see, it's already as fast as unsafe on mac.

Navigation Menu

Repository navigation

Remove unsafe code from "Vectorization Guidelines" examples. #127506

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Remove unsafe code from "Vectorization Guidelines" examples. #127506

Description

Activity

dotnet-policy-service commented on Apr 28, 2026

EgorBo commented on Apr 28, 2026

dotnet-policy-service commented on Apr 28, 2026

unsafePtr commented on Jul 23, 2026

tannergooding commented on Jul 23, 2026

tannergooding commented on Jul 23, 2026

EgorBo commented on Jul 23, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions