Skip to content

Commit 137a670

Browse files
Fix Tensor correctness and accelerate contiguous operations (dotnet#134914)
## Summary - Fix Tensor shape, storage, broadcasting, overlap, resize, and enumeration correctness; cover the behavior with regression tests and document intentional differences from NumPy. Empty shapes use zero automatically computed strides so unreachable products cannot overflow before the zero-length dimension. - Dispatch elementwise Tensor operations to TensorPrimitives for dense layouts and contiguous trailing slices, including compatible outer-dimension broadcasts. Preserve indexed traversal for incompatible layouts and ordering-sensitive reductions. - Accelerate strided flattening, equality, non-dense random fills, and non-dense Resize without changing logical iteration order. - Correct vector tangent range reduction discovered by CI: use guaranteed fused multiply-add on FMA hardware and an input-dependent scalar fallback for cancellation-sensitive vectors on other hardware, retaining vector evaluation for safe inputs. ## Bug reports and fuzzing findings - `ReadOnlyTensorSpan<T>` now rejects incompatible `System.Array` element types (dotnet#133459), and `Tensor<T>` slice pinning starts at the slice offset (dotnet#134691). - `ResizeTo` fills unwritten logical destination elements with `default(T)` and rejects ambiguous overlapping destination layouts; the behavior is documented (dotnet#128555). - Cropped `TensorSpan<T>.FlattenTo` now copies contiguous rows rather than individual elements, addressing the reported regression (dotnet#121463); the exact issue repro is measured below. - This PR addresses the `Tensor` findings [3, 6, 7, 8, 9, and 13 in the community SharpFuzz report](https://github.com/Mrnikbobjeff/runtime/blob/combined-sharpfuzz-net11-findings/src/libraries/Fuzzing/Findings/FINDINGS.md): strided `ToString`, broadcast pairwise comparisons, empty `*Any` comparisons, preserving the element when squeezing all singleton dimensions, `ResizeTo` zero-fill, and constructor/slice exception contracts. Squeezing retains a rank-one `[1]` shape rather than NumPy's rank-zero shape; this intentional difference is documented in the library README. Findings 4, 5, 10, 11, and 12 concern `TensorPrimitives` and are not addressed here. - **Partially addresses dotnet#125663:** contiguous non-dense slices avoid per-element copying in several operations, but the broader non-dense performance and allocation work remains open. ## Measurements Local BenchmarkDotNet on Windows x64 (Ryzen 9 7950X, .NET 11). Representative results below compare the prior implementation to this branch except where noted: | Scenario | Prior | Current | | --- | ---: | ---: | | Dense Add (64x16) | 12.69 us | 0.121 us | | Gapped-row Add (64x16) | 11.58 us | 2.29 us | | Broadcast-row Add (64x16) | 12.58 us | 1.91 us | | Transposed Add (64x16) | 12.54 us | 11.60 us | | Gapped-row FlattenTo | 5.66 us | 0.79 us | | Gapped-row SequenceEqual | 6.41 us | 0.57 us | | dotnet#121463 cropped FlattenTo repro (3x512x512 bytes) | 2.030 ms | 19.71 us | The dotnet#121463 row uses the issue's original 3x2000x2048 input cropped to 3x512x512, run locally against a saved pre-optimization assembly and this PR's Release assembly on the same .NET 11 host (approximately 103x faster). Additional local comparisons against equivalent indexed loops: non-dense Resize 1,909 -> 289 ns; uniform fill 5.56 -> 2.41 us; Gaussian fill 13.97 -> 10.88 us. Unary float Sin versus equivalent scalar iteration: dense 2.88 -> 0.63 us, gapped rows 8.41 -> 3.15 us. No additional managed allocation was measured in these samples. The transposed case is not addressed by the trailing-slice optimization. Following review, sliced elementwise dispatch is gated centrally at 32 elements, or 16 for tensor/tensor binary operations. Against the previously published PR head, the threshold follow-up improved four-element gapped Add from 237 to 135 ns and Abs from 134 to 68 ns; sixteen-element Add measured 235 to 243 ns, and retained sliced paths had small differences in both directions. Normal-path pool returns are retained without exceptional-path `finally` cleanup. Removing the reinstated `finally` blocks changed permutation codegen: rank-three permutation measured about 45 to 59 ns in the focused comparison, with unchanged allocations. These follow-up changes are not claimed as universally performance-neutral. The empty-stride correction was also compared locally with the preceding PR head across 12 dense-construction and reshape cases (ranks 1, 3, and 6, empty and nonempty). Allocation counts were unchanged; throughput measurements were noisy across repeated runs, so no performance improvement is claimed for this correctness fix. CI exposed tangent range-reduction errors in the newly activated vector path. Without fused multiply-add, `Tan(-32.986717f)` returned about -148998.98 instead of the scalar/reference result -177349.88. The correction is in `TensorPrimitives.Tan`, preserving Tensor's vector dispatch and existing tangent forwarding tolerances. FMA hardware uses guaranteed fused reduction rather than `MultiplyAddEstimate`, whose fused behavior is not guaranteed across runtimes. Other hardware retains non-fused vector evaluation when `abs(f) >= dn / 256`; otherwise the affected vector uses the existing scalar fallback. The threshold scales with the reduction count and has a documented rounding-error bound. With FMA unavailable, dense 4x1024 float Tan measured 4.31 -> 4.70 us on random inputs in [-1, 1] and 4.42 -> 8.08 us on [-50, 50], comparing the original vector implementation with the accuracy correction. Against a local global-FMA-gate implementation that scalarizes every non-FMA input, the corresponding results were 11.03 -> 4.70 us and 13.73 -> 8.08 us. Allocation counts remain unchanged. On FMA hardware, float and double kernels at all three widths retain the original instruction counts and code sizes; register allocation differs. FMA throughput measurements varied with code placement, so no precise performance-neutrality claim is made. ## Validation - System.Numerics.Tensors.Tests: 6,100 net11.0 and 200 net481 tests passed in each of five local Windows x64 configurations: default intrinsics, FMA-enabled 256-bit vectors, FMA-enabled 128-bit vectors, non-FMA 128-bit vectors, and intrinsics disabled. ARM64/Mono and Linux CI have not yet validated the tangent correction. - System.Numerics.Tensors Release analyzer build succeeded with zero warnings or errors. - Direct float/double vector-operator probes covered 1,966,080 checks per FMA configuration across all three widths, small and large random inputs, broad exponents, and homogeneous cancellation-boundary vectors. No tolerance failures occurred with or without FMA; non-FMA 256/512-bit operators were exercised through their software vector implementations on this host. - Independent deterministic layout oracle: 28,049 checks passed across 1,500 cases covering sliced, permuted, gapped, broadcast, and empty layouts. Resolves dotnet#128555 Resolves dotnet#133459 Resolves dotnet#134691 Resolves dotnet#121463 > [!NOTE] > This pull request description was generated by GitHub Copilot. --------- Co-authored-by: Copilot App <[email protected]>
1 parent e5e565b commit 137a670

21 files changed

Lines changed: 3365 additions & 526 deletions
Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,34 @@
11
# System.Numerics.Tensors
22

33
Provides APIs for performing primitive operations over tensors represented by spans of memory.
4+
5+
Some shape and storage behavior intentionally differs from NumPy:
6+
7+
- Creating a tensor from an empty shape (`[]`) produces shape `[0]`, with no elements. In NumPy,
8+
shape `()` has rank zero but contains one scalar element; `np.empty(())` leaves that element
9+
uninitialized ("empty" refers to initialization, not the number of elements).
10+
- Squeezing shape `[1, 1]` produces shape `[1]`, rather than NumPy's rank-zero shape `()`.
11+
Both contain one element and can broadcast as a scalar; code that inspects rank, selects
12+
axes, or indexes the result must account for the retained dimension.
13+
- Strides must be nonnegative. NumPy can represent a reversed view with a negative stride
14+
(for example, `array[::-1]`); use `Tensor.ReverseDimension` with dimension `0` to
15+
reverse the first axis instead. This produces a new tensor, not a reversed view.
16+
- When growing a tensor, `Tensor.Resize` and `Tensor.ResizeTo` fill new elements with
17+
`default(T)`. Resizing `[1, 2]` to five `int` elements yields `[1, 2, 0, 0, 0]`,
18+
whereas `np.resize` repeats the input and yields `[1, 2, 1, 2, 1]`.
19+
- `Tensor.ResizeTo` and concatenation into an existing destination reject a zero-stride
20+
dimension with more than one logical element: multiple output indexes would refer to the
21+
same storage and could not hold distinct values. Zero strides in singleton dimensions
22+
and empty destinations do not have this conflict.
23+
24+
When strides are omitted, a shape containing a zero-length dimension has zero strides
25+
in every dimension. Its element count and storage requirement are zero regardless of
26+
the other dimension lengths. Negative lengths remain invalid, and explicitly supplied
27+
strides must still satisfy the normal layout validation.
28+
29+
Overlapping sources and destinations are supported for equal-length dense copies, which
30+
use the same overlap-safe behavior as `Span<T>.CopyTo`, and for elementwise operations on identical
31+
non-broadcast views. An in-place reversal of a dense tensor also needs no temporary
32+
storage. Other overlapping tensor layouts throw `ArgumentException` before writing:
33+
copying them correctly could require buffering an amount of data proportional to the
34+
tensor's size. Nonoverlapping strided copies do not create such a buffer.

‎src/libraries/System.Numerics.Tensors/src/Resources/Strings.resx‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -129,6 +129,9 @@
129129
<data name="Argument_InputAndDestinationSpanMustNotOverlap" xml:space="preserve">
130130
<value>The destination span may only overlap with an input span if the two spans start at the same memory location.</value>
131131
</data>
132+
<data name="Argument_OverlappingTensorLayoutsNotSupported" xml:space="preserve">
133+
<value>The input and destination tensor spans overlap in an unsupported layout.</value>
134+
</data>
132135
<data name="Argument_DestinationSpansMustNotOverlap" xml:space="preserve">
133136
<value>Destination spans must not overlap with each other.</value>
134137
</data>
@@ -138,6 +141,9 @@
138141
<data name="Argument_InvalidEnumValue" xml:space="preserve">
139142
<value>The value '{0}' is not valid for this usage of the type {1}.</value>
140143
</data>
144+
<data name="InvalidOperation_EnumerationNotPositioned" xml:space="preserve">
145+
<value>Enumeration has not started or has already finished.</value>
146+
</data>
141147
<data name="Argument_TypeContainsReferences" xml:space="preserve">
142148
<value>The type '{0}' is not supported because it contains references.</value>
143149
</data>
@@ -168,6 +174,9 @@
168174
<data name="ThrowArgument_DimensionsNotSame" xml:space="preserve">
169175
<value>Number of dimensions to slice does not equal the number of dimensions in the span</value>
170176
</data>
177+
<data name="ThrowArgument_DestinationHasOverlappingElements" xml:space="preserve">
178+
<value>The destination contains overlapping elements and cannot represent distinct output values.</value>
179+
</data>
171180
<data name="ThrowArgument_FilterTensorMustEqualTensorLength" xml:space="preserve">
172181
<value>The total length of the filter tensor must equal the length of the tensor to be filtered.</value>
173182
</data>

‎src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/IReadOnlyTensor_1.cs‎

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -49,14 +49,14 @@ public interface IReadOnlyTensor<TSelf, T> : IReadOnlyTensor
4949

5050
/// <summary>Copies the contents of the tensor into a destination tensor span.</summary>
5151
/// <param name="destination">The destination tensor span.</param>
52-
/// <exception cref="ArgumentException"><paramref name="destination" /> is shorter than the source tensor.</exception>
53-
/// <remarks>This method copies all of the source tensor to <paramref name="destination" /> even if they overlap.</remarks>
52+
/// <exception cref="ArgumentException"><paramref name="destination" /> is shorter than the source tensor, or the source and destination overlap in an unsupported layout.</exception>
53+
/// <remarks>Overlapping dense tensors with equal element counts and identical views are supported. Other overlapping layouts are rejected before copying.</remarks>
5454
void CopyTo(scoped in TensorSpan<T> destination);
5555

5656
/// <summary>Flattens the contents of the tensor into a destination span.</summary>
5757
/// <param name="destination">The destination span.</param>
58-
/// <exception cref="ArgumentException"><paramref name="destination" /> is shorter than the source tensor.</exception>
59-
/// <remarks>This method copies all of the source tensor to <paramref name="destination" /> even if they overlap.</remarks>
58+
/// <exception cref="ArgumentException"><paramref name="destination" /> is shorter than the source tensor, or the source and destination overlap in an unsupported layout.</exception>
59+
/// <remarks>Overlapping dense tensors are supported. Other overlapping layouts are rejected before copying.</remarks>
6060
void FlattenTo(scoped Span<T> destination);
6161

6262
/// <summary>Returns a span that can be used to access the flattened elements for a given dimension.</summary>
@@ -110,17 +110,19 @@ public interface IReadOnlyTensor<TSelf, T> : IReadOnlyTensor
110110
/// <summary>Attempts to copy the contents of this tensor into a destination tensor span and returns a value to indicate whether or not the operation succeeded.</summary>
111111
/// <param name="destination">The target of the copy operation.</param>
112112
/// <returns><see langword="true"/> if the copy operation succeeded; otherwise, <c>false</c>.</returns>
113+
/// <exception cref="ArgumentException">The source and <paramref name="destination" /> overlap in an unsupported layout.</exception>
113114
/// <remarks>
114-
/// <para>If the source and <paramref name="destination" /> overlap, the entirety of the source is handled as if it was copied to a temporary location before it is copied to <paramref name="destination" />.</para>
115+
/// <para>Overlapping dense tensors with equal element counts and identical views are supported. Other overlapping layouts throw <see cref="ArgumentException" /> before copying.</para>
115116
/// <para>If the <paramref name="destination" /> length is shorter than the source, no items are copied and the method returns <c>false</c>.</para>
116117
/// </remarks>
117118
bool TryCopyTo(scoped in TensorSpan<T> destination);
118119

119120
/// <summary>Attempts to flatten the contents of this tensor into a destination span and returns a value to indicate whether or not the operation succeeded.</summary>
120121
/// <param name="destination">The target of the copy operation.</param>
121122
/// <returns><see langword="true"/> if the copy operation succeeded; otherwise, <c>false</c>.</returns>
123+
/// <exception cref="ArgumentException">The source and <paramref name="destination" /> overlap in an unsupported layout.</exception>
122124
/// <remarks>
123-
/// <para>If the source and <paramref name="destination" /> overlap, the entirety of the source is handled as if it was flattened to a temporary location before it is copied to <paramref name="destination" />.</para>
125+
/// <para>Overlapping dense tensors are supported. Other overlapping layouts throw <see cref="ArgumentException" /> before copying.</para>
124126
/// <para>If the <paramref name="destination" /> length is shorter than the source, no items are copied and the method returns <c>false</c>.</para>
125127
/// </remarks>
126128
bool TryFlattenTo(scoped Span<T> destination);

‎src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/ReadOnlyTensorDimensionSpan_1.cs‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ internal ReadOnlyTensorDimensionSpan(ReadOnlyTensorSpan<T> tensor, int dimension
2525
dimension += 1;
2626

2727
_tensor = tensor;
28-
_length = TensorPrimitives.Product(tensor.Lengths[..dimension]);
28+
_length = TensorShape.GetProduct(tensor.Lengths[..dimension]);
2929
_dimension = dimension;
3030
_sliceShape = TensorShape.Create((dimension != tensor.Rank) ? tensor.Lengths[dimension..] : [1], tensor.Strides[dimension..], tensor.IsPinned);
3131
}

‎src/libraries/System.Numerics.Tensors/src/System/Numerics/Tensors/netcore/ReadOnlyTensorSpan_1.cs‎

Lines changed: 30 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -148,8 +148,11 @@ public ReadOnlyTensorSpan(ReadOnlySpan<T> span, scoped ReadOnlySpan<nint> length
148148
/// <para>Returns default when <paramref name="array"/> is null.</para>
149149
/// <para>The created tensor span has a single dimension that is the same length as <paramref name="array" />.</para>
150150
/// </remarks>
151+
/// <exception cref="ArrayTypeMismatchException">The type of <paramref name="array"/> is not compatible with an array of <typeparamref name="T"/>.</exception>
151152
public ReadOnlyTensorSpan(Array? array)
152153
{
154+
ThrowHelper.ThrowIfArrayTypeMismatch<T>(array, isReadOnly: true);
155+
153156
_shape = TensorShape.Create(array);
154157
_reference = ref (array is not null)
155158
? ref Unsafe.As<byte, T>(ref MemoryMarshal.GetArrayDataReference(array))
@@ -175,8 +178,11 @@ public ReadOnlyTensorSpan(Array? array)
175178
/// * <paramref name="strides" /> is not empty and contains an element that is negative.
176179
/// * <paramref name="strides" /> is not empty and contains an element that is zero in a non leading position.
177180
/// </exception>
181+
/// <exception cref="ArrayTypeMismatchException">The type of <paramref name="array"/> is not compatible with an array of <typeparamref name="T"/>.</exception>
178182
public ReadOnlyTensorSpan(Array? array, scoped ReadOnlySpan<int> start, scoped ReadOnlySpan<nint> lengths, scoped ReadOnlySpan<nint> strides)
179183
{
184+
ThrowHelper.ThrowIfArrayTypeMismatch<T>(array, isReadOnly: true);
185+
180186
_shape = TensorShape.Create(array, start, lengths, strides, out nint linearOffset);
181187
_reference = ref (array is not null)
182188
? ref Unsafe.Add(ref Unsafe.As<byte, T>(ref MemoryMarshal.GetArrayDataReference(array)), linearOffset)
@@ -402,7 +408,7 @@ public ReadOnlySpan<T> GetSpan(scoped ReadOnlySpan<NIndex> startIndexes, int len
402408
/// <inheritdoc cref="IReadOnlyTensor{TSelf, T}.Slice(ReadOnlySpan{nint})" />
403409
public ReadOnlyTensorSpan<T> Slice(params scoped ReadOnlySpan<nint> startIndexes)
404410
{
405-
TensorShape shape = _shape.Slice<TensorShape.GetOffsetAndLengthForNInt, nint>(startIndexes, out nint linearOffset);
411+
TensorShape shape = _shape.Slice<TensorShape.GetOffsetAndLengthForSlice, nint>(startIndexes, out nint linearOffset);
406412
return new ReadOnlyTensorSpan<T>(
407413
ref Unsafe.Add(ref _reference, linearOffset),
408414
shape
@@ -447,7 +453,8 @@ ref Unsafe.Add(ref _reference, linearOffset),
447453
/// <inheritdoc cref="IReadOnlyTensor{TSelf, T}.TryCopyTo(in TensorSpan{T})" />
448454
public bool TryCopyTo(scoped in TensorSpan<T> destination)
449455
{
450-
if (TensorShape.AreCompatible(destination._shape, _shape, false))
456+
if ((_shape.FlattenedLength <= destination.FlattenedLength) &&
457+
TensorShape.AreCompatible(destination._shape, _shape, false))
451458
{
452459
TensorOperation.Invoke<TensorOperation.CopyTo<T>, T, T>(this, destination);
453460
return true;
@@ -536,46 +543,55 @@ ReadOnlyTensorSpan<T> IReadOnlyTensor<ReadOnlyTensorSpan<T>, T>.ToDenseTensor()
536543
public ref struct Enumerator : IEnumerator<T>
537544
{
538545
private readonly ReadOnlyTensorSpan<T> _span;
539-
private readonly nint[] _indexes;
540546
private nint _linearOffset;
541547
private nint _itemsEnumerated;
548+
private bool _hasCurrent;
542549

543550
internal Enumerator(ReadOnlyTensorSpan<T> span)
544551
{
545552
_span = span;
546-
_indexes = new nint[span.Rank];
547-
548-
_indexes[^1] = -1;
549-
550-
_linearOffset = 0 - (!span.IsEmpty ? span.Strides[^1] : 0);
553+
_linearOffset = 0;
551554
_itemsEnumerated = 0;
555+
_hasCurrent = false;
552556
}
553557

554558
/// <summary>Gets the element at the current position of the enumerator.</summary>
555-
public readonly ref readonly T Current => ref Unsafe.Add(ref _span._reference, _linearOffset);
559+
public readonly ref readonly T Current
560+
{
561+
get
562+
{
563+
if (!_hasCurrent)
564+
{
565+
ThrowHelper.ThrowInvalidOperation_EnumerationNotPositioned();
566+
}
567+
return ref Unsafe.Add(ref _span._reference, _linearOffset);
568+
}
569+
}
556570

557571
/// <summary>Advances the enumerator to the next element of the tensor span.</summary>
558572
public bool MoveNext()
559573
{
560574
if (_itemsEnumerated == _span._shape.FlattenedLength)
561575
{
576+
_hasCurrent = false;
562577
return false;
563578
}
564579

565-
_linearOffset = _span._shape.AdjustToNextIndex(_span._shape, _linearOffset, _indexes);
580+
_linearOffset = _span.IsDense
581+
? _itemsEnumerated
582+
: _span._shape.GetLinearOffsetForDimension(_itemsEnumerated, _span.Rank);
566583

567584
_itemsEnumerated++;
585+
_hasCurrent = true;
568586
return true;
569587
}
570588

571589
/// <summary>Sets the enumerator to its initial position, which is before the first element in the tensor span.</summary>
572590
public void Reset()
573591
{
574-
Array.Clear(_indexes);
575-
_indexes[^1] = -1;
576-
577-
_linearOffset = 0 - (!_span.IsEmpty ? _span.Strides[^1] : 0);
592+
_linearOffset = 0;
578593
_itemsEnumerated = 0;
594+
_hasCurrent = false;
579595
}
580596

581597
//

0 commit comments

Comments
 (0)