Reduce steady-state allocations in inference and training hot paths #7684

A source-level audit found several avoidable managed allocations in ML.NET inference, training, and shared data-pipeline hot paths. These occur per row, example, class, or training iteration rather than only during model or pipeline initialisation.

The existing performance suite does not adequately detect these regressions:

• RecommendedConfig does not enable BenchmarkDotNet's MemoryDiagnoser.
• BenchmarksTest checks only that benchmarks build and execute, not allocation thresholds.
• The batch inference benchmarks at StochasticDualCoordinateAscentClassifierBench.cs:166-172 call the lazy Transform method without enumerating its output, so they do not measure scoring.

Describe the solution you'd like

Add allocation-focused benchmarks and remove the highest-frequency allocations while preserving numerical behaviour and existing concurrency guarantees.

• Enable allocation reporting for relevant performance benchmarks.
• Add separate inference benchmarks for allocating and reusable PredictionEngine overloads, including dense and sparse vector outputs.
• Add fixed-shape ONNX and TensorFlow per-row inference benchmarks.
• Add controlled binary and multiclass SDCA training benchmarks.
• Ensure batch inference benchmarks enumerate the resulting IDataView.
• Cache TensorFlow runners, input/output names and shape arrays where their lifetimes permit.
• Reuse TensorFlow and ONNX fixed-size input buffers rather than copying them into new managed arrays per row.
• Replace the SDCA visitor delegate with an atomic operation on the dual-table abstraction.
• Replace hot internal VBuffer.Items() calls with span/index-based loops or an allocation-free enumerator.
• Reuse the OVA softmax scratch buffer and avoid Parallel.For below a measured class-count threshold.
• Reuse same-sized sparse prediction arrays through VBuffer.CopyTo(Span<T>).
• Construct n-gram weighting delegates once per getter rather than once per row.
• Investigate pooling the lightweight cursor-consolidation wrappers.
• Replace per-row Convert.ChangeType calls with converters selected during getter construction.

Acceptance criteria:

• Allocation and throughput results are recorded before and after each change.
• Fixed-shape ONNX and TensorFlow inference no longer copies full input vectors into new managed arrays on every row where backend lifetime rules permit reuse.
• SDCA performs no delegate or closure allocation per dual update.
• Targeted FFM, FastTree and prediction materialisation paths do not allocate iterator objects per row.
• Reused sparse prediction outputs do not allocate a replacement array when their length is unchanged.
• Numerical output and deterministic single-threaded training behaviour remain unchanged.
• Buffers remain scoped per mapper, cursor, or prediction engine where required for thread safety.

Describe alternatives you've considered

Relying on garbage collection or GC tuning would only mitigate the symptoms and would not remove allocation volume from the hottest loops.

Advising callers to use PredictionEngine.Predict(..., ref prediction) is useful, but it addresses only the result-object allocation. It does not fix tensor copies, sparse output arrays, multiclass scoring temporaries, or training allocations.

Global shared buffer pools were also considered, but per-engine or per-cursor reuse is safer because PredictionEngine and several mapper caches are intentionally not thread-safe.

Additional context

ML.NET already has suitable allocation-conscious patterns, particularly VBufferEditor.Create(ref destination, ...), reusable getter buffers, and preallocated trainer scratch arrays. The proposed changes should extend those existing patterns rather than introduce broadly shared mutable state.

This issue is based on static source inspection. Runtime profiling and allocation baselines should be added before implementation to quantify each finding and prioritise the work.