Skip to content

perf(ops): lower dynamic_slice to MLX's dynamic slice primitive - #220

Merged
tillahoffmann merged 2 commits into
tillahoffmann:mainfrom
michaelellis003:fix/dynamic-slice-view
Jul 18, 2026
Merged

tillahoffmann merged 2 commits into
tillahoffmann:mainfrom
michaelellis003:fix/dynamic-slice-view

Conversation

@michaelellis003

Copy link
Copy Markdown
Contributor

Complements #219 (the dynamic_update_slice write side; context in #215). Independent change — only HandleDynamicSlice is touched — so it can land before or after #219 with at most a trivial rebase on slice.cc.

Problem

stablehlo.dynamic_slice lowers to one take plus index arithmetic per axis. Inside while-loop bodies this runs every iteration (a scan consuming xs reads one row per step), and the gather materializes a copy of each slice.

Fix

Lower to MLX's dynamic-start slice(a, start, axes, slice_size): one lazy primitive per call, StableHLO's per-dim start clamping preserved, and eval produces a view of the slice instead of a gather copy.

Measurements (M3 Pro, 36 GB)

A scan reading 4 MB rows (T=200, N=1e6) drops 99 → 44 ms. Dispatch-bound tiny rows (T=1000, N=256) are unchanged. The #215 stacked-scan MWE is unaffected by this PR alone (it has no reads); combined with #219 it keeps #219's numbers.

Testing

Full test suite on Metal: 2519 passed, 383 skipped, 30 xfailed. Values and grads through scans consuming xs match CPU references.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves the performance of stablehlo.dynamic_slice lowering by replacing the existing per-axis take chain (with index arithmetic) with MLX’s native dynamic-start slice(a, start, axes, slice_size) primitive, which is especially beneficial inside while-loop/scan bodies.

Changes:

  • Reworked HandleDynamicSlice to build a dynamic start vector and call mlx::core::slice directly.
  • Preserved StableHLO start-index clamping semantics while switching to a single MLX primitive.

Comment on lines +107 to +112
const int size = static_cast<int>(sliceSizes[static_cast<unsigned>(d)]);
const int dimSize = input->shape(d);
// Clamp start per StableHLO spec: max(0, min(start, dim_size - size)).
auto start =
mlx::core::clip(mlx::core::astype(mlx::core::reshape(*idx, {1}), mlx::core::int32),
mlx::core::array(0), mlx::core::array(dimSize - size));
outSize.push_back(size);
}
values.emplace(ToKey(op->getResult(0)), std::move(result));
auto startArr = startParts.size() == 1 ? startParts[0] : mlx::core::concatenate(startParts, 0);
michaelellis003 and others added 2 commits July 18, 2026 12:06
stablehlo.dynamic_slice lowered to one take plus index arithmetic per
axis. Lower it to MLX's slice(a, start, axes, slice_size) instead: one
lazy primitive whose eval is a view of the slice, with StableHLO start
clamping preserved per dim. Complements tillahoffmann#219 (the dynamic_update_slice
write side): a scan consuming large xs rows drops 2x (96 -> 44 ms for
T=200 reads of 4 MB rows on an M3 Pro) because the per-iteration row
read becomes a view instead of a gather copy. Full suite: 2519 passed,
383 skipped, 30 xfailed on Metal.
mlx::core::slice takes `start` as `const array&`, so std::move on the
argument is a no-op and trips clang-tidy's performance-move-const-arg
(warnings-as-errors). Pass startArr directly; keep std::move on the
by-value axes/slice_size arguments.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
@tillahoffmann
tillahoffmann force-pushed the fix/dynamic-slice-view branch from 3ecf034 to f9a9677 Compare July 18, 2026 11:18
@tillahoffmann
tillahoffmann merged commit 7b3dc36 into tillahoffmann:main Jul 18, 2026
14 checks passed
tillahoffmann added a commit that referenced this pull request Jul 18, 2026
…e ops (#222)

Extract the per-axis start-index clamp shared by dynamic_slice (#220) and dynamic_update_slice (#219) into a single BuildClampedStarts helper. Behavior-preserving; also guards the clamp upper bound with std::max(0, ...) and moves the single-axis start array.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants