Repository navigation
perf(ops): lower dynamic_slice to MLX's dynamic slice primitive - #220
Merged
tillahoffmann merged 2 commits intoJul 18, 2026
Merged
Conversation
michaelellis003
force-pushed
the
fix/dynamic-slice-view
branch
from
July 17, 2026 21:00
491b961 to
b599530
Compare
tillahoffmann
force-pushed
the
fix/dynamic-slice-view
branch
from
July 18, 2026 10:36
b599530 to
3ecf034
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR improves the performance of stablehlo.dynamic_slice lowering by replacing the existing per-axis take chain (with index arithmetic) with MLX’s native dynamic-start slice(a, start, axes, slice_size) primitive, which is especially beneficial inside while-loop/scan bodies.
Changes:
- Reworked
HandleDynamicSliceto build a dynamicstartvector and callmlx::core::slicedirectly. - Preserved StableHLO start-index clamping semantics while switching to a single MLX primitive.
Comment on lines
+107
to
+112
| const int size = static_cast<int>(sliceSizes[static_cast<unsigned>(d)]); | ||
| const int dimSize = input->shape(d); | ||
| // Clamp start per StableHLO spec: max(0, min(start, dim_size - size)). | ||
| auto start = | ||
| mlx::core::clip(mlx::core::astype(mlx::core::reshape(*idx, {1}), mlx::core::int32), | ||
| mlx::core::array(0), mlx::core::array(dimSize - size)); |
| outSize.push_back(size); | ||
| } | ||
| values.emplace(ToKey(op->getResult(0)), std::move(result)); | ||
| auto startArr = startParts.size() == 1 ? startParts[0] : mlx::core::concatenate(startParts, 0); |
stablehlo.dynamic_slice lowered to one take plus index arithmetic per axis. Lower it to MLX's slice(a, start, axes, slice_size) instead: one lazy primitive whose eval is a view of the slice, with StableHLO start clamping preserved per dim. Complements tillahoffmann#219 (the dynamic_update_slice write side): a scan consuming large xs rows drops 2x (96 -> 44 ms for T=200 reads of 4 MB rows on an M3 Pro) because the per-iteration row read becomes a view instead of a gather copy. Full suite: 2519 passed, 383 skipped, 30 xfailed on Metal.
mlx::core::slice takes `start` as `const array&`, so std::move on the argument is a no-op and trips clang-tidy's performance-move-const-arg (warnings-as-errors). Pass startArr directly; keep std::move on the by-value axes/slice_size arguments. Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
tillahoffmann
force-pushed
the
fix/dynamic-slice-view
branch
from
July 18, 2026 11:18
3ecf034 to
f9a9677
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Complements #219 (the
dynamic_update_slicewrite side; context in #215). Independent change — onlyHandleDynamicSliceis touched — so it can land before or after #219 with at most a trivial rebase onslice.cc.Problem
stablehlo.dynamic_slicelowers to onetakeplus index arithmetic per axis. Inside while-loop bodies this runs every iteration (a scan consumingxsreads one row per step), and the gather materializes a copy of each slice.Fix
Lower to MLX's dynamic-start
slice(a, start, axes, slice_size): one lazy primitive per call, StableHLO's per-dim start clamping preserved, and eval produces a view of the slice instead of a gather copy.Measurements (M3 Pro, 36 GB)
A scan reading 4 MB rows (T=200, N=1e6) drops 99 → 44 ms. Dispatch-bound tiny rows (T=1000, N=256) are unchanged. The #215 stacked-scan MWE is unaffected by this PR alone (it has no reads); combined with #219 it keeps #219's numbers.
Testing
Full test suite on Metal: 2519 passed, 383 skipped, 30 xfailed. Values and grads through scans consuming
xsmatch CPU references.