You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Splitting the mixers' projections with the Neural Engine #690
On an M4 Pro, the FFN split from #260 leaves the ANE waiting while the GPU runs the mixers' input and output projections, which are most of the rest of each layer. I extended the split to those projections, calibrated per Mac the same way as the FFN. At 14K and 32K tokens it is 1.62× and 1.53× faster than main on Qwen3.8-27B (MLX 4-bit). Decode is unchanged, and upstream's regression check passes. I'm opening a PR with the details and would appreciate any feedback on the approach.
Activity