Repository navigation
Commit 822d047
Fix bool mask handling in 1-pass SDPA decode kernel. Small reproducer:
```
import torch
import torch.nn.functional as F
q = torch.randn(1, 1, 8, 64, device="mps")
k = torch.randn(1, 1, 512, 64, device="mps")
v = torch.randn(1, 1, 512, 64, device="mps")
mask = torch.ones(8, 512, dtype=torch.bool, device="mps")
mps_op = torch.ops.aten._scaled_dot_product_attention_math_for_mps
out, _ = mps_op(q, k, v, mask)
ref = F.scaled_dot_product_attention(q.cpu(), k.cpu(), v.cpu(), attn_mask=mask.cpu())
print("max |mps - cpu| =", (out.cpu() - ref).abs().max().item())
```
Pull Request resolved: #182285
Approved by: https://github.com/malfet
(cherry picked from commit f6c73b4)
Co-authored-by: Isalia20 <[email protected]>
1 parent 5c5e523 commit 822d047
2 files changed
Lines changed: 19 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
37 | | - | |
38 | | - | |
39 | | - | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
40 | 40 | | |
41 | 41 | | |
42 | 42 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9953 | 9953 | | |
9954 | 9954 | | |
9955 | 9955 | | |
| 9956 | + | |
| 9957 | + | |
| 9958 | + | |
| 9959 | + | |
| 9960 | + | |
| 9961 | + | |
| 9962 | + | |
| 9963 | + | |
| 9964 | + | |
| 9965 | + | |
| 9966 | + | |
| 9967 | + | |
| 9968 | + | |
| 9969 | + | |
| 9970 | + | |
| 9971 | + | |
9956 | 9972 | | |
9957 | 9973 | | |
9958 | 9974 | | |
| |||
0 commit comments