Skip to content

Commit 80c5ae2

Browse files
committed
[PyTorch] Use 128-token splits in the tied te.ops GroupedLinear test
te.ops.GroupedLinear now validates on device that every split is a multiple of split_alignment (default 128), so 16-token splits trap in the forward pass. Signed-off-by: Alp Dener <[email protected]>
1 parent 6a4839a commit 80c5ae2

1 file changed

Lines changed: 2 additions & 1 deletion

File tree

‎tests/pytorch/test_tied_delayed_wgrad.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -279,7 +279,8 @@ def test_tied_grouped_linear_ops_gradients(set_to_none, backward_dw_order, packe
279279
pytest.skip("packed grouped weights require a supported GPU and cuBLASLt")
280280
monkeypatch.setenv("NVTE_GROUPED_LINEAR_SINGLE_PARAM", "1" if packed else "0")
281281
torch.manual_seed(9)
282-
num_groups, hidden_size, tokens_per_group, microbatches = 2, 16, 16, 2
282+
# Split sizes must be multiples of te.ops.GroupedLinear's default split_alignment (128).
283+
num_groups, hidden_size, tokens_per_group, microbatches = 2, 16, 128, 2
283284
delayed = nn.ModuleList(
284285
[
285286
te.ops.GroupedLinear(

0 commit comments

Comments
 (0)