Repository navigation
Commit d91f231
PR #41808: [XLA:GPU] Migrate CollectivePermute command buffer support to thunk
Imported from GitHub PR openxla/xla#41808
PR description:
📝 Summary of Changes
Migrates CollectivePermute command-buffer support from `CollectivePermuteCmd` to `CollectivePermuteThunk` by appending the thunk directly as a command. Removes the old `CollectivePermuteCmd` implementation and command type. Adds command-buffer create/update coverage and real NCCL multi-GPU execution tests for `CollectivePermuteThunk`.
🎯 Justification
This continues the migration of GPU collective command-buffer handling into thunks, reducing duplicated command wrapper logic and aligning CollectivePermute with the thunk-owned command-buffer path. Workloads using CollectivePermute under GPU command buffers benefit from exercising the same thunk implementation for eager execution and command-buffer recording.
🚀 Kind of Contribution
🧪 Tests
♻️ Cleanup
📊 Benchmark (for Performance Improvements)
N/A. This PR is not a performance improvement.
🧪 Unit Tests:
Updated `//xla/backends/gpu/runtime:collective_permute_thunk_test` to cover CollectivePermute thunk command-buffer create/update behavior and source-target remapping logic.
🧪 Execution Tests:
Added `//xla/backends/gpu/runtime:collective_permute_thunk_multigpu_test`, covering real NCCL CollectivePermute execution on 2 GPUs through:
- `ExecuteOnStream`
- command-buffer create mode with warm-up
- command-buffer update mode
Verified with:
`bazel test //xla/backends/gpu/runtime:collective_permute_thunk_test //xla/backends/gpu/runtime:collective_permute_thunk_multigpu_test`
Copybara import of the project:
--
7e5aa8f1d0ba7764836dbf8aaf6a811e9a8baaf5 by Shawn Wang <[email protected]>:
[xla:gpu] Migrate CollectivePermuteCmd to thunk command
--
2a81fcd0df75baaf6b855e4700a1afb4bbb9799d by Shawn Wang <[email protected]>:
[xla:gpu] Add CollectivePermuteThunk multigpu tests
--
3091a945c2cdc7001f6fd082c03643d3c41d072e by Shawn Wang <[email protected]>:
Fix clang-format for collective permute test
--
2d16ad745025e77b116716c6129b8bfc84dccfb4 by Shawn Wang <[email protected]>:
Refactor CollectivePermuteThunk multi-GPU test
--
e49a9476cf4eee52eafc0f444b95c2cf2f7ceac5 by Shawn Wang <[email protected]>:
Fix clang-format
--
97e7f54d3d75a572d802e2e6cefd82f6264dc6c1 by Shawn Wang <[email protected]>:
Fix clang-tidy issues in collective-permute tests
--
51a14bd549ab8282e78d11881615f2b83af6c989 by Shawn Wang <[email protected]>:
Fix ASSERT_OK include-cleaner issue in multigpu test
--
546b99468761decd0d4c1408d9cc15690fa617ed by Shawn Wang <[email protected]>:
Fix collective permute multigpu lambda return types
Merging this change closes #41808
PiperOrigin-RevId: 9176136481 parent 7963804 commit d91f231
8 files changed
Lines changed: 553 additions & 138 deletions
File tree
- third_party/xla/xla/backends/gpu/runtime
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
221 | 221 | | |
222 | 222 | | |
223 | 223 | | |
224 | | - | |
225 | 224 | | |
226 | 225 | | |
227 | 226 | | |
228 | 227 | | |
229 | | - | |
230 | 228 | | |
231 | 229 | | |
232 | 230 | | |
| |||
236 | 234 | | |
237 | 235 | | |
238 | 236 | | |
239 | | - | |
240 | | - | |
241 | 237 | | |
242 | 238 | | |
243 | 239 | | |
| |||
2691 | 2687 | | |
2692 | 2688 | | |
2693 | 2689 | | |
| 2690 | + | |
| 2691 | + | |
2694 | 2692 | | |
2695 | 2693 | | |
2696 | 2694 | | |
| 2695 | + | |
2697 | 2696 | | |
2698 | 2697 | | |
2699 | 2698 | | |
| |||
2702 | 2701 | | |
2703 | 2702 | | |
2704 | 2703 | | |
| 2704 | + | |
2705 | 2705 | | |
2706 | 2706 | | |
2707 | 2707 | | |
| |||
2713 | 2713 | | |
2714 | 2714 | | |
2715 | 2715 | | |
| 2716 | + | |
| 2717 | + | |
| 2718 | + | |
| 2719 | + | |
2716 | 2720 | | |
2717 | 2721 | | |
| 2722 | + | |
| 2723 | + | |
2718 | 2724 | | |
| 2725 | + | |
| 2726 | + | |
2719 | 2727 | | |
| 2728 | + | |
2720 | 2729 | | |
| 2730 | + | |
2721 | 2731 | | |
2722 | | - | |
| 2732 | + | |
2723 | 2733 | | |
2724 | 2734 | | |
2725 | 2735 | | |
2726 | 2736 | | |
2727 | 2737 | | |
| 2738 | + | |
| 2739 | + | |
| 2740 | + | |
2728 | 2741 | | |
2729 | 2742 | | |
2730 | 2743 | | |
2731 | 2744 | | |
2732 | 2745 | | |
| 2746 | + | |
| 2747 | + | |
| 2748 | + | |
| 2749 | + | |
| 2750 | + | |
| 2751 | + | |
| 2752 | + | |
| 2753 | + | |
| 2754 | + | |
| 2755 | + | |
| 2756 | + | |
| 2757 | + | |
| 2758 | + | |
| 2759 | + | |
| 2760 | + | |
| 2761 | + | |
| 2762 | + | |
| 2763 | + | |
| 2764 | + | |
| 2765 | + | |
| 2766 | + | |
| 2767 | + | |
| 2768 | + | |
| 2769 | + | |
| 2770 | + | |
| 2771 | + | |
| 2772 | + | |
| 2773 | + | |
| 2774 | + | |
| 2775 | + | |
2733 | 2776 | | |
2734 | 2777 | | |
2735 | 2778 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
| 177 | + | |
| 178 | + | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
0 commit comments