🚀 The feature, motivation and pitch
If we set NCCL_DEBUG=INFO and NCCL_DEBUG_SUBSYS=TUNING, we should be able to see different algorithm being used.
Alternatives
No response
Additional context
No response
cc @H-Huang @awgu @wanchaol @fegin @fduwjj @wz337 @wconstab @d4l3k @pragupta @msaroufim @dcci