-
Notifications
You must be signed in to change notification settings - Fork 26.3k
Description
🐛 Describe the bug
Hi, all. I'm developing the replay to Execution Trace. Confusing about the 2 points below and not sure whether it is bug. Can you help confirm/explain? Thanks!
-
Consider
torch.distributed.batch_isend_irecv()is invoked.
Several send/recv will be invoked in coalesed range. startCoalesing bumps upseqCollective_, but notseqP2P_. Is it by design? -
Consider allgather is invoked with different size, where it will be coverted to multi
_broadcast_oop()in coalesed range, which is implemented by collective.
Compared with coalesed send/recv, each collective will always bumpseqCollective_and creatework. However, theworkis returned but not used.
However, coalesed p2p will not bumpseqP2P_and not createword. Why is the pattern different?


Versions
tot main
cc @XilunWu @H-Huang @awgu @kwen2501 @wanchaol @fegin @fduwjj @wz337 @wconstab @d4l3k @c-p-i-o
