-
Notifications
You must be signed in to change notification settings - Fork 26.3k
Open
Labels
featureA request for a proper, new feature.A request for a proper, new feature.oncall: distributedAdd this issue/PR to distributed oncall triage queueAdd this issue/PR to distributed oncall triage queuetriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate moduleThis issue has been looked at a team member, and triaged and prioritized into an appropriate module
Description
In #22036 we added sparse allreduce for ProcessGroupGloo. It works for sparse CUDA tensors, but doesn't leverage InfiniBand like NCCL does. Therefore, we should have a sparse allreduce implementation for ProcessGroupNCCL as well.
NProkoptsev, powderluv, GeneDer, LithiumH, oriondollar and 5 morewillwilliams
Metadata
Metadata
Assignees
Labels
featureA request for a proper, new feature.A request for a proper, new feature.oncall: distributedAdd this issue/PR to distributed oncall triage queueAdd this issue/PR to distributed oncall triage queuetriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate moduleThis issue has been looked at a team member, and triaged and prioritized into an appropriate module