-
Notifications
You must be signed in to change notification settings - Fork 26.3k
Open
Labels
oncall: distributedAdd this issue/PR to distributed oncall triage queueAdd this issue/PR to distributed oncall triage queuetriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate moduleThis issue has been looked at a team member, and triaged and prioritized into an appropriate module
Description
After #20910, data parallel implementation still needs the following improvements:
- Fix C++ data parallel for BN
- Make sure C++ data parallel work for double backward
- Move
reduce_addandreduce_add_coalescedintorch/cuda/comm.pyto C++. - Move
BroadcastandReduceAddCoalescedfromtorch/nn/parallel/_functions.pyto C++. - Make C++ data parallel use
ReduceAddCoalesced. - Consolidate C++ and Python module replicate.
Metadata
Metadata
Assignees
Labels
oncall: distributedAdd this issue/PR to distributed oncall triage queueAdd this issue/PR to distributed oncall triage queuetriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate moduleThis issue has been looked at a team member, and triaged and prioritized into an appropriate module