add doc regarding that grads are pointing to bucket views in DDP#43282
add doc regarding that grads are pointing to bucket views in DDP#43282zhaojuanmao wants to merge 1 commit intogh/zhaojuanmao/50/basefrom
Conversation
add doc regarding that grads are pointing to bucket views in DDP Differential Revision: [D23220110](https://our.internmc.facebook.com/intern/diff/D23220110/) [ghstack-poisoned]
add doc regarding that grads are pointing to bucket views in DDP Differential Revision: [D23220110](https://our.internmc.facebook.com/intern/diff/D23220110/) ghstack-source-id: 110266907 Pull Request resolved: #43282
💊 CI failures summary and remediationsAs of commit 5c3e819 (more details on the Dr. CI page): ✅ None of the CI failures appear to be your fault 💚
🚧 1 fixed upstream failure:These were probably caused by upstream breakages that were already fixed.
Please rebase on the
|
| reduction functions might not get called. | ||
|
|
||
| .. warning:: | ||
| Please noted that gradients are views of buffer tensors maintained by |
|
|
||
| .. warning:: | ||
| Please noted that gradients are views of buffer tensors maintained by | ||
| DistributedDataParallel reducer since PyTorch 1.7. This can help save |
There was a problem hiding this comment.
double quote all code text, i.e., DistributedDataParallel . Also applies to detach_(), Optimizer.zero_grad(), and torch/optim/optimizer.py.
| please refer to the Optimizer.zero_grad() function in | ||
| torch/optim/optimizer.py as the solution. |
There was a problem hiding this comment.
can we convert this to a link? e.g.
please refer to the ``zero_grad()`` function in :class:`torch.optim.Optimizer` as a solution.
There was a problem hiding this comment.
I am not 100% sure the above would work. Please confirm that by building the docs.
|
it is not needed any more |
3 participants
Stack from ghstack:
add doc regarding that grads are pointing to bucket views in DDP
Differential Revision: D23220110