-
Notifications
You must be signed in to change notification settings - Fork 26.3k
Closed
Labels
high prioritymodule: ciRelated to continuous integrationRelated to continuous integrationtriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate moduleThis issue has been looked at a team member, and triaged and prioritized into an appropriate module
Description
This has to be the weirdest CI failure I have seen. On #22043 , a PR filed today, one of the CI test run fails with
Jun 21 00:31:06 ======================================================================
Jun 21 00:31:06 ERROR: test_clip_grad_norm (__main__.TestNN)
Jun 21 00:31:06 ----------------------------------------------------------------------
Jun 21 00:31:06 Traceback (most recent call last):
Jun 21 00:31:06 File "test_nn.py", line 1830, in test_clip_grad_norm
Jun 21 00:31:06 norm = clip_grad_norm_(l.parameters(), max_norm, norm_type=norm_type)
Jun 21 00:31:06 File "/opt/conda/lib/python3.6/site-packages/torch/nn/utils/clip_grad.py", line 36, in clip_grad_norm_
Jun 21 00:31:06 clip_coef = torch.tensor(max_norm, device=device) / (total_norm + 1e-6)
Jun 21 00:31:06 UnboundLocalError: local variable 'device' referenced before assignment
Jun 21 00:31:06
The PR doesn't touch this line. And this line of code printed in error trace
clip_coef = torch.tensor(max_norm, device=device) / (total_norm + 1e-6
does not even match what the line is on master, which is
pytorch/torch/nn/utils/clip_grad.py
Line 35 in d4119f8
| clip_coef = max_norm / (total_norm + 1e-6) |
and is 2 years old.
The latest update to that file
clip_grad.py is on March 1st, and doesn't even touch this line.
Here is link to the build: https://circleci.com/gh/pytorch/pytorch/2043106?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link
Hmm, commit hash collision?
Metadata
Metadata
Assignees
Labels
high prioritymodule: ciRelated to continuous integrationRelated to continuous integrationtriagedThis issue has been looked at a team member, and triaged and prioritized into an appropriate moduleThis issue has been looked at a team member, and triaged and prioritized into an appropriate module