Repository navigation
NoneType has no attribute to #76
Description
I get the following error when I run my training. When I comment out this line the training works but the loss doesn't decrease
File "/storage/home/ec2-user/ner/trainer/trainer.py", line 131, in _train_epoch
self.model.step()
File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/deepspeed/pt/deepspeed_light.py", line 628, in step
fp32_param.grad = fp16_param.grad.to(fp32_param.dtype)
AttributeError: 'NoneType' object has no attribute 'to'
self.optimizer.step()
File "/home/ec2-user/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/deepspeed/pt/fp16_unfused_optimizer.py", line 165, in step
fp32_param.grad = fp16_param.grad.to(fp32_param.dtype)
AttributeError: 'NoneType' object has no attribute 'to'
Activity
Tejaswgupta commentedon Feb 14, 2020
I would like to contribute
jeffra commentedon Feb 15, 2020
Hi @Tejaswgupta, thanks for reaching out and wanting to contribute. Is there something specific you have in mind you'd like to contribute? If it was this issue I do have a PR being reviewed currently that fixes this issue. We're in the process of trying to create some new issues that would be good to take care of as a new user. Keep an eye out for those :)
@dapurv5, Thanks for bringing this bug to our attention. We previously fixed this type of bug in a different code path but now have added unit tests for it and fixed it in your code path and one other. The PR should be merged soon.
Tejaswgupta commentedon Feb 17, 2020
Sure @jeffra :)
Updated code based on name change from "sparse transformer" to "spars…
[runtime] add pin_memory(tensor) to accelerator runtime support (deep…