🐛 Bug

initializedContextIds_ field in DistEngine grows without bound

Expected behavior

We should cleanup entries for this map as autograd context ids are cleaned up. Or we should have a field in DistAutogradContext instead.

Additional context

https://github.com/pytorch/pytorch/blob/master/torch/csrc/distributed/autograd/engine/dist_engine.h#L86

cc @ezyang @gchanan @zou3519 @jerryzh168 @pietern @mrshenli @pritamdamania87 @zhaojuanmao @satgera @gqchen @aazzolini @rohan-varma @xush6528