load_state_dict creates reference cycle, is there a python garbage collect operation before raise CUDA out-of-memory error? #20199
Description
🐛 Bug
Python's garbage collection invokes every 700 object creations (by default), so there should be a situation when the object is not referred but still occupies the CUDA memory (when reference cycles happen), so I'm asking for an additional garbage collecting operation if the CUDA memory is out.