Is your feature request related to a problem? Please describe.
Currently, we have 2 enhanced all_gather() APIs for distributed data parallel, they are based on ignite APIs.
As they can be also used to sync data in regular PyTorch programs for metrics, classification report, etc. we need to adjust them based on PyTorch dist APIs.
And also need to fix the ignite 1024 length issue when all_gather strings.