Is your feature request related to a problem? Please describe.
Now we already have some distributed training test cases, like: https://github.com/Project-MONAI/MONAI/blob/master/tests/test_handler_rocauc_dist.py
Need to be executed in our CI system.
We can run with 2 GPUs in 1 node first.