Is your feature request related to a problem? Please describe.
Currently, each rank is logging during multi-gpu training.

Describe the solution you'd like
Remove redundant logs.

Describe alternatives you've considered
n/a

Additional context
n/a