🚀 Feature
import ignite.distibuted as idist
with idist.rank_process_first(rank=1):
data_setup_code(...)
where behind we would be doing something like:
if idist.get_local_rank() != rank:
idist.barrier()
data_setup_code(...)
if idist.get_local_rank() == rank:
idist.barrier()